ML on small devices

Putting intelligence on chips small enough to be in disconcerting places

The art of doing ML stuff on small controllers, such as single board computers or even individual microcontrollers, or some weird chip embedded in some medical device. A.k.a. edge ml,1

I do not have much to say here RN.

Making neural models small

Obviously if your target model is a neural net one important step is making it be as small as possible in the sense of having as few parameters as possible.

Low precision nets

There is another sense of small: Using fixed point, or even single bit, arithmetic so that the numbers involved are compact. TBD.

Multi-task learning

Training one learning algorithm to solve several problems simultaneously. Probably needs its own page.


Tensorflow appears to have intimate microcontroller integration via TensorFlow Lite for Microcontrollers.

Other frameworks convert to intermediate format ONNX which can be run on microcontrollers, although I suspect with higher overhead.


Cai, Han, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. 2020. “Once-for-All: Train One Network and Specialize It for Efficient Deployment.” In.
Chen, Tianqi, Ian Goodfellow, and Jonathon Shlens. 2015. Net2Net: Accelerating Learning via Knowledge Transfer.” November 17, 2015.
Cheng, Yu, Duo Wang, Pan Zhou, and Tao Zhang. 2017. “A Survey of Model Compression and Acceleration for Deep Neural Networks.” October 23, 2017.
He, Yihui, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, and Song Han. 2019. AMC: AutoML for Model Compression and Acceleration on Mobile Devices.” January 15, 2019.
Howard, Andrew G., Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications.” April 16, 2017.
Shi, Lei, Shikun Feng, and ZhifanZhu. 2016. “Functional Hashing for Compressing Neural Networks.” May 20, 2016.
Wang, Yunhe, Chang Xu, Chao Xu, and Dacheng Tao. 2019. “Packing Convolutional Neural Networks in the Frequency Domain.” IEEE Transactions on Pattern Analysis and Machine Intelligence 41 (10): 2495–2510.
Warden, Pete, and Daniel Situnayake. 2020. TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers. O’Reilly Media, Incorporated.

  1. I do not like this term, because it tends to imply that we care especially about some kind of centre-edge distinction, which we only do sometimes. It tends to imply that large NN models in data centres are the default type of ML. Chris Mountford’s Hasn’t AI Been the Wrong Edgy for Too Long?, mentioned in the comments riffs on this harder than I imagined, though↩︎

1 comment

GitHub-flavored Markdown & a sane subset of HTML is supported.