Gerald Haslhofer 
Survey of ML on the edge
Case studies
NLP
- Smart reply Google Blog and research paper
- On-Device Conversational Modeling with TensorFlow Lite
- Reddit thread Run BERT on mobile device using TinyBERT
Vision
- Vision: MobileNet Open Sourced: https://ai.googleblog.com/2017/06/mobilenets-open-source-models-for.html
Frameworks
- ML Kit with Tensorflow Lite
- Smart-reply
[Sample] to try out (https://www.tensorflow.org/lite/models/smart_reply/overview)
- Smart-reply
- ONNX Runtime from Microsoft. V1 of runtime launched 10/2019. Works together with NN API for Android
- NNAPI: Android developer documentation
[Sample] for NNAPI (https://github.com/android/ndk-samples/tree/master/nn_sample)
- DNNLibrary is a wrapper of NNAPI. It lets you easily make the use of the new NNAPI introduced in Android 8.1. You can convert your onnx model into daq and run the model directly.
Sample https://github.com/daquexian/dnnlibrary-example
- DNNLibrary is a wrapper of NNAPI. It lets you easily make the use of the new NNAPI introduced in Android 8.1. You can convert your onnx model into daq and run the model directly.
- NNAPI: Android developer documentation
-
Caffe2 / PyTorch
AI camera Sample
- Unclear if Caffe2 uses NNAPI (see thread)
- PyTorch Mobile, also mentioned in
Generating smaller networks
- Neural projection networks. https://arxiv.org/pdf/1708.00630.pdf A joint learning framework based on neural projections to learn lightweight neural network models for performing efficient inference on device.
- Learn2Compress models: https://ai.googleblog.com/2018/05/custom-on-device-ml-models.html
