The TensorFlow dev summit concluded a few days ago. I enjoyed it much and to say it was beyond what I anticipated would be an understatement. There were many exciting announcements and I will try to cover some of them in this post.
tldr;
There were other announcements that this post does not cover such as Swift for TensorFlow, Reinforcement Learning with TF-Agents, TensorFlow Extended, TensorFlow Probability, Tensorboard, TensorFlow Hub, TF Lattice, Tensorflow.jl, Mesh-Tensorflow, and TensorFlow Federated.
TensorFlow 2.0 Preview
The availability of 2.0 Preview was the major announcement and having played with it in my personal projects IMHO it is by far the best TensorFlow version in terms of experimentation, debugging and user experience.
Website: https://www.tensorflow.org/alpha/
Issue Tracker: https://github.com/orgs/tensorflow/projects/4
Usability and API Cleanup
The big news is that tf.Session is going away and tf.keras, the high-level API, is the new way of building, training and exporting/saving models. The models can now be written using Sequential API (easiest way to get started)
Or by sub-classing the tf.keras.Model class (similar to PyTorch, Chainer)
Experimentation: The eager execution is enabled by default which is useful during prototyping/research phase where you just want to try things and debug as you go along. I prefer this approach than tf.Session as I can now stick import pdb; pdb.set_trace() and inspect the tensors. (Disclaimer: there is no one size fits all as some people like to create their nodes first and prune later).
Autograph: After the experimentation phase, users can make use of Autograph that compiles the code into an optimized graph using @tf.function decorator. Autograph also supports control flow in python so no more tf.cond and tf.while_loop.
Scalable Training: The keras API can be used for large scale multi-gpu training using tf.distribute strategies. The built-in strategies can handle distribution on devices of various architecture. The support for training across multiple nodes/machines is planned in the near future (available in nightly build) and TPU support will be available in next TensorFlow release.
Deployment: tf.keras models can be exported directly to SavedModel that works across the TensorFlow ecosystem (e.g TensorFlow serving, TensorFlow.js, TensorFlow Lite)
API Consolidation:
- Layers are consolidated under tf.keras.layers
- Optimizers are consolidated under tf.keras.optimizer
- Metrics are consolidated under tf.keras.metrics
- Losses are consolidated under tf.keras.losses
- Single LSTM layer: no more conditional checks for CudnnLSTM and LSTM
- Tensorboard consolidation via callbacks in model.fit
TensorFlow.js 1.0
TensorFlow.js (tfjs) is a javascript library for training and deploying machine learning models. It has no dependency on external libraries such as CUDA, useful in scenarios where user data should not leave their browser and makes the additional platform(Web, Mobile, Electron) available to TensorFlow ecosystem.
Website: https://www.tensorflow.org/js
TensorFlow.js comes with off-the-shelf models, users can also convert their python models to tjfs format or build and train models in browser or Node.js.
Build: The API to build model is compatible with Keras API which means the code to build a model looks very similar to the tf.keras sequential model.
The data pipeline code also looks similar to Python tf.data.
The model training can be visualized in the browser using tfjs-vis library or in tensorboard using callbacks in the model.fit (similar to Python).
Who is using TensorFlow.js: Uber uses TensorFlow.js in their model-agnostic visualization tool: Manifold, Clinic.js uses it to diagnose performance issues in Node.js apps and Magenta Studio uses it in Electron.js based desktop app: Ableton Live.
TensorFlow Lite
As the name suggests, TensorFlow Lite is a lightweight cross-platform solution for mobile and embedded devices. This means the same platform that supports high-end mobile phones with GPU, can also support low power embedded devices. That to me sounds like an incredible effort, kudos to the team and contributors.
Website: https://www.tensorflow.org/lite
There are 2B devices running TensorFlow lite in production. Google assistant switched from their in-house developed highly optimized models to TensorFlow Lite and they found it met or beat their existing libraries in performance. NetEase, China uses TensorFlow Lite in its translation app to accelerate on-device OCR.
The large part of the talk covered the product roadmap and features in development. The major themes included Usability, Performance, Optimization, and Documentation.
Usability: The model conversion tool allows users to easily convert their models saved in SavedModel format into TF Lite format. TF Lite supports limited ops and no control flow support (i.e loops and conds) but users can use TensorFlow Select to include other ops from TensorFlow at the cost of increased binary size. The pipeline work includes the support for control flow and selective registration i.e only select the ops that you want and it should lead to reduced binary size. They are also building a new converter tool that will identify model conversion issues and suggest possible fixes.
Performance: Edge TPU is incredibly performant (e.g 2ms MobileNet V1 inference compared to 16ms on GPU and 64ms on CPU/Quantized). The Model Benchmark tool now supports threading, per op profiling and support for Android NN API. The hardware-agnostic abstraction layer — “TF Lite delegate” lets the user focus on model and let delegate choose where to execute the graph operations e.g. on GPU or CPU. As of now, they have Edge-TPU delegate and GPU delegate. They are also working on Arm and x86 CPU optimizations.
Optimization: TF Lite supports post-training quantization that gives better performance on CPU. The planned pipeline work includes the support for easy to use Keras-based API to perform quantized training and connection-pruning that will result in further improvements to model compression and faster execution with little degradation in model accuracy.
Documentation: The new website includes better documentation and five ready to use models in the model repository. They are working on adding more tutorials and ready to use models.
Pete Warden gave a demo of TensorFlow Lite running on tiny SparkFun Edge DevBoard. Jun Tate-Gans introduced two hardware products: Coral Dev Board that features Edge TPU and runs on Linux and Android with 1GB RAM, onboard WIFI and Bluetooth. Coral USB Accelerator that brings Edge TPU to Raspberry Pi boards.
Exciting stuff, I cannot wait to try Edge TPUs especially after learning about the incredible performance boost over CPU and GPU.
Almost forgot, yes TF Lite also supports TensorFlow 2.0.
That’s it folks. Hope you found it useful. Watch the TensorFlow channel to learn more the dev summit.