LiteRT, Google’s “trendy” on-device inference framework developed from TensorFlow Lite (TFLite), has launched superior acceleration capabilities, primarily based on a ”next-generation GPU engine” referred to as ML Drift.
Google mentioned that this milestone, introduced January 28, solidifies LiteRT as a common on-device framework and represents a big leap over its predecessor, TFLite. LiteRT delivers 1.4x sooner GPU efficiency than TFLite, offers a unified workflow for GPU and NPU acceleration throughout edge platforms, helps superior cross-platform deployment for generative AI fashions, and provides first-class PyTorch/JAX assist by means of seamless mannequin conversion, Google mentioned. The corporate previewed LiteRT’s new acceleration capabilities final Could.
Discovered on GitHub, LiteRT powers apps used every single day, delivering low latency and excessive privateness on billions of units, Google mentioned. By way of the brand new ML Drift GPU engine, LiteRT helps OpenCL, OpenGL, Metallic, and WebGPU, permitting builders to deploy fashions throughout, cell, desktop, and internet. For Android, LiteRT routinely prioritizes when out there for peak efficiency, whereas falling again to OpenGL for broader system protection. As well as, LiteRT offers a unified, simplified NPU deployment workflow that abstracts away low-level, vendor-specific SDKs and handles fragmentation throughout quite a few SoC (system on chip) variants, in accordance with Google.
