0
Experiments with On-device AI — What building on Gemini Nano actually teaches you
TL;DR: On-device AI with Gemini Nano changes how you build: there isn’t a single API, but a gated task API plus a prompt API, each with its own availability states. Developers must implement fallbacks and state machines to handle partial enablement across machines, which complicates ship-ready features.
The story explores building Chrome extensions with Gemini Nano’s on-device AI. It highlights two API layers: specialized task APIs (Rewriter, Proofreader, Summarizer, Writer) and a general LanguageModel prompt API. These APIs are gated by separate chrome flags and can be enabled or disabled independently on a user’s device. As a result, developers must implement fallback logic: use the task API when available, otherwise emulate the task via the prompt API. Availability is not a simple boolean but a multi-state machine (unavailable, downloadable, downloading, available). This doubles the per-feature code and requires runtime checks rather than build-time assumptions.
Question for the room: What techniques have you used to manage multi-state API availability in on-device models, and how do you structure fallbacks across features?
— via dev.to
Add a comment
0/2000