Jargon:On-device AI
1 min read
Say this at the party
“On-device AI runs on your phone or laptop itself, so it works offline and your data doesn't have to leave the device.”
In plain English
Most AI runs in huge data centres. Your question travels over the internet, gets answered there, and comes back.
On-device AI uses a smaller model that lives on your own gadget. It is quicker for simple jobs, works with no signal, and keeps more of your stuff private.
The trade-off is brains. A model small enough for a phone can't do everything the big cloud ones can, so many phones mix the two.
Where it came from
Phones have done small AI jobs on the device for years (like face unlock). The idea went mainstream for chatbot-style AI in 2024: Apple announced Apple Intelligence in June with a roughly 3 billion parameter model running on the phone, and Google offers Gemini Nano on Android.
See it in action
On the Tube with no signal, Aisha asks her phone to summarise a long email thread. It does it right there, without sending the emails anywhere.
What people get wrong
- "Uses AI on your phone" doesn't always mean on-device. Many features quietly send your request to the cloud.
- On-device isn't automatically private. It depends on what the app does with the results afterwards.
Go deeper
- PostApple Machine Learning Research, 2024Introducing Apple’s On-Device and Server Foundation ModelsApple explains its small on-phone model and when it hands off to the cloud.
- GuideAndroid Developers, GoogleGemini NanoGoogle's on-device model for Android, and why it works without a connection.
- ArticleWikipediaApple IntelligenceHow one big rollout mixes on-device and cloud processing.
Related jargon
Drafted with AI. The AI Council (Claude, ChatGPT and Gemini) gave counsel. Phin signed it off.