Jargon:Small language model
1 min read
Say this at the party
“A slimmed-down AI language model, small enough to run on your own phone or laptop instead of a giant data centre.”
In plain English
The big chatbots run on huge models with hundreds of billions of parameters (the settings a model learns). They need warehouses of computers.
A small language model does the same kind of job with far fewer, from millions up to a few billion. It's quicker and cheaper, and it can work on your own device without the internet.
The trade-off is that it knows less and makes more mistakes on hard questions. For simple everyday jobs, it's often plenty.
Where it came from
There's no single coiner. The idea got a big push in 2023 and 2024 when Microsoft's Phi models showed that small models trained on carefully chosen data could punch well above their weight. Its 2024 Phi-3 report was subtitled "A Highly Capable Language Model Locally on Your Phone".
See it in action
Tom is on a train with no signal. His phone still suggests a reply to a text and tidies up his notes, because a small model is running on the phone itself.
What people get wrong
- Small doesn't mean simple. Some of today's small models beat big models from a couple of years ago.
- There's no official size cut-off. Different companies draw the line in different places.
Go deeper
- ArticleWikipediaSmall language modelA short overview of what counts as small and the main model families.
- PaperAbdin et al., Microsoft, 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your PhoneMicrosoft's 3.8 billion parameter model, small enough to run on a phone.
- GuideIBMWhat are Small Language Models (SLM)?A clear comparison of small and large models and when to use each.
Related jargon
Drafted with AI. The AI Council (Claude, ChatGPT and Gemini) gave counsel. Phin signed it off.