Jargon:Parameters
Say it: puh-RAM-it-ers · 1 min read
Say this at the party
“The billions of tiny settings inside an AI model that get tuned during training. They're where everything it learned is stored.”
In plain English
Think of a mixing desk in a recording studio, but with billions of sliders instead of a few dozen. Each slider is a parameter.
Training is the process of nudging every slider, a tiny bit at a time, until the model's answers come out right. When training ends, the sliders are fixed.
When you hear "a 70 billion parameter model", that's the number of sliders. It's a rough guide to size, a bit like engine size in a car.
Where it came from
"Parameter" is an old maths and statistics word for a value that defines how a formula behaves. Parameter counts became a headline number for AI in 2020, when OpenAI's GPT-3 paper described a model with 175 billion of them.
See it in action
A small model with about 4 billion parameters can run on a decent phone. The biggest models have hundreds of billions and need rooms full of specialist computers.
What people get wrong
- More parameters doesn't guarantee a smarter model. How well it was trained, and on what, matters just as much.
- Parameters aren't facts stored in neat boxes. What the model knows is spread across all of them at once.
Go deeper
- GuideGoogle for DevelopersMachine Learning GlossaryDefines parameters as the weights and biases a model learns during training.
- PaperBrown et al., OpenAI, 2020Language Models are Few-Shot LearnersThe GPT-3 paper. Its 175 billion parameters made model size a headline number.
- ArticleWikipediaLarge language modelContext on how parameter counts and model weights fit into today's chatbots.
Related jargon
Drafted with AI. The AI Council (Claude, ChatGPT and Gemini) gave counsel. Phin signed it off.