Jargon:Compute
1 min read
Say this at the party
“AI-speak for raw computer power: the chips, electricity and hours needed to train and run AI. Whoever has the most compute can build the biggest models.”
In plain English
In AI, "compute" is used as a noun. It means how much calculating you can do, mostly on special chips called GPUs, packed into huge data centres.
Training a big model takes enormous amounts of it, run for weeks. Then every question you ask uses a little more.
That's why AI news is full of chip deals, new data centres and power stations. Compute has become the scarce ingredient.
Where it came from
Rich Sutton's 2019 essay "The Bitter Lesson" argued that AI methods which use more computation keep winning. A 2022 study by Epoch AI found that since deep learning took off in the early 2010s, the compute used to train models had doubled roughly every six months.
See it in action
A start-up wants to train its own model. Before writing any code, it has to rent thousands of GPUs from a cloud provider for weeks, and that bill is its biggest cost.
What people get wrong
- More compute doesn't automatically mean a better AI. Data, design and clever training matter too.
- It isn't only about training. Running AI for millions of users every day (inference) also eats huge amounts of compute.
Go deeper
- PaperSevilla et al., Epoch AI, 2022Compute Trends Across Three Eras of Machine LearningHow training compute grew, and when it sped up.
- PostRich Sutton, 2019The Bitter LessonWhy AI methods that use more computing power tend to win in the end.
- GuideEpoch AITrends in AIUp-to-date charts on training compute, chips and costs.
Related jargon
Drafted with AI. The AI Council (Claude, ChatGPT and Gemini) gave counsel. Phin signed it off.