Jargon:Large language model (LLM)
1 min read
Say this at the party
“The kind of AI behind ChatGPT, Claude and Gemini: a program trained on huge amounts of text to predict what words come next.”
In plain English
An LLM learns by reading an enormous amount of text, from books and websites to code, and practising one task over and over: guess the next word.
Do that billions of times and something surprising happens. To guess well, it has to pick up grammar, facts, styles and a rough sense of how ideas connect. That's why it can write, summarise, translate and answer questions.
"Large" means two things: the mountain of text it learns from, and the billions of settings inside it that get tuned along the way.
Where it came from
Language models that predict the next word have been around for decades. The big leap was the transformer, a design Google researchers published in 2017. OpenAI's GPT-3 in 2020 showed that making these models much bigger made them much more capable, and ChatGPT put one in everyone's hands in November 2022.
See it in action
You type "Write a polite email turning down a meeting". The LLM doesn't fetch a template. It writes the most likely good reply, word by word, drawing on all the writing it has seen.
What people get wrong
- An LLM doesn't look things up unless it's given a search tool. On its own, it answers from what it learned in training, which is why it can be out of date or simply wrong.
- LLM and AI aren't the same thing. An LLM is one kind of AI. Chat apps wrap one in a harness with memory, search and tools.
Where Ugenie fits
Ugenie's AI Council puts the same question to several LLMs at once and compares their answers.
Get Ugenie →Go deeper
- TalkAndrej Karpathy, 2023[1hr Talk] Intro to Large Language ModelsThe best hour-long intro, and it works for non-experts too.
- VideoGrant Sanderson, 3Blue1BrownLarge Language Models explained brieflyEight minutes, beautifully animated.
- PaperVaswani et al., Google, 2017Attention Is All You NeedThe paper that introduced the transformer, the design inside every LLM.
- PaperBrown et al., OpenAI, 2020Language Models are Few-Shot LearnersGPT-3: the paper that showed bigger really is different.
Where it comes up
Related jargon
Drafted with AI, checked by the AI Council (Claude, ChatGPT and Gemini) and signed off by Phin.