Jargon:Embedding
1 min read
Say this at the party
“An embedding turns words into a list of numbers, so things that mean similar stuff end up close together, like pins on a map of meaning.”
In plain English
Computers are good with numbers and bad with meaning. An embedding is a trick that turns a word, sentence or picture into a long list of numbers.
The clever bit is where those numbers put things. Pieces of text with similar meanings land near each other, so "puppy" sits close to "dog" and far from "invoice".
Once meaning is a place on a map, a computer can find "things like this one" by looking at what is nearby.
Where it came from
Researchers worked on turning words into numbers for decades. The idea went mainstream in 2013, when a Google team led by Tomas Mikolov released word2vec, which could learn these word maps far faster than earlier methods.
See it in action
Tom searches his notes app for "holiday ideas". It finds a note titled "places to visit in Portugal" even though it never says "holiday", because the two sit close together in embedding space.
What people get wrong
- An embedding isn't a summary you can read. It's just a long list of numbers that only makes sense next to other embeddings.
- Close together means similar in meaning, not true. An embedding can't tell you whether a sentence is correct.
Go deeper
- PaperMikolov et al., Google, 2013Efficient Estimation of Word Representations in Vector SpaceThe word2vec paper that made word embeddings fast and popular.
- ArticleWikipediaWord embedding - WikipediaA clear overview of the idea and its history, from early work to word2vec.
- GuideGoogle, Machine Learning Crash CourseEmbeddings | Machine Learning | Google for DevelopersGoogle's beginner course module, with pictures of how embeddings group similar things.
Related jargon
Drafted with AI. The AI Council (Claude, ChatGPT and Gemini) gave counsel. Phin signed it off.