Jargon:AI Council
2 min read
Say this at the party
“Asking several AIs the same question, letting them mark each other's answers, then having one AI sum it all up into a single reply.”
In plain English
An LLM (large language model) is the kind of AI behind ChatGPT, Gemini and Claude. A council sends your question to several of them at once. It's like asking a few friends instead of just one. And yes, a council gives counsel.
Andrej Karpathy's version has three steps. First, each AI answers on its own. Next, each one reads the others' answers, with the names hidden, and ranks them. Last, a "Chairman" AI reads the lot and writes one combined answer.
Karpathy used his council to help him read books. The AIs kept voting GPT-5.1 the most insightful. Karpathy disagreed and found it "too wordy". So AI judges and human judges don't always see eye to eye.
Where it came from
In April 2024, researchers at Cohere, an AI company, suggested swapping one AI judge for a "jury" of different AIs. In June 2024, Justin Zhao and colleagues named a "Language Model Council", where 20 AIs set tests and marked each other. The phrase went mainstream on 22 November 2025, when Andrej Karpathy, a founding member of OpenAI, shared llm-council on GitHub (a site for sharing code) as a "fun Saturday hack".
See it in action
You ask Perplexity's Model Council whether to fix or replace your old boiler. Three AIs, such as Claude, GPT and Gemini, answer at once. A fourth AI writes one reply showing where they agree and where they differ. Where they disagree, you dig deeper.
What people get wrong
- If they all agree, it isn't automatically true. The AIs learnt from a lot of the same text, so they can share blind spots. The disagreements are often the most useful bit.
- It isn't a new, smarter kind of AI. It's a way of using the AIs we already have. Karpathy's version is a few hundred lines of code.
- Karpathy didn't invent it. He made it famous, but researchers wrote about councils and juries of AIs in 2024.
- It costs more and takes longer than asking one AI. Your question also goes to several companies at once, so think twice before sharing anything private.
Where Ugenie fits
Ugenie uses a council to fact-check every Jargon entry, including this one. It catches more slips than one AI alone, but it can still get things wrong.
Get Ugenie →Go deeper
- PostAndrej Karpathy, 2025GitHub - karpathy/llm-council: LLM Council works together to answer your hardest questionsThe weekend project that made the phrase famous, with the three steps explained.
- PaperZhao et al., 2024Language Model Council: Democratically Benchmarking Foundation Models on Highly Subjective TasksThe earliest use of the "council" name: 20 AIs setting and marking each other's tests.
- PaperVerga et al., Cohere, 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse ModelsShows a jury of smaller AIs can mark answers more like humans do, for far less money.
- PostPerplexity, 2026Introducing Model CouncilThe idea turned into a paid product you can actually use.
- ArticleMichael Nuñez, VentureBeat, 2025A weekend 'vibe code' hack by Andrej Karpathy quietly sketches the missing layer of enterprise AI orchestrationGood coverage of Karpathy's project, plus a taste of how big the claims got.
Related jargon
Drafted with AI. The AI Council (Claude, ChatGPT and Gemini) gave counsel. Phin signed it off.