Jargon:AI safety
1 min read
Say this at the party
“AI safety is the work of stopping AI from causing harm, whether by accident, by misuse, or by doing something nobody asked for.”
In plain English
Every powerful tool needs safety work. Cars got seatbelts, medicines get trials. AI safety is the same idea for AI.
It covers a wide range. Some people test whether a chatbot will help build a weapon. Some study how models work inside. Some write the rules governments might use.
Alignment (making the AI want the right thing) is one part of it. Safety also includes testing, monitoring and making systems hard to misuse.
Where it came from
Researchers have studied AI risks for decades, and a 2016 paper called "Concrete Problems in AI Safety" helped turn it into everyday lab work. It went mainstream in 2023, when the UK hosted a global AI Safety Summit at Bletchley Park and set up a government AI Safety Institute (now called the AI Security Institute).
See it in action
Before a new model is released, a team spends weeks trying to make it misbehave: asking for dangerous instructions, tricking it with clever prompts, and checking it says no. That testing is AI safety in action.
What people get wrong
- It isn't only about the end of the world. Plenty of safety work is about present-day harms like scams, bias and dangerous advice.
- "Safe" is not the same as "censored". Good safety work tries to block real harm without making the AI refuse ordinary, harmless requests.
Go deeper
- ArticleWikipediaAI safetyA broad overview of the field, its history and the main research areas.
- ArticleUK Government, 2023The Bletchley Declaration by Countries Attending the AI Safety Summit, 1-2 November 2023The short statement countries signed at the first global AI Safety Summit, held in the UK.
- GuideUK AI Security InstituteThe AI Security Institute (AISI)The UK's state-backed lab that tests advanced AI models for risks.
- PaperAmodei et al., Google Brain and OpenAI, 2016Concrete Problems in AI SafetyThe paper that turned AI safety worries into concrete, testable research problems.
Related jargon
Drafted with AI. The AI Council (Claude, ChatGPT and Gemini) gave counsel. Phin signed it off.