Introduction to AI Safety DeCal — Fall 2026

The Berkeley AI Safety Initiative is running an AI Safety DeCal!

Location: Wheeler 120
Time: Mondays, 6–8 PM
First class: Monday, September 14
Units: 1

Recent incidents like the OpenAI model escaping containment and hacking HuggingFace have highlighted the real-world risks of advanced AI systems. Prominent AI researchers such as Yoshua Bengio, Stuart Russell, and Geoffrey Hinton are sounding the alarm about potential catastrophic consequences as the race to build Superintelligence accelerates. Despite this, there are not enough people focused on ensuring we develop this transformative technology responsibly.

In this DeCal, you will gain an understanding of the problems in AI safety and some of the key technical research directions that aim to solve them including Mechanistic Interpretability, Evaluations, Alignment etc. You will learn about the theoretical and practical risks associated with developing advanced AI systems, the difficulties inherent to addressing them, the current state of research regarding solutions, and various AI Safety research opportunities like Anthropic Fellows, MATS Program, SPAR, etc.

Instructors: Prakrat Agrawal, Ali Narin, Brandon Qi

Apply: Fill out the application form
Slack: Join the Berkeley AI Safety Slack

Schedule

Week	Date	Lecture	Readings & Assignments	Lecturer(s)
1	Sep 14	Logistics and Overview of AI Safety	Before class What is AI Alignment? Specification Gaming: How AI Can Turn Your Wishes Against You (video) Why AI alignment could be hard with modern deep learning In class (pick one) Agentic Misalignment: LLMs as Insider Threats How we could stumble into AI catastrophe Core Views on AI Safety Reasoning through arguments against taking AI safety seriously (Bengio) Reflection 1 (due before Week 2) Full reading list	Prakrat, Ali
2	Sep 21	Machine Learning and LLM Fundamentals	Before class (pick 2–3) Visualizing the Deep Learning Revolution (Ngo 2023) Dive Into Deep Learning: Introduction (before §1.4) 3Blue1Brown: But What Is a Neural Network? and Gradient Descent DPO RLHF (InstructGPT / PPO) DPO vs PPO In class (pick one) GPT-3 Understanding Reasoning LLMs Chinchilla scaling laws InstructGPT A Visual Guide to Mixture of Experts Reflection 2 Full reading list (incl. optional)	Brandon
3	Sep 28	AGI Timelines, Takeoff, and Imagining the Future	Before class AI 2027 (header + main timeline forecast; skim the rest) In class (pick one) Steinhardt: More Is Different for AI, Future ML Systems Will Be Qualitatively Different, Thought Experiments Provide a Third Anchor Situational Awareness (first two sections + overview) Why transformative AI is really, really hard to achieve On Recent AI Model Progress METR: Measuring AI Ability to Complete Long Tasks Reflection 3 Full reading list (incl. optional)	Prakrat, Ali
4	Oct 5	The Alignment Problem	Before class Intro to AI Safety, Remastered (video) The Alignment Problem from a Deep Learning Perspective In class (pick one) MONA: Myopic Optimization with Non-myopic Approval (Shah et al. 2025) The Alignment Problem from a DL Perspective Benefits of Assistance over Reward Learning Inoculation Prompting Reflection 4 Full reading list	Brandon
5	Oct 12	Inner Misalignment, Deception, and Alignment Faking	Before class ML systems will have weird failure modes (Steinhardt 2022) Alignment Faking in Large Language Models (Anthropic 2024) Towards evaluations-based safety cases for AI scheming (skim) In class (pick one) Frontier Models are Capable of In-context Scheming (Meinke et al. 2025) Scheming AIs, Section 1 (Carlsmith 2023) AI Control: Improving Safety Despite Intentional Subversion (Greenblatt et al. 2024) AgentMisalignment Evaluating Frontier Models for Stealth and Situational Awareness Reflection 5 Full reading list (incl. optional)	Brandon
6	Oct 19	Control and Oversight	Before class AI Control: Improving Safety Despite Intentional Subversion In class (pick one) Weak-to-strong generalization AI safety via debate Detecting misbehavior in frontier reasoning models Neural Chameleons AgentMisalignment Reflection 6 Full reading list (incl. optional)	Brandon, Ali (tentative)
7	Oct 26	Adversarial Robustness	Before class Universal and Transferable Adversarial Attacks on Aligned LMs (GCG; Zou et al. 2023) Towards Evaluating the Robustness of Neural Networks (C&W; skim) In class (pick one) Circuit Breakers (Zou et al. 2024) Safety Alignment Should Be Made More Than Just a Few Tokens Deep (Qi et al. 2024) AutoDAN (Liu et al. 2023) Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples Trading Inference-Time Compute for Adversarial Robustness Reflection 7 Full reading list	Ali
8	Nov 2	Interpretability, Unlearning, and Representation Engineering	Before class Zoom In: An Introduction to Circuits Representation Engineering (Zou et al. 2023) In class (pick one) Automatically Interpreting Millions of Features in LLMs Locating and Editing Factual Associations in GPT Inference-Time Intervention Mechanisms of Introspective Awareness The Platonic Representation Hypothesis Reflection 8 Full reading list (incl. optional)	Ali
9	Nov 9	Additional Research Agendas & Miscellaneous Topics	Before class (pick two) Auditing language models for hidden objectives (Anthropic 2025) Alignment careers guide (BlueDot Impact) Surfacing Pathological Behaviors in Language Models Natural Emergent Misalignment From Reward Hacking in Production RL Deliberative Alignment Subliminal Learning Sabotage Evaluations for Frontier Models (Benton et al. 2024) In class No in-class readings. Reflection 9 Reflection 10 Full reading list	Prakrat, Brandon
10	Nov 16			Prakrat, Brandon
11	Nov 23	Guest Speakers and Lectures on Special Topics	Speakers TBD. These weeks depend on speaker availability, so guest lectures will likely be dispersed throughout the semester. Two weeks are reserved for speakers and for any delays or cancellations.	TBA
12	Nov 30