Learn mechanistic interpretability by reverse-engineering neural networks.
Each lesson has a short article, coding exercises in Python, and a small game where you put the ideas to work. The early lessons run entirely in your browser so you don’t need any GPUs to get started.
def compute(x):
s = 0
for j in range(3):
s += x[j]
return sAs software engineers, we were used to writing programs by hand. If we didn’t understand how they worked, we
traced through them in a debugger, sprinkled print statements throughout the code, or just sat and stared at
functions until they started to make sense.
Unfortunately, none of that works for neural networks. Sure, in one sense they’re software like any other. They have ordinary mathematical computations and you can look at their source code line by line to see how those computations are implemented.
But the real core of the neural network is the weights, and right now, nobody really understands how those weights turn into language and behavior. We’re starting to make progress on the problem though, and some of the best techniques fall under the umbrella of “mechanistic interpretability.” Mechanistic interpretability is the art and science of reverse-engineering how LLMs and other deep learning networks actually do what they do.
There are lots of resources for learning mechanistic interpretability, but many of them are targeted toward academics and researchers. This course is targeted toward software engineers like you. As a programmer, you already know how to read unfamiliar code, form a hypothesis about what it does, and test it by exploring the code’s behavior. Mechanistic interpretability uses the same core skills with some new tools: things like linear probes, sparse autoencoders, and circuit tracing. You might never have encountered these before, but by the time you’re done with this course, you’ll know how to apply them to understand and modify LLMs and similar neural networks.
There’s no getting around it: you’ll need more math than you might be accustomed to using. Thankfully, the math isn’t complicated. You’ve probably seen at least some of it in high school, and the rest we’ll teach you along the way by leveraging your programming skills. Each lesson has a short article, coding exercises in Python, and a small game where you put the ideas to work. The early lessons run entirely in your browser so you don’t need any GPUs to get started.
Start with Math Notation Is Just Code.
Lessons
Math for Reverse-Engineering Neural Networks
- Math Notation Is Just Code
Learn to read mathematical equations as executable programs.
Game: Formula Forensics
Vectors: Directions Are Features
Game: Feature Hunt
Mechanistic Interpretability for Software Engineers
Neural Networks Are Just Programs
Game: The Neural Crash Dump
- Linear probes
- Sparse autoencoders
- Circuit tracing
Your progress is saved in this browser, on this device only. It is never uploaded, and it will be lost if you clear your browser's storage.