Skip to content

Learn mechanistic interpretability by reverse-engineering neural networks.

Each lesson has a short article, coding exercises in Python, and a small game where you put the ideas to work. The early lessons run entirely in your browser so you don’t need any GPUs to get started.

Start with Math Notation Is Just Code

def compute(x):
    s = 0
    for j in range(3):
        s += x[j]
    return s
s=∑j=02xj{\htmlData{link=result}{s}} =\mathop{\htmlData{link=sum}{\sum}}\limits_{\htmlData{link=index}{j}=\htmlData{link=bounds}{0}}^{\htmlData{link=bounds}{2}} \htmlData{link=term}{x_{\htmlData{link=index}{j}}}

As software engineers, we were used to writing programs by hand. If we didn’t understand how they worked, we traced through them in a debugger, sprinkled print statements throughout the code, or just sat and stared at functions until they started to make sense.

Unfortunately, none of that works for neural networks. Sure, in one sense they’re software like any other. They have ordinary mathematical computations and you can look at their source code line by line to see how those computations are implemented.

But the real core of the neural network is the weights, and right now, nobody really understands how those weights turn into language and behavior. We’re starting to make progress on the problem though, and some of the best techniques fall under the umbrella of “mechanistic interpretability.” Mechanistic interpretability is the art and science of reverse-engineering how LLMs and other deep learning networks actually do what they do.

There are lots of resources for learning mechanistic interpretability, but many of them are targeted toward academics and researchers. This course is targeted toward software engineers like you. As a programmer, you already know how to read unfamiliar code, form a hypothesis about what it does, and test it by exploring the code’s behavior. Mechanistic interpretability uses the same core skills with some new tools: things like linear probes, sparse autoencoders, and circuit tracing. You might never have encountered these before, but by the time you’re done with this course, you’ll know how to apply them to understand and modify LLMs and similar neural networks.

There’s no getting around it: you’ll need more math than you might be accustomed to using. Thankfully, the math isn’t complicated. You’ve probably seen at least some of it in high school, and the rest we’ll teach you along the way by leveraging your programming skills. Each lesson has a short article, coding exercises in Python, and a small game where you put the ideas to work. The early lessons run entirely in your browser so you don’t need any GPUs to get started.

Start with Math Notation Is Just Code.

Lessons

Math for Reverse-Engineering Neural Networks

  1. Lesson 050 minutes

    Math Notation Is Just Code

    Learn to read mathematical equations as executable programs.

    12 exercises

    Game: Formula Forensics

  2. Lesson 1Coming next

    Vectors: Directions Are Features

    Game: Feature Hunt

  3. Later in the course

    • Linear probes
    • Sparse autoencoders
    • Circuit tracing

Your progress is saved in this browser, on this device only. It is never uploaded, and it will be lost if you clear your browser's storage.