If you haven’t seen mathematical equations for a few years (or decades!) this page is for you. We’ll use your experience as a software engineer to reintroduce just enough of the language of mathematical notation (“MN”)1 to start work on mechanistic interpretability.
Let’s start with a simple loop.
def compute(x): s = 0 for j in range(3): s += x[j] return sWith x = [2, 4, 8, 16], what does compute(x) return?
1. Summation Notation Is a Loop
Section titled “1. Summation Notation Is a Loop”As a Python programmer, you’re familiar with special keywords or builtins like for and range to express loops. MN is similar,
except that it frequently uses Greek letters, subscripts, and superscripts. This is initially confusing, but some
of it stems from the annoying convention that mathematicians omit the multiplication sign when multiplying numbers.2
So while we read ab as a two-character string, mathematicians read it as times .3
ML researchers also like to act fancy by using lots of Greek letters.4 Like variables in programs, these letters can mean anything, but certain conventions have developed. For example, you’re probably familiar with the lowercase letter to represent the ratio of a circle’s circumference to its diameter. But in other fields of math, is sometimes used to represent a probability distribution, and in reinforcement learning it’s used to represent a particular way of choosing an action (a “policy”). A large part of getting comfortable with mathematical notation is understanding and internalizing these conventions.
We’ll start with the simplest and most common convention: The uppercase letter Sigma () represents a sum. It’s
almost exactly equivalent to a for loop where we’re adding things up inside the loop. And just as we have to define
the bounds of a for loop in code (e.g. for i in range(1, 5)) we also have to define the bounds when using .
We do this by writing the lower bound as a subscript, and the upper bound as a superscript. We also use the subscript
to name the variable that we’re looping over. So if we want to represent for i in range(1,5) we’ll
write . Notice that while our loop was over range(1,5), our summation is from to . This is just
because Python’s range function excludes the final value, while includes it.
In Python, we access elements of lists or arrays with indexing. For example, to get the th element of , we write
x[j]. In MN, we’ll again use subscripts: we write the th element of as .
Let’s put it together now. First, a simple Python loop:
s = 0for i in range(1, 10): s += x[i]And now its MN equivalent:
x = [4, 1, 6, 2, 9, 3, 5, 8, 7, 2], summed from i = 1 to 9.
Step 1 of 10
Let’s do some simple exercises to lock in the idea. Please: it’s critical to do these without AI help. You’re here to learn, and if you’re having the AI do it for you, you’re not learning. On the other hand, if you don’t understand something, I fully encourage you to ask an AI for a better explanation.
Now on to the exercises!
With x = [3, 5, 7, 11], evaluate the sum.
Implement this sum in compute. Arrays have at least four elements.
To leave the editor using only the keyboard, press Esc, then Tab.
2. Subscripts Are Indexes
Section titled “2. Subscripts Are Indexes”We’re now going to focus on a very important equation in AI5:
This may look a little intimidating at first, but it’s just more of what we’ve already learned about indexes and subscripts.
We’ll take it bit by bit6. First, let’s look at . In this equation, is a “free index”, in that it doesn’t
vary in the summation. So mentally, just think that we’ve defined an array y and we’re setting the th element,
y[i]. is similar: we have another array, b, and again we’re accessing the th element.
What is and where did it come from? Think of it as an implicit index that we need to loop over to set all the elements of . We use it in a few different places: , , and . Because these all share the same index, we can assume that the length of is the same as the length of , and both are the same as the size of the first dimension of .
Now on to the sum. Let’s skip the summation itself for now and take a look at . The two indexes, and ,
tell us that this is a 2D array.7 In Python, we would write this simply as W[i][j] (or W[i,j] if you’re
using NumPy).
Now back to the summation. Notice that we’re summing from to . Because we’re “declaring” in the summation, we call it a “bound” index. We don’t specify in the subscript, so we’re holding it constant. Think of this as a scoping rule: has equation-level scope, and is scoped to the summation. We’re doing that MN thing of not explicitly saying what is, and trusting the reader to figure out from context that it’s probably the size of the second dimension of , or the number of elements in , and that these two sizes are probably the same. Because there’s no operator between and , we use the MN default of multiplication.
We’ve spent a lot of time breaking down the equation in MN, so now let’s see how it might look in Python.
Recall that we have two loops: The implicit loop over values of i, and the summation loop over values of j. We also
have three variables: x, W, and b.
def compute(x, W, b): i_range = len(W) # Could also have used len(b) y = [0] * i_range for i in range(i_range): # Our implicit outer loop y[i] = b[i] # First the b_i term, to which we'll add the summation j_range = len(x) # Could also have used len(W[0]) for j in range(j_range): pass # You get to fill in this part in the exercise below. return yWhen computing a fixed output element y_i, which index varies within the summation?
Supply the missing expression.
def compute(x, W, b):
y = [0] * len(W)
for i in range(len(W)):
y[i] = b[i]
for j in range(len(x)):
y[i] += ______
return y
W = [[2, 0, 1], [1, 3, -1]], x = [3, 1, 2], b = [1, -2].
Step 1 of 8
3. Shapes Are a Type System
Section titled “3. Shapes Are a Type System”In programming, we rely on types to ensure that the parameters we use make sense. Even in Python, with its loose
“duck typing”, the interpreter throws an exception at runtime if we do things like None + 1. Or we can use
programs like mypy to do type checking before we run our program.
Similarly, in MN, we have something that is kind of like types. Specifically, our expressions have shapes.
Shapes are exactly the same as the dimensions of a multidimensional array. For example, a 2D array with m rows and
n columns has shape (m, n).
There are a couple of other conventions when using shapes:
- A normal value that is not an array is a scalar, and has no shape (or we might say it has an empty shape or zero dimensions).
- You might see the shape of a 1D array with len n written as
(n,). This is because writing e.g.(5,)in Python is the expression for a tuple with one element (the element5), and numpy’s shape property gives the shape as a tuple. MN refers to 1D arrays as vectors. - A 2D array is called a matrix, and an array of more than two dimensions is called a tensor.8 In fact, a tensor can have any number of dimensions - 0, 1, 2, or more. We’ll be working with tensors a lot.
- In MN, you might see something like . This tells you not just the shape of the matrix (m rows, n columns), but also the type of elements in the matrix - ℝ is the MN notation for real numbers. You might also see integers (ℤ), natural numbers (ℕ), rational numbers (ℚ), and rarely complex numbers (ℂ).
Shapes and Arithmetic
Section titled “Shapes and Arithmetic”In general, to do things like addition and subtraction, your operands have to be the same shape.
Strictly speaking, in MN you can’t add a scalar and a vector . Many ML papers do it anyway
(“with abuse of notation”), because it’s easier than writing the notation for adding .
Most of the time though you’ll be working in PyTorch or numpy, and both of these support broadcasting.
Broadcasting is somewhat like type coercion, in that it
applies a set of rules to get otherwise incompatible shapes to work together. We
won’t get into the rules here9, but as one example torch.tensor([2, 3, 4]) + 1 gives tensor([3, 4, 5]).
MN does however allow you to do multiplication and division with a scalar and a vector, for example or .
Reading Shapes from an Equation
Section titled “Reading Shapes from an Equation”Let’s go back to our old friend:
Here we can see that is a 1D vector (or tensor). If has elements, then its shape is (m,).
We know from the summation that has columns, and we know it has rows because it shares the index with .
This means that is a matrix with shape (m, n). Also from the summation, we know that is a vector with elements.
Finally, we know that is a vector with elements because it also shares the index with .
W.shape == (4, 3) and x.shape == (3,). What is the shape of y?
This code assumes W is square. Repair it for any W with shape (m, n).
To leave the editor using only the keyboard, press Esc, then Tab.
4. Superscripts and Conventions
Section titled “4. Superscripts and Conventions”We’ve seen how to interpret subscripts as indexes. Now we’re going to move on to superscripts. You’re already familiar with exponents, like or . They’re the normal kind of repeated multiplication that you learned in school. But in MN, we often use superscripts as indexes as well. And just as with subscripts, knowing conventions is extremely helpful when trying to read ML papers and blog posts.
Let’s look at one example where we use both superscripts and subscripts as indexes:
This is the formula for the update to a neural network’s residual stream10. The residual stream is the “backbone” of a transformer network, and helps carry information between layers. The residual stream is a matrix, but it’s better to think of it as a 2D nested array: One array for each token position in our input. As usual here we’re using to indicate which element of the array we’re working with, and as we just mentioned, each element of the array is itself an array.
The residual stream changes after each layer of the network, so we need a way to say which layer we’re talking about. By convention, you’ll usually see used to represent the layer index11, like we do here. And usually, you’ll see the in parentheses to help indicate that it’s an index rather than an exponent, but some authors don’t follow this convention.
We can also see that the formula is recursive: We define the vector for the next layer (layer ) as the sum of the output of layer () and the current vector ().
I want to mention one more thing that might trip you up if you haven’t seen it for a while: a T that looks like an exponent, e.g. or . This isn’t an exponent or an index, but rather indicates the transpose of a matrix. We’ll get into transposition later when we cover matrices in their own lesson.
x holds the residual stream at every layer: x[0] holds the vector for every token position at layer 0, x[1] at layer 1, and so on. Which Python expression gives this value?
x is the residual stream and a is each layer's output. Both are indexed as x[layer][i], where x[layer][i] is the vector for token position i. Return the residual stream after layer + 1, as a list with one vector for each i.
To leave the editor using only the keyboard, press Esc, then Tab.
5. Putting It Together
Section titled “5. Putting It Together”Now it’s time to put everything we’ve learned together: summation, indexes, shapes, and superscripts!
Max and ReLU
Section titled “Max and ReLU”We’re going to add one small new piece: just as in Python, MN has some functions that are ‘built-in’ in the
sense that the convention around these functions is so strong that we can regard them as fully defined just from
the function name. One of these functions is and it works exactly like Python’s max function: its value is
the larger of the two arguments. When one of the arguments to is 0, it becomes .
ML gives this one-argument function a special name: a Rectified Linear Unit (ReLU)12.
Two Equations
Section titled “Two Equations”Now, let’s take a look at the equations we’ll be using, and their arguments’ types (shapes). First, the arguments:
And now let’s look at the two equations we’ll use:
Equation 1, which defines , should look familiar. It’s just the equation from Section 2, wrapped in a ReLU.
Equation 2, which defines , is also very similar to , with one key difference: instead of in the summation, we use , which is the result of Equation 1.
Looking back at the arguments, we see three numbers that define the shapes of all the arguments: , , and . The three things that aren’t mentioned there are , , and . But by examining the equations, we can see that because and is indexed by , then must range from to . Looking at the summation in Equation 2 confirms this. Equation 1’s summation gives us the bounds for , which we can see ranges from to . And finally, because is indexed by and , then must range from to .
Let’s look at Equation 1 in a little more detail. Compare the equation from Section 2 and this new equation:
You can see that the two equations are exactly the same, except for the ReLU (the ). Just as with Section 2’s output , Equation 1’s output is a vector. And we know that has elements, since it’s indexed by .
Now that we understand Equation 1, let’s take a look at Equation 2. It’s almost exactly the same as the Section 2 equation - the only difference is that we have to take the from Equation 1 and plug it into the summation. Since the output of Equation 2, , is indexed by k we know that will have elements.
Reading Equations in Order
Section titled “Reading Equations in Order”We can read this pair of equations in order like lines of a program. Because sums over all elements of , we need to calculate Equation 2 after Equation 1. Doing Equation 1 and then Equation 2 is just like writing two functions, where the second function takes as a parameter the output of the first function.
Type-Checking the Equations
Section titled “Type-Checking the Equations”We’ve said before that shapes kind of work like types. Let’s do our own “type checks” on the two equations.
First, let’s look at Equation 1. We know its output is going to be with one dimension of size based on our previous analysis of the range of . is also the same size, for the same reason. In the summation, we know that has shape , and we’re doing two loops: one walking over each row of , and one walking over each column of that row. is also indexed by , so we know must have the same number of elements as has columns.
The sum gives us one number for each , and also has one value for each , so adding them is valid. Applying doesn’t change the shape, so Equation 1 passes the type check!
Equation 2 is the same, just with different subscript letters. Once again we’re doing two loops: one walking over each row of , and one walking over each column of that row. In the summation, is also indexed by , so we know must have the same number of elements as has columns. The sum gives us one number for each , and also has one value for each , so adding them is valid. Equation 2 passes the type check!
Specification, Not Implementation
Section titled “Specification, Not Implementation”So far, we’ve been thinking of MN as a programming language, but another way to think about it is as a specification. That is, the equations tell us what to compute, but not how to compute it. We’ve been talking about nested loops, but you could also do the exact same calculation using NumPy functions or a GPU kernel written in CUDA. In Lesson 2, we’ll see that each equation can be simplified using matrix multiplications, and the two equations shrink to:
Now try it yourself. First, work out by hand for a small input. Then write the code for both equations.
With x = [1, 2], W = [[1, -1], [0, 2], [-1, 1]], and b = [0, -1, 0], what is h?
Implement both equations and return z. Don't assume any fixed dimensions.
To leave the editor using only the keyboard, press Esc, then Tab.
Check Your Understanding
Section titled “Check Your Understanding”We’ve completed a refresher of basic mathematical notation, and started to get a feel for how to read MN and turn it into a program.
Now here’s a challenge problem to check your understanding! It combines the residual stream ideas from section 4 with the kinds of weighted sums we’ve been working on for most of this lesson.
If you have trouble, refer back to the earlier exercises you completed and see how you can use them to solve this problem.
Implement this equation in the compute function. x has shape (n,), and W has shape (L, n, n). W[layer] is the weight matrix for that layer. Start with , apply the update for each layer ℓ from 0 to L - 1, and return , with shape (n,).
To leave the editor using only the keyboard, press Esc, then Tab.
Formula Forensics
Section titled “Formula Forensics”Now for some debugging practice! In the Formula Forensics game, each problem gives you a mathematical formula and code that implements it incorrectly. As a programmer, you know what to do: Find the bug and fix it!
Companion game
Formula Forensics
Use each specification to find and repair the bug in a numerical function.
Play Formula ForensicsNext: Vectors: Directions Are Features
Section titled “Next: Vectors: Directions Are Features”So far we’ve treated vectors like arrays or lists: just a series of numbers. But you’re here to learn mechanistic interpretability!13 In the next lesson, we’ll take an interpretability view of a vector. We’ll see that we can treat a vector as a direction, and how that direction relates to features that a model might have learned.
Footnotes
Section titled “Footnotes”-
I hope you’re okay with me calling mathematical notation “MN.” We programmers have a long tradition of naming languages things like C, C++, C#, D, F, so MN fits right in. ↩
-
Not using multi-letter names also means they tend to dip into Greek when they run out of Latin characters. We’ll talk more about that in a later lesson. ↩
-
This fact causes mathematicians no end of self-inflicted pain, as expressed in common mathematical laments like “with abuse of notation” and “subscripts omitted for simplicity.” ↩
-
For example, see the discussion of “mathiness” in Troubling Trends in Machine Learning Scholarship. ↩
-
Each is the output of a neuron in a neural network, before any nonlinearity. Later, we’ll see how to represent this equation more concisely with matrix math. ↩
-
Get it? Bit! Because we’re programmers! So funny! ↩
-
Mathematicians really like single letters, and they like to concatenate them and trust that you’ll figure out the meaning from context. When you see multi-character things like , it’s almost always a bunch of single-character variables rather than a multi-character variable. ↩
-
This is different than what mathematicians mean by a tensor. You may run into mathematical tensors occasionally when reading ML papers, but it’s pretty rare, and the “n-dimensional array” meaning is more common in ML. ↩
-
You can read the full rules here. If you think you’ve really mastered the broadcasting rules, you might want to try these puzzles. Or you might decide not to worry about it; this is the kind of thing coding agents are really good at. ↩
-
We’ll get into the details of what a residual stream is much later, after we’ve built up some more preliminaries. Suffice it to say, it’s an extremely important piece of modern deep learning and mechanistic interpretability. ↩
-
We use instead of to avoid confusion with the number . ↩
-
ReLUs are used in many older LLMs, though most recent LLM architectures replace ReLU with other similarly-shaped functions instead. We’ll get to why ReLUs are useful in a later lesson. ↩
-
At least I hope so. This whole thing has been rather awkward for us otherwise. ↩
Your progress is saved in this browser, on this device only. It is never uploaded, and it will be lost if you clear your browser's storage.