Day 13 - The Symbolic Interpreter InstructionTranslator: One Handler Per Instruction

The previous post said Dynamo "symbolically executes the bytecode" and fast-forwarded past it. This post fills that hole. The place is torch/_dynamo/symbolic_convert.py, which the codebase itself calls the heart of Dynamo: a class named InstructionTranslator that rewrites CPython's interpreter at the Python level, except what runs on it are not real values but symbols.

The stop before the door: convert_frame

First, connect to the end of Day 12. When the C-level frame hook finds no usable cache for a frame, it calls the Python-side callback, which lives in convert_frame.py. It does some administration: check whether this frame should be touched at all (some files are on a skip list), check whether the recompile count has exceeded its limit (if so, give up and run eager), then call _compile(), all the way into trace_frame(), where the InstructionTranslator is built and the real translation begins.

So the layering is: the C layer catches the frame, convert_frame guards the gate, InstructionTranslator does the work.

Recap: CPython is a stack machine

To see how Dynamo simulates CPython, remember how CPython itself runs. It is a stack machine: each frame has a value stack, and every bytecode is an operation on that stack. LOAD_FAST x pushes local variable x, BINARY_MULTIPLY pops two values, multiplies, pushes the result, STORE_FAST z pops the top into local z.

(Instruction names here are Python 3.10 style; from 3.11 on many merge into things like BINARY_OP, same meaning.)

InstructionTranslator: the same machine, run on symbols

InstructionTranslator copies this machine and swaps out two core parts:

• stack: a Python list holding not real values but VariableTrackers, the symbolic stand-in for every Python value (the protagonist of the next post; treat it as a black box for now).
• symbolic_locals: a dict mapping variable names to VariableTrackers, simulating the frame's locals.

Then, instruction execution: the class has one method per opcode, with the same name. The LOAD_FAST instruction is handled by the method named LOAD_FAST. A metaclass (BytecodeDispatchTableMeta) collects these methods into a dispatch_table at class definition time, and the main loop is: fetch the next instruction, look up the table, call the handler.

What each handler does is translate the operation CPython would perform on real values into the same operation on symbols.

Walking through, one instruction at a time

Take a small function and actually run it:

def f(x, y):
    z = x * y
    return z + 1

The bytecode is roughly:

LOAD_FAST    x
LOAD_FAST    y
BINARY_MULTIPLY
STORE_FAST   z
LOAD_FAST    z
LOAD_CONST   1
BINARY_ADD
RETURN_VALUE

As InstructionTranslator walks it, the state evolves like this:

See the pattern: most instructions only touch the stack and locals and never enter the graph. STORE_FAST is one dict assignment; LOAD_CONST just pushes a constant stand-in. The only instructions that add FX nodes are the ones that hit tensor operations: BINARY_MULTIPLY sees both operands are tensor stand-ins and records a mul.

This is the essence of how Dynamo extracts a computation graph: Python semantics are absorbed on the spot by the interpreter; tensor semantics are recorded to run later. Loops unroll during translation, ifs pick a side during translation (as long as the condition does not depend on a tensor's real value), list and dict operations complete entirely in the symbolic world, and what remains in the graph is pure tensor computation. That is why the graph Dynamo hands the backend is so clean.

RETURN_VALUE is the finish line: translation ends, the graph is finalized, handed to the backend, and new bytecode is generated, the second half of what Day 12 described (graph finalization details are Day 17's topic).

What a handler cannot do is where the graph breaks

This design also directly explains where graph breaks come from. Dispatch reaches a handler; the handler finds it cannot walk this operation symbolically, for example a branch that needs a tensor's real value as a Python bool, so it calls unimplemented() and concedes, triggering a graph break. In other words, Day 12's "an instruction that cannot be symbolized" lands in the implementation as "some handler reached its own surrender branch". The whole machinery (what happens after the surrender, how resume functions are generated) is Day 19's topic.

Watch it walk

TORCH_LOGS has an artifact that prints this process directly:

TORCH_LOGS="trace_bytecode" python your_script.py

The output is one instruction per line with the current stack contents, a live version of the table above. Together with Day 12's TORCH_LOGS="bytecode" (the before/after of the rewritten bytecode), you have a full live broadcast of Dynamo's front end.

Next post

This post kept calling the things on the stack "symbolic stand-ins" and left it vague. The next post (Day 14) is about exactly that: VariableTracker. Dynamo prepares one kind of wrapper for every kind of Python value; tensors, constants, lists, functions, nn.Modules each get their own class, and each decides "what happens when I am called, what happens when an attribute is read". Understanding this type system also unlocks two important facts: why calling your own function does not break the graph (it gets inlined), and how a Python int gets "baked" into the graph as a constant.