Day 16 - SideEffects: How Mutating Python Passes Through a Pure Graph

The question Day 15 left open: self.counter += 1, log.append(...), writing a global. These side effects cannot enter the graph (it is purely functional) and cannot be dropped (semantics would break). Dynamo's answer is a third ledger: SideEffects. The first two books were guards recording "premises" and Source recording "provenance"; this one records "mutations".

Why the graph must stay pure

Once the FX graph reaches a backend, the backend needs freedom to reorder, fuse, and delete nodes; Inductor's whole optimization suite (Day 26) is built on that freedom. If the graph hid a fact like "the third node sneakily writes a global", reordering would no longer be safe. So Dynamo's promise to the backend is: the graph computes values and never touches the world.

But your code does touch the world. Dropping mutations is wrong; stuffing them into the graph destroys the promise. Only one road remains: during translation, do not actually mutate, just record; after the graph runs, catch up.

How the ledger is kept

The SideEffects object hangs off OutputGraph (Day 17's protagonist). During translation, every mutated VariableTracker carries a mutation_type tag, four kinds along two axes: is the value or an attribute being mutated, and did the object come from outside or get born mid-trace:

The existing versus new split is the line between life and death: mutations to existing objects must be replayed, because the outside world can see those objects; objects born during tracing that never escape the function (not returned, not stuffed into an existing structure) get their whole ledger entry struck out (prune_dead_object_new), not even reconstructed.

One easily missed detail: while translation is still running, the ledger is the single source of truth. Read self.counter after self.counter += 1 and you get the new value from the ledger, not the stale one on the object. The real object is never touched.

Verify by hand

import torch
 
log = []
 
class Model(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.calls = 0
 
    def forward(self, x):
        self.calls += 1
        log.append(self.calls)
        return x * 2
 
m = torch.compile(Model())
m(torch.randn(4))
m(torch.randn(4))
print(m.calls, log)   # 2 [1, 2]

The graph TORCH_LOGS=graph_code prints holds a single multiply:

def forward(self, L_x_: "f32[4]"):
    l_x_ = L_x_
    mul = l_x_ * 2;  l_x_ = None
    return (mul,)

calls and log are nowhere in the graph, yet the run matches eager exactly: the counter incremented, the list grew. The mutations did not enter the graph and were not dropped; they took the third road.

What replay looks like

When the graph is harvested (Day 17's compile_subgraph), SideEffects hands the ledger to PyCodegen (Day 18), which generates bytecode placed after the "call the graph" part: a STORE_ATTR writes the final calls back, the new contents of log get filled in. Two key points:

• Replay writes final state, not the process. append ten times in a loop, and replay fills in ten elements once; counter += 1 three times, and one store writes the final value. Nobody can observe the intermediate values anyway, because the graph executes atomically; the outside world only looks again after the graph finishes.
• Replay comes after the graph. The generated bytecode calls __compiled_fn, collects outputs, and only then replays. However long the graph grinds on the GPU, the Python-level world stays untouched for exactly that long.

Tensor in-place ops (x.add_(1)) are not this ledger's business: they are tensor compute, they go straight into the graph, and AOTAutograd's functionalization cleans them up later (Day 23). SideEffects covers the Python layer: attributes, containers, globals, closure cell writes.

When the books cannot close, break

Meet a mutation that cannot be modeled, say calling a state-changing method on a C extension object Dynamo cannot represent, and translation can only raise its hand: graph break (Day 13's unimplemented()). And the break itself involves the ledger: at the moment of the split, the first half's books must be settled, every recorded mutation replayed, before control returns to CPython, because the real world must be up to date. This is one more reason graph breaks are expensive: not only does the graph get chopped up, the ledger is forced into early settlement.

Next post

Nodes growing (Day 13), values wrapped (Day 14), guards piling up (Day 15), mutations on the books (today). All of it flows into one object: OutputGraph. The next post (Day 17) looks at how it registers inputs, how it stacks tracers, and how, at the instant of a RETURN or a graph break, it gathers everything into one fx.Graph, hands it to the backend, and trades it for a compiled function.