Day 17 - OutputGraph: Gathering the Scattered Output Into One Graph

InstructionTranslator produces nodes, guards, and the mutation ledger, all flowing into OutputGraph; at RETURN or graph break, compile_subgraph harvests the graph, hands it to the backend, and installs __compiled_fn into globals

The last four posts each covered one production line: nodes (Day 13), wrapping (Day 14), guards (Day 15), the mutation ledger (Day 16). This post covers the warehouse. InstructionTranslator is the pen, OutputGraph is the paper: one compilation of one frame gets exactly one OutputGraph, and every product gets written onto it.

One graph and its surroundings

member	holds
`graph` (written via SubgraphTracer)	the growing fx.Graph
`graphargs`	the graph's input list, each with a Source
`side_effects`	Day 16's mutation ledger
guards (accumulated via TracingContext)	Day 15's premise set
`install_global` staging	things waiting to enter frame globals; compiled results go here

Inputs are not declared, they register on use

Dynamo does not read the function signature to decide graph inputs. A tensor becomes an input only when actually used: create_graph_input builds the placeholder and a GraphArg is registered. Consequences:

Names come from Source: L['x'] becomes placeholder l_x_, so reading the graph maps straight back to the original variable.
Arguments passed but never used never appear in the graph; the same object used twice registers once (deduplicated by Source). Before harvest, remove_unused_graphargs sweeps once more to pull inputs that became dead code along the way.
nn.Module parameters and buffers take their own channel: register_attr_or_module hangs them into the graph (as get_attr nodes or promoted to inputs, depending on configuration). Day 14's claim that "parameters become graph inputs" lives here.

A stack of tracers, not one

Normally a single SubgraphTracer writes nodes. When a higher-order op arrives (torch.cond, activation checkpoint, ops whose arguments are functions), the branch must become its own subgraph: push a new SubgraphTracer, the branch's nodes go into the subgraph, and any outer value the branch touches gets lifted on the spot into a subgraph input. fx itself has no such nesting management; a large part of why SubgraphTracer exists as a wrapper is exactly this.

Read graph_code by hand

import torch
 
bias = torch.randn(4)
 
def f(x, y):
    return (x @ y + bias).relu()
 
torch.compile(f)(torch.randn(4, 4), torch.randn(4, 4))

TORCH_LOGS="graph_code" python demo.py

Output (excerpted and tidied):

def forward(self, L_x_: "f32[4, 4]", L_y_: "f32[4, 4]", G_bias_: "f32[4]"):
    l_x_ = L_x_
    l_y_ = L_y_
    g_bias_ = G_bias_
    matmul = l_x_ @ l_y_;  l_x_ = l_y_ = None
    add = matmul + g_bias_;  matmul = g_bias_ = None
    relu = add.relu();  add = None
    return (relu,)

Line by line:

The three placeholder names are three Source chains: L['x'], L['y'], G['bias']. The global bias is an input, not a constant, because tensor values always stay inputs (Day 14's betting principle).
Intermediates get = None the moment they are spent: references returned early, memory freed early.
The output is always a tuple, even for a single value.

compile_subgraph: the harvest moment

Only two triggers: RETURN (the whole frame translated) or graph break (cannot go on). Harvesting is a chain of steps:

Liveness: which values on the symbolic stack and in locals are still needed afterwards; those must become graph outputs, or nothing connects after the split.
Settle side_effects (Day 16).
Wire the live values into the output node; remove_unused_graphargs clears unused inputs.
call_user_compiler: hand the GraphModule to the backend (inductor, eager, or your own) and get back a callable compiled_fn. A backend blowup gets wrapped into BackendCompilerFailed.
install_global: the compiled_fn enters the frame's globals under a name like __compiled_fn_1. The new bytecode later reaches it with a single LOAD_GLOBAL.

One cost-saving detail: if at harvest the graph turns out empty (the code had no tensor compute at all), Dynamo skips the backend entirely rather than waste a compilation.

Key takeaways

Both are global values from outside, yet the tensor enters the graph as an input while the int gets baked into a constant, and the direction is set by whether baking pays off. Baking an int carries a real optimization dividend: constants fold, loop bounds specialize, the generated code gets faster, and the bet is that it rarely changes, so baking is worth it. The tensor is the exact opposite: its values differ every batch, so baking would mean recompiling on every call, a guaranteed loss; the values stay an input forever, and the kernel only truly depends on attributes like dtype, shape, and stride. Guards keep the books for both choices: the tensor takes one TENSOR_MATCH that pins attributes but never values, so new data never recompiles; the baked int takes one EQUALS_MATCH that recompiles the moment the value changes, with automatic dynamic (Day 20) as the stop-loss when the bet goes wrong.

__compiled_fn_1 is lying in globals, the graph's inputs and outputs are settled, but CPython will not figure out how to use it on its own. One last step is missing: generate a fresh piece of bytecode that spells out "load, stage arguments, call, unpack outputs, replay the ledger, return" and swap it in for the original function's code. The next post (Day 18) covers PyCodegen and the bytecode toolbox underneath it.