The previous post kept vaguely calling the things on the symbolic stack "stand-ins". This post makes them concrete. Under torch/_dynamo/variables/, Dynamo prepares a wrapper class for every kind of Python value tracing can meet, collectively VariableTracker. This type system is Dynamo's complete answer to the question "how dynamic is Python".
Why wrap at all
While translating bytecode, Dynamo holds no real values, yet an instruction's behavior depends on the value's type: the same call instruction means wildly different things depending on whether the callee is torch.sin or a function you wrote. So every value needs a stand-in that knows what it is, so a handler can ask it: what happens when you are multiplied? when you are called? when an attribute is read from you?
Those questions are exactly the VariableTracker interface: call_function(), call_method(), as_proxy() (hand out the proxy used in the graph), as_python_constant() (if I am really a constant, surrender the actual value), reconstruct() (how to rebuild me in the rewritten bytecode). Each subclass answers in its own way.
The family
A few members you will hit most often; their behavioral differences are Dynamo's behavior in miniature:
| Python value | wrapped as | key behavior |
|---|---|---|
| tensor | TensorVariable | carries an FX proxy; operations add graph nodes |
| int, str, None | ConstantVariable | absorbed into the graph as a constant, not an input |
| list, tuple | ListVariable etc. | elements wrapped individually, ops simulated symbolically |
| dict | ConstDictVariable | same; key lookups resolved at translation time |
| your function | UserFunctionVariable | inlined on call, translation continues inside |
nn.Module | NNModuleVariable | attribute chains tracked, parameters become graph inputs |
| other objects | UserDefinedObjectVariable | tracked attribute by attribute, as far as possible |
Two behaviors deserve a closer look.
ConstantVariable: a Python int gets baked into the graph. Your function received n=3; the graph will not have an input named n. Instead x * n becomes the node x * 3 directly. The graph is easier to optimize (constants fold, kernels specialize); the price is that this graph only holds for n=3. Who pays that price? The bookkeeping is done by the next post's guards.
UserFunctionVariable: a call is not a break point, it is an entrance. When Dynamo meets a call to a function you wrote, it does not break the graph; it opens an InliningInstructionTranslator (a subclass of the machine from the previous post), dives into the callee's bytecode, keeps translating, and splices the result back onto the original stack. The whole call chain is flattened into one graph. Which functions to dive into and which to skip (numpy internals, say) is decided by the lists in trace_rules.py.
There is also a performance role: LazyVariableTracker. Wrapping itself has a cost, and a value never touched is not worth expanding, so it first gets a lazy shell and only materializes when actually used. That is where compile time gets saved.
Source: where did this value come from
VariableTracker has one more key field, source, answering a different question: how to fetch this value at runtime.
cfg = Config() # a global object, cfg.scale == 2
def f(x):
return x * cfg.scaleWhen tracing cfg.scale, Dynamo records its origin as a chain: AttrSource(GlobalSource('cfg'), 'scale'), printed as G['cfg'].scale. A function argument is a LocalSource, printed as L['x']. There are only two kinds of roots (locals, globals); everything else chains on with AttrSource, GetItemSource, and so on, so any value's access path can be expressed.
The chain has two uses, matching Dynamo's two outputs:
source.make_guard(...): generate guards.cfg.scalewas baked into the constant2, so the premise "G['cfg'].scale == 2" must be recorded and checked on the next call. Changecfg.scale, the guard fails, recompilation follows.source.reconstruct(codegen): generate bytecode. The rewritten bytecode must stage the graph's inputs, and it does so by following this chain to emit instructions likeLOAD_GLOBAL cfg,LOAD_ATTR scale.
Two factories: with or without a Source
The wrapping entrance is VariableTracker.build(tx, value, source=...), which routes to two factories by whether a source exists:
VariableBuilder: values with a source, i.e. arriving from the outside world (arguments, globals, attributes). They may differ on the next call, so wrapping comes with installing guards.SourcelessBuilder: values born mid-trace (an intermediate list computed at translation time, say). They do not come from outside; the next call will recreate them identically, so no guard needed.
One sentence for this split: guards watch the boundary between the graph and the outside world, not the graph's interior.
Next post
The price of baking cfg.scale into a constant is the premise "G['cfg'].scale must still be 2". This post showed where premises grow from; the next (Day 15) looks at what they look like and how they are checked: the kinds of guards, how to read each line TORCH_LOGS=guards prints, and how, to keep the per-call check fast, Dynamo compiles the whole guard set into a C++ tree.