Describe the bug
Setting an attention/MLP fork flag directly on the config bridge.cfg.use_hook_mlp_in = True is accepted, but has no effect. The corresponding hook never fires and caches nothing. Only the set_use_* setters work because they additionally propagate the flag to the block components, which hold their own config object rather than sharing bridge.cfg.
Measured on gpt2 (compatibility mode):
bridge.cfg.use_hook_mlp_in = True -> run_with_cache(names_filter=["blocks.0.hook_mlp_in"]) == 0 entries
bridge.set_use_hook_mlp_in(True) -> run_with_cache(names_filter=["blocks.0.hook_mlp_in"]) == 1 entry
Direct config assignment is a natural thing to try (it is how the flags read in cfg), and on HookedTransformer the block reads the shared config so it works there.
Code example
from transformer_lens.model_bridge import TransformerBridge
bridge = TransformerBridge.boot_transformers("gpt2", device="cpu")
bridge.enable_compatibility_mode()
bridge.cfg.use_hook_mlp_in = True # accepted, no error
_, cache = bridge.run_with_cache("hello", names_filter=["blocks.0.hook_mlp_in"])
print(len(cache)) # 0 <- silently not enabled
bridge.set_use_hook_mlp_in(True) # supported route
_, cache = bridge.run_with_cache("hello", names_filter=["blocks.0.hook_mlp_in"])
print(len(cache)) # 1
System Info
Installed from source; macOS arm64, CPU, fp32, Python 3.12, transformers 5.13.0. **Reproduces identically on dev, measured on with byte-identical results.
Expected behaviour & fix pointer
Ideally make the assignment work, alternatively make it impossible to get wrong silently:
- Preferred: have the blocks read the flag from the bridge's config rather than a private copy, so assignment and setter agree. The setters (
set_use_split_qkv_input at transformer_lens/model_bridge/transformer_bridge.py:3608, set_use_attn_in at :3623, set_use_hook_mlp_in at :3641) each call _propagate_attention_flag precisely because the copies diverge — that propagation step is the workaround for the underlying split.
- Alternative (smaller): make these fields properties on the bridge config whose setter performs the propagation, so
cfg.use_hook_mlp_in = True routes through the same path as the setter.
Whichever route, the migration guide's wording needs to match: it currently suggests direct cfg assignment is honored "when blocks share the bridge's cfg", a condition that is never true on a constructed bridge. That clause should be removed or corrected.
Acceptance:
Additional context
Checklist
Describe the bug
Setting an attention/MLP fork flag directly on the config
bridge.cfg.use_hook_mlp_in = Trueis accepted, but has no effect. The corresponding hook never fires and caches nothing. Only theset_use_*setters work because they additionally propagate the flag to the block components, which hold their own config object rather than sharingbridge.cfg.Measured on gpt2 (compatibility mode):
Direct config assignment is a natural thing to try (it is how the flags read in
cfg), and onHookedTransformerthe block reads the shared config so it works there.Code example
System Info
Installed from source; macOS arm64, CPU, fp32, Python 3.12, transformers 5.13.0. **Reproduces identically on
dev, measured on with byte-identical results.Expected behaviour & fix pointer
Ideally make the assignment work, alternatively make it impossible to get wrong silently:
set_use_split_qkv_inputattransformer_lens/model_bridge/transformer_bridge.py:3608,set_use_attn_inat:3623,set_use_hook_mlp_inat:3641) each call_propagate_attention_flagprecisely because the copies diverge — that propagation step is the workaround for the underlying split.cfg.use_hook_mlp_in = Trueroutes through the same path as the setter.Whichever route, the migration guide's wording needs to match: it currently suggests direct
cfgassignment is honored "when blocks share the bridge'scfg", a condition that is never true on a constructed bridge. That clause should be removed or corrected.Acceptance:
bridge.cfg.use_hook_mlp_in = Trueeither enables the hook or raises/warns; it must not be a silent no-opuse_attn_result,use_attn_in,use_split_qkv_inputtests/unit/model_bridge/asserting assignment and setter produce identical hook behaviourmake unit-testpassesuv run mypy .passesAdditional context
Checklist