Skip to content

[Bug Report] Assigning cfg.use_attn_result / use_hook_mlp_in / use_split_qkv_input directly is silently ignored #1689

Description

@jlarson4

Describe the bug

Setting an attention/MLP fork flag directly on the config bridge.cfg.use_hook_mlp_in = True is accepted, but has no effect. The corresponding hook never fires and caches nothing. Only the set_use_* setters work because they additionally propagate the flag to the block components, which hold their own config object rather than sharing bridge.cfg.

Measured on gpt2 (compatibility mode):

bridge.cfg.use_hook_mlp_in = True   -> run_with_cache(names_filter=["blocks.0.hook_mlp_in"]) == 0 entries
bridge.set_use_hook_mlp_in(True)    -> run_with_cache(names_filter=["blocks.0.hook_mlp_in"]) == 1 entry

Direct config assignment is a natural thing to try (it is how the flags read in cfg), and on HookedTransformer the block reads the shared config so it works there.

Code example

from transformer_lens.model_bridge import TransformerBridge

bridge = TransformerBridge.boot_transformers("gpt2", device="cpu")
bridge.enable_compatibility_mode()

bridge.cfg.use_hook_mlp_in = True                       # accepted, no error
_, cache = bridge.run_with_cache("hello", names_filter=["blocks.0.hook_mlp_in"])
print(len(cache))                                       # 0   <- silently not enabled

bridge.set_use_hook_mlp_in(True)                        # supported route
_, cache = bridge.run_with_cache("hello", names_filter=["blocks.0.hook_mlp_in"])
print(len(cache))                                       # 1

System Info

Installed from source; macOS arm64, CPU, fp32, Python 3.12, transformers 5.13.0. **Reproduces identically on dev, measured on with byte-identical results.

Expected behaviour & fix pointer

Ideally make the assignment work, alternatively make it impossible to get wrong silently:

  • Preferred: have the blocks read the flag from the bridge's config rather than a private copy, so assignment and setter agree. The setters (set_use_split_qkv_input at transformer_lens/model_bridge/transformer_bridge.py:3608, set_use_attn_in at :3623, set_use_hook_mlp_in at :3641) each call _propagate_attention_flag precisely because the copies diverge — that propagation step is the workaround for the underlying split.
  • Alternative (smaller): make these fields properties on the bridge config whose setter performs the propagation, so cfg.use_hook_mlp_in = True routes through the same path as the setter.

Whichever route, the migration guide's wording needs to match: it currently suggests direct cfg assignment is honored "when blocks share the bridge's cfg", a condition that is never true on a constructed bridge. That clause should be removed or corrected.

Acceptance:

  • bridge.cfg.use_hook_mlp_in = True either enables the hook or raises/warns; it must not be a silent no-op
  • Same for use_attn_result, use_attn_in, use_split_qkv_input
  • Mutual-exclusivity and unsupported-architecture errors from the setters still fire on the assignment path
  • The migration guide no longer implies direct assignment works
  • Test under tests/unit/model_bridge/ asserting assignment and setter produce identical hook behaviour
  • make unit-test passes
  • uv run mypy . passes

Additional context

Checklist

  • I have checked that there is no similar issue in the repo (required)

Metadata

Metadata

Assignees

Labels

TransformerBridgeBug specific to the new TransformerBridge systembugSomething isn't workingcomplexity-highVery complicated changes for people to address who are quite familiar with the codehelp wantedExtra attention is needed

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions