Skip to content

Inspect & vLLM driver Features & Bugs#1492

Merged
jlarson4 merged 7 commits into
dev-4.xfrom
review/dev-4.x
Jul 7, 2026
Merged

Inspect & vLLM driver Features & Bugs#1492
jlarson4 merged 7 commits into
dev-4.xfrom
review/dev-4.x

Conversation

@jlarson4

@jlarson4 jlarson4 commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Description

Reviewing 4.x for additional interp relevant features that we can add to our two existing drivers

  • vLLM zero-capture footgun. names = list(capture) no longer collapses () to None (which was read as "copy every buffer"). A hookless forward(return_logits=False) now skips the GPU to CPU crossing entirely on both single and batched paths.
  • n_layers guard. num_hidden_layers <= 0 (or non-int) raises at boot instead of leaving a raw {I} template in the non-fireable hook set.
  • weight_access flag. vLLM driver now honestly advertises weight_access (its get_param/tl_get_param RPC is what logit reconstruction already uses).
  • OPT/XGLM mlp_out. SymbolicBridge.mlp.hook_in/hook_out were dead HookPoints on every boot_transformers for fc-split architectures (OPT/XGLM). Caching missed them and interventions silently no-op'd. Fixed with a permanent hook mirror in component_setup._wire_symbolic_hooks, plus Inspect-provider fc-split support (_locate_mlp) and OPT's 2-D FFN handling.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

@jlarson4 jlarson4 merged commit 2bb3c02 into dev-4.x Jul 7, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant