Add Llama 3 Megatron model support

[FurtherAI]

Summary

• add a minimal Llama 3 dense model handler and declarative registry coverage for text-only Llama 3, 3.1, 3.2, and 3.3 checkpoints
• use the stock Megatron Bridge provider and existing dense LoRA path, including native vLLM LoRA export and serving
• generalize packed prefix-tree rotary validation and stabilize symbolic context-parallel FlexAttention stage execution
• align model-support trainability and train/inference fixtures with their production runtime contracts

Implementation

The Llama-specific production surface is a 21-line handler plus registry entries. Generic changes repair or improve existing shared paths:

• restore dense FC2 LoRA export through the wrapped module
• keep identity-LoRA creation CUDA-free
• make CP range kernels polymorphic and prewarm their general forms
• normalize and stabilize sparse Flex stage layouts while retaining symbolic query/key lengths
• construct controlled train/inference parity trajectories explicitly, avoiding auto-capture semantics in the workflow oracle

The FlexAttention change preserves the existing compile configuration and invokes PyTorch's same higher-order operator after ART validation. It removes exact sequence-length guards introduced by the public Python wrapper, which previously caused workload-dependent late graph specialization.

Validation

Model: meta-llama/Llama-3.2-1B-Instruct (full 16-layer, 1.241B-parameter checkpoint where applicable).

• FP32 Hugging Face parity: output 0.005352%, loss 0.002253%, worst gradient 0.042681% mean absolute percentage error
• LoRA coverage/export: 7/7 targets
• distributed correctness: all 12 RL/SFT TP/DP/CP variants pass, including TP2/DP2/CP2; worst gradient 0.000840% RL and 0.000800% SFT
• packed/unpacked arbitrary prefix-tree parity: all four scenarios pass; worst logits 0.010725%
• sensitivity: all 24 applicable injected faults detected
• native vLLM LoRA: step-0 and step-1 adapters load, coexist, and serve; merged-weight serving also passes
• train/inference parity: 2.428707% mean absolute percentage error and 0.001223025 top-20 KL over 128 tokens and eight depth-3 branch views, under unchanged 4% / 0.002 gates
• length trainability: mean absolute length error reaches 0.75, under the fixed 1.5 threshold
• full-model E2E: 20 optimizer/update/serve steps at 122,880 packed tokens, 888.745 tok/s, with zero late Dynamo graphs or Inductor kernels
• latest-main preflight: 85 focused tests pass; changed-file Ruff, format, ty, lock, and diff checks pass

Scope

The branch is rebased onto main at 93f3832a. It adds 335 lines and removes 112 lines across 18 files. Model-specific behavior stays in the handler and registry; generic changes have no Llama conditionals.