Skip to content

Fix ART runtime compatibility with vLLM 0.25#776

Merged
bradhilton merged 1 commit into
mainfrom
agent/vllm-runtime-compatibility
Jul 25, 2026
Merged

Fix ART runtime compatibility with vLLM 0.25#776
bradhilton merged 1 commit into
mainfrom
agent/vllm-runtime-compatibility

Conversation

@bradhilton

@bradhilton bradhilton commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • support both the vLLM 0.23/0.24 OpenAIServing layout and the vLLM 0.25 GenerateBaseServing layout when installing ART's LoRA update coordinator
  • fail closed when an unexpected nested import is missing instead of masking an incompatible runtime
  • execute the CPU-safe compatibility patch in ART's release smoke check

Root cause

Caladan's DeepSeek V4 image intentionally installs art-vllm-runtime without dependencies alongside vLLM 0.25.1. vLLM 0.25 moved the shared serving constructor from vllm.entrypoints.openai.engine.serving.OpenAIServing to vllm.entrypoints.generate.base.serving.GenerateBaseServing. ART imported only the old location, so the production canary crash-looped before model startup.

Validation

  • focused runtime layout/import/plugin tests: 3 passed
  • related DSV4/runtime patch tests: 4 passed
  • direct source checks against vLLM 0.23 and vLLM 0.25.1
  • Ruff, format, ty, lock check, and pre-commit passed
  • Caladan Make ./.art default root path when not in git repo #89 disposable DeepSeek vLLM 0.25.1 image build executed the focused coordinator patch successfully before GPU startup

The full runtime-isolation file has 9 passing tests and 4 unrelated host-environment failures caused by vLLM INFO output contaminating JSON and GPU-sensitive TileLang imports on CPU.

@bradhilton
bradhilton marked this pull request as ready for review July 25, 2026 15:22
@bradhilton
bradhilton merged commit 42ca789 into main Jul 25, 2026
7 checks passed
@bradhilton
bradhilton deleted the agent/vllm-runtime-compatibility branch July 25, 2026 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant