Skip to content

Resolve deployment-owned vLLM serving plans for Slurm #866

Description

@nabinchha

Priority Level

High

Task Summary

Implement the package-owned vLLM resolver that converts one typed deployment declaration plus planner-supplied placement into process, readiness, backend-endpoint, and logical-endpoint records.

Technical Details & Implementation Plan

  • Dispatch on the deployment server discriminator with vLLM as the only implemented backend.
  • Keep model alias, model source, and optional served model name distinct.
  • Resolve tensor parallelism within a node and pipeline parallelism across nodes in a replica group.
  • Derive replica groups, ranks, lanes, rendezvous inputs, and lane-head backend endpoints from planner-owned placement.
  • Carry per-deployment startup and distributed-initialization deadlines, launch standoff/stagger, queue backpressure, readiness configuration, tokenized extra arguments, and typed environment bindings.
  • Reject extra arguments that override compiler/runtime-owned model identity, ports, topology, rendezvous, launch timing, or middleware behavior.
  • Emit typed logical-endpoint inputs so the runtime can aggregate readiness and load-balance healthy backends while preserving overload responses.
  • Map inspected vLLM runtime versions to the supported single-node, multi-node, readiness, and backpressure behavior.

Acceptance criteria

  • A single-node deployment resolves deterministically without importing scheduler or shell modules.
  • A multi-node deployment derives the expected replica groups, ranks, pipeline parallelism, and lane-head endpoints.
  • Two deployments may select different compatible serving images without endpoint or resource identity collisions.
  • Invalid node, GPU, tensor-parallel, replica-group, expert-parallel, and backend-version combinations fail before submission.
  • Default and overridden launch-timing and queue-backpressure values serialize in golden records.
  • A rank failure is represented as failure of the coordinated deployment; no per-replica restart contract is introduced.

Out of scope

  • Slurm submission or node allocation.
  • Shell rendering, process supervision, and cleanup.
  • Image building or registry mutation.
  • Dynamo, SGLang, serving plugins, and per-replica recovery.

Investigation / Context

This is a feature lane under #850. Portable deployment intent remains separate from backend-specific process resolution so configuration stays declarative.

Agent Plan / Findings

Build against the reviewed shared configuration, image-inspection, placement, and runtime record contracts. Keep the resolver pure with focused single-node and multi-node golden tests.

Dependencies

Blocked by the shared-contract and fake-infrastructure foundation work tracked by #850.

Metadata

Metadata

Assignees

Labels

taskInternal development task

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions