Skip to content

docs: add llm-d multi-node inference guide - #304

Merged
EdisonSu768 merged 2 commits into
masterfrom
docs/llm-d-multinode-inference
Aug 10, 2026
Merged

docs: add llm-d multi-node inference guide#304
EdisonSu768 merged 2 commits into
masterfrom
docs/llm-d-multinode-inference

Conversation

@EdisonSu768

Copy link
Copy Markdown
Member

Summary

  • explain the boundary between llm-d routing, LeaderWorkerSet workload management, and vLLM/NCCL model parallelism
  • add a reusable two-node, four-GPU-per-node LLMInferenceService template using TP=4 and PP=2
  • document prerequisites, deployment, real-request verification, and common scheduler, CRD, NCCL, backend, and KV-cache failures
  • link the guide from the Inference Service and vLLM Expert Parallel pages

Validation

  • yarn lint
  • yarn build
  • parsed the two-document YAML template
  • validated both embedded vLLM launch scripts with bash -n

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 8, 2026

Copy link
Copy Markdown

Deploying alauda-ai with  Cloudflare Pages  Cloudflare Pages

Latest commit: 52566a8
Status: ✅  Deploy successful!
Preview URL: https://fec14e46.alauda-ai.pages.dev
Branch Preview URL: https://docs-llm-d-multinode-inferen.alauda-ai.pages.dev

View logs

@EdisonSu768
EdisonSu768 merged commit 77b57ad into master Aug 10, 2026
2 checks passed
@EdisonSu768
EdisonSu768 deleted the docs/llm-d-multinode-inference branch August 10, 2026 02:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant