docs(models): qualify full GLM-5.3 on H200 Serverless - #406
Open
earlvanze wants to merge 4 commits into
Open
Conversation
earlvanze
changed the title
earlvanze
marked this pull request as ready for review
September 7, 2026 21:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
Add full
LibertAIDAI/GLM-5.3-NVFP4support on the existingglm_moe_dsapath while preserving GLM-5.2 aliases, with config coverage and measured H200 Serverless evidence. This is full GLM-5.3, not Flash.H200 Serverless qualification
Vast Serverless provisioned one H200 NVL worker. FreeToken 0.1.2 loaded the full checkpoint, allocated 8,384 KV tokens, and captured CUDA graphs for batch sizes 1, 2, 4, and 8. Authenticated routing and worker creation were observed.
Semantic smoke: exact
42, finish reasonstop, 10.402 seconds.At c16, eight requests ran and eight queued. Throughput was effectively saturated at c8. All concurrency responses hit the fixed 128-token cap; this is throughput evidence, not completed-answer quality.
RTX PRO 6000 comparison
The earlier one-RTX PRO 6000 direct-worker screen is retained as preliminary comparison evidence: c8 3.937 and c16 6.297 aggregate output tok/s with 32-token caps, all truncated during reasoning.
Verification and limits
Model readiness, authenticated Serverless inference, worker creation, and semantic correctness are verified. Idle stop and cached restart remain unverified because the instance was deleted. No complete lifecycle or per-request throughput claim.
The working-tree test run passed 541 tests, including the new config coverage. A separately prepared GLM reasoning-parser fix remains out of scope for this PR.
AI-assisted under operator direction; raw H200 and RTX PRO 6000 evidence is included for review.