[AMD][MI300X] Extend GPT-OSS FP4 TP=8 search to conc=1 (extends interactivity frontier to ~249 tps/user)

[ramineroane]

Summary

Extends the TP=8 concurrency search for gptoss-fp4-mi300x-vllm from [4..16] to [1..16], adding a single new low-concurrency point that pushes the interactivity Pareto frontier rightward.

Motivation

The current best 1k/1k single-user point on the InferenceX gpt-oss-120b vLLM FP4 MI300X frontier is ~234 tokens/sec/user (TP=8 conc=4 → 219 TPS/GPU). The latency-optimal regime (smallest M, all GPUs, no batching) was not being measured.

Adding conc=1 captures that regime and gives the dashboard a true low-latency endpoint for users prioritizing interactive single-user use cases (chat, copilot, agentic).

Measured result

Single MI300X node, image vllm/vllm-openai-rocm:v0.17.0. Bench harness: benchmarks/single_node/gptoss_fp4_mi300x.sh (no script change needed — it already accepts conc=1).

Server flags (unchanged from existing config):

--attention-backend ROCM_AITER_UNIFIED_ATTN
-cc.pass_config.fuse_rope_kvcache=True
-cc.use_inductor_graph_partition=True
--tensor-parallel-size=8
--gpu-memory-utilization 0.95
--max-model-len 2248
--block-size=64
--no-enable-prefix-caching

Bench params: --random-input-len 1024 --random-output-len 1024 --random-range-ratio 0.8 --num-prompts 64 --max-concurrency 1 --num-warmups 1 --ignore-eos.

Metric	Value
Total TPS	493
Output TPS	247
TPS/GPU	61.7
Mean ITL	4.02 ms
Tokens/sec/user	248.9
Mean TTFT	33.6 ms
Completed	64/64

Pareto positioning

Frontier point	tps/user	tps/gpu
TP=8 conc=4 (existing best in interactivity dir.)	234.5	219
TP=8 conc=1 (this PR)	248.9	62

Net effect: a new datapoint at the high-interactivity tail; no existing points are displaced.

Impact on CI

• Adds a single conc point to the existing TP=8 sweep for gptoss-fp4-mi300x-vllm.
• 8k/1k row updated symmetrically.
• No script, image, or harness changes. Only .github/configs/amd-master.yaml.

cc @seungrokj @functionstackx @chunfangamd

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[chunfangamd]

lgtm

[chunfangamd]

@claude /sweep test-config --config-files .github/configs/amd-master.yaml --config-keys gptoss-fp4-mi300x-vllm

[Klaud-Cold]

Claude finished @chunfangamd's task in 0s —— View job

---

I'll analyze this and get back to you.

[seungrokj]

/sweep test-config --config-files .github/configs/amd-master.yaml --config-keys gptoss-fp4-mi300x-vllm

[github-actions]

@seungrokj Kicking off a sweep.

Run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/24657932304
Command: test-config --config-files .github/configs/amd-master.yaml --config-keys gptoss-fp4-mi300x-vllm
Pinned ref: 699643e
Approval: not required (trusted collaborator).

[seungrokj]

@claude @Klaud-Cold update perf-changelog.yaml

[seungrokj]

[seungrokj]

Lgtm

[seungrokj]

@functionstackx @cquil11 can you plz merge this?

[functionstackx]

@ramineroane @seungrokj at the new interactivity ranges, h100 beats mi300 of 210 to 250 tok/s/user. any plans for imrpovement?

[functionstackx]

approved. feel free to merge