Skip to content

Win: Use DXGI for accurate WDDM VRAM memory budget - #4457

Open
dhiltgen wants to merge 3 commits into
ml-explore:mainfrom
dhiltgen:win-wddm-memory-budget
Open

Win: Use DXGI for accurate WDDM VRAM memory budget#4457
dhiltgen wants to merge 3 commits into
ml-explore:mainfrom
dhiltgen:win-wddm-memory-budget

Conversation

@dhiltgen

@dhiltgen dhiltgen commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

WDDM implements VRAM paging, which can result in a performance cliff at the limit. This wires up DXGI to expose more accurate VRAM budget.

  • ☑️ I understand it is strictly prohibited to use AI to write PR description
  • AI usage disclosure: co-developed with a coding agent

@dhiltgen

dhiltgen commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

This slightly contrived example runs mlx-lm and evicts old caches under memory pressure.
On a 5090 using Llama-3.2-3B-Instruct-4bit, you can toggle by adding/removing --retain-all

--conversations 31 --prompt-tokens 8192 --generation-tokens 4 --prefill-step-size 512 --reserve-gib 2 --retain-all

mlx-lm-wddm-cache.py

@dhiltgen dhiltgen changed the title Use DXGI for accurate WDDM VRAM memory budget Win: Use DXGI for accurate WDDM VRAM memory budget Sep 3, 2026
@zcbenz
zcbenz force-pushed the win-wddm-memory-budget branch from 006767f to c2fccfd Compare September 7, 2026 23:48
Comment thread mlx/backend/cuda/windows_memory.cpp Outdated
@dhiltgen
dhiltgen force-pushed the win-wddm-memory-budget branch from c2fccfd to 3f5473a Compare September 8, 2026 21:38
@zcbenz
zcbenz force-pushed the win-wddm-memory-budget branch from 3f5473a to 31991cf Compare September 11, 2026 00:51
@zcbenz
zcbenz force-pushed the win-wddm-memory-budget branch from 31991cf to 3384e43 Compare September 11, 2026 00:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants