Cut idle embedding memory without changing nomic quality - #8

Open

TorstenDittmann wants to merge 11 commits into

torsten/idle-embedding-memory-91e8

Commits on Sep 15, 2026

fix: cut idle embedding RSS without changing nomic quality

cursoragent
and
TorstenDittmann
committed
283fb39
fix: route ONNX allocations through jemalloc so idle unload returns RSS

cursoragent
and
TorstenDittmann
committed
616f526
fix: install make in the Docker builder so jemalloc can compile

cursoragent
and
TorstenDittmann
committed
2e2099d
fix: keep a RAM-capped session pool while the model is loaded

cursoragent
and
TorstenDittmann
committed
7de5851

Commits on Sep 16, 2026

fix: default to one ONNX session and cover idle/nomic e2e

cursoragent
and
TorstenDittmann
committed
926c437
fix: split intra-op threads across every pool session

cursoragent
and
TorstenDittmann
committed
66c141d
fix: warmup the rebuilt serving session after a RAM cap

cursoragent
and
TorstenDittmann
committed
6641232
test: drop helper-only pool default assertions

cursoragent
and
TorstenDittmann
committed
3e5806f
fix: keep tokio workers free during model load

cursoragent
and
TorstenDittmann
committed
05a38d7
fix: expire health 503 and load models on spawn_blocking

cursoragent
and
TorstenDittmann
committed
dad9f7a
fix: keep model-load ownership if /embed is cancelled

cursoragent
and
TorstenDittmann
committed
8ac6006