automatic unloading mechanism in Docker Model, similar to the one in Ollama

[brunocarvalhodearaujo]

I am migrating from Ollama to Docker Model, and I’ve noticed that DMR does not seem to automatically unload models when another model is requested, whereas Ollama handled this automatically. This can lead to errors when requesting new models, especially if there are memory or resource limitations in the environment where Docker Model is running.

Segue o exemplo abaixo para ilustrar o problema:

docker model run gpt-oss:20B "Hi"
Thinking:
We need to respond to a simple "Hi". Probably friendly.

docker model run gemma4:12B "Hi"
Failed to generate a response: error response: status=500 body=unable to load runner: error waiting for runner to be ready: llama.cpp terminated unexpectedly: llama.cpp failed: not enough GPU memory to load the model (CUDA)

Verbose output:
0.01.795.624 E ggml_backend_cuda_buffer_type_alloc_buffer: allocating 7010.14 MiB on device 0: cudaMalloc failed: out of memory
0.01.795.630 E alloc_tensor_range: failed to allocate CUDA0 buffer of size 7350663424
0.02.003.518 E llama_model_load: error loading model: unable to allocate CUDA0 buffer
0.02.003.524 E llama_model_load_from_file_impl: failed to load model
0.02.003.529 E cmn  common_init_: failed to load model '/models/bundles/sha256/a3d3a4bff696915699db32fbbb0a29754df3d814d0d577610450a97cb279746a/model/gemma-4-12b-it-UD-Q4_K_XL.gguf'
0.02.003.534 E srv    load_model: failed to load model, '/models/bundles/sha256/a3d3a4bff696915699db32fbbb0a29754df3d814d0d577610450a97cb279746a/model/gemma-4-12b-it-UD-Q4_K_XL.gguf'
0.02.003.536 I srv    operator(): operator(): cleaning up before exit...
0.02.004.449 E srv  llama_server: exiting due to model loading error
bug in the model. its type will be overridden
0.01.426.561 W load: special_eog_ids contains '<|tool_response>', removing '</s>' token from EOG list

It is necessary to use the docker model unload command to manually release the previous model before loading a new one, which is not as convenient as Ollama's automatic behavior. There is an opportunity to implement an automatic unloading mechanism in Docker Model—similar to Ollama's—to improve the user experience and prevent memory issues??