0.20.28

Endpoints

Presets

Inference optimization for a specific model and hardware requires the use of agents. This update introduces a dstack primitive called endpoint presets that automates and streamlines the use of agents to build and reuse optimized inference configurations.

To create a preset, define an endpoint configuration:

type: endpoint
name: qwen25-7b

model:
  base: Qwen/Qwen2.5-7B-Instruct

env:
  - HF_TOKEN

Then, pass it todstack endpoint preset create:

dstack endpoint preset create -f endpoint.dstack.yml
[2026-07-15 11:32:01] Starting endpoint preset creation for Qwen/Qwen2.5-7B-Instruct. Allowed fleets: gpu-fleet.
[2026-07-15 11:41:06] Prototype task qwen25-7b-a1b2c3-2 verified vLLM on an L4:24GB.
[2026-07-15 11:52:06] Final service qwen25-7b-a1b2c3-3 verified with context length 32768.
[2026-07-15 11:52:18] Benchmark via guidellm 0.7.1: 32/32 requests succeeded.
[2026-07-15 11:52:18] Saved endpoint preset 8f3a12c4 for Qwen/Qwen2.5-7B-Instruct.

This command uses the locally installed claude CLI along with dstack's bundled skills. The agent uses dstack tasks to find the best serving configuration for the available fleet offers. It then submits the configuration as a dstack service for a final benchmark.

Use dstack endpoint preset to list existing presets:

dstack endpoint preset list
 MODEL                     GPU                    CONTEXT  BENCHMARK                           CREATED
 Qwen/Qwen2.5-7B-Instruct
    preset=8f3a12c4        nvidia:16GB..24GB:1..  32K      concurrency=1 464 tok/s TTFT 312ms  1 hour ago

To deploy a preset as a service, pass the endpoint configuration to dstack endpoint preset apply.

Note

The endpoints feature is experimental and may change.

Roadmap

Here's what's coming soon in endpoint presets:

• Support multiple trials, allowing the agent to improve benchmark results based on previous trials.
• Allow the endpoint configuration to define custom agent instructions, such as target metrics and the experimentation approach.
• Control which experiments the agent may perform, including modifying serving framework code and generating custom kernels.

See the documentation for more details.

Runs

dstack inside dstack

Sometimes, runs may need to call dstack themselves, for example, to inspect runs, submit another run, or attach to one.

This update makes this easy:

type: task
image: dstackai/dstack
dstack: true
env:
  - DSTACK_TOKEN
commands:
  - dstack ps

In this case, dstack exposes mounts /run/dstack/server.sock inside the container and sets DSTACK_SERVER_URL and DSTACK_PROJECT accordingly.

This gives the run access to the dstack server, allowing it to manage existing runs, submit new ones with dstack apply, and attach to them with dstack attach.

Authentication remains explicit, so DSTACK_TOKEN must be passed through env or secrets.

See documentation for more details.

Server

OpenTelemetry

The dstack server now integrates with OpenTelemetry SDK for exporting server traces, logs, and metrics. This can be used for monitoring the server via a LGTM (Grafana) observability stack or any other solution supporting OLTP. See documentation for more details.

Backends

Vast.ai spot instances

The vastai backend now supports interruptible spot instances, offering more competitive prices.

$ dstack offer -b vastai --gpu B200,B300 --spot-auto
 #   BACKEND               RESOURCES                                              INSTANCE TYPE  PRICE
 1   vastai (us-virginia)  cpu=96 mem=510GB disk=100GB gpu=B200:180GB:4 (spot)    45069637       $25.1215
 2   vastai (us-virginia)  cpu=96 mem=510GB disk=100GB gpu=B200:180GB:4           45069637       $28.6215
 3   vastai (ca-)          cpu=192 mem=2059GB disk=100GB gpu=B300:270GB:8 (spot)  43033760       $40.0694
 4   vastai (ca-)          cpu=192 mem=2059GB disk=100GB gpu=B300:270GB:8         43033760       $62.5694
 5   vastai (us-virginia)  cpu=192 mem=2043GB disk=100GB gpu=B200:180GB:8 (spot)  45069635       $50.1215
 6   vastai (us-virginia)  cpu=192 mem=2043GB disk=100GB gpu=B200:180GB:8         45069635       $57.1215
 7   vastai (us-oregon)    cpu=24 mem=31GB disk=100GB gpu=B200:180GB:1 (spot)     33945613       $5.651
 8   vastai (us-oregon)    cpu=24 mem=31GB disk=100GB gpu=B200:180GB:1            33945613       $6.901
 9   vastai (us-virginia)  cpu=48 mem=127GB disk=100GB gpu=B200:180GB:2 (spot)    45081639       $15.1215
 10  vastai (us-virginia)  cpu=48 mem=127GB disk=100GB gpu=B200:180GB:2           45081639       $16.3715

What's changed

• Ignore cachetools-related typing error in AWS backend by @un-def in #4022
• Fix Git.ls_remote() return type error by @un-def in #4024
• Pass effective Requirements to Compute's run_job/run_jobs by @un-def in #4020
• Kubernetes: Rework offers and resource handling by @un-def in #4021
• Do not ignore Vast.ai instance termination errors by @jvstme in #4030
• Migrate off deprecated Vast.ai /api/v0/instances by @jvstme in #4031
• [Feature] Supporting dstack-inside-dstack for runs by @peterschmidt85 in #4023
• Vast.ai: Support spot instances by @trashhalo in #4025
• Update Vast.ai spot instances support by @jvstme in #4036
• Update gateway connection error log level by @jvstme in #4035
• Detect Vast.ai interruptions during provisioning by @jvstme in #4037
• Support server monitoring with OpenTelemetry by @r4victor in #4034
• Update VastAIAPIClient by @jvstme in #4038
• [Feature] Endpoint presets by @peterschmidt85 in #4028

New contributors

• @trashhalo made their first contribution in #4025

Full changelog: 0.20.27...0.20.28