build(deps): update trl requirement from <=0.21.0 to <=1.10.0

[dependabot]

Updates the requirements on trl to permit the latest version.

Release notes

Sourced from trl's releases.

v1.10.0

Features

🎓 DistillationTrainer is now a stable trainer

After a ~30-PR refactor that reshaped its data contract, loss path, generation stack, config surface, and tests, DistillationTrainer and DistillationConfig graduate from trl.experimental.distillation to the top-level trl package. Same import surface as SFT / DPO / GRPO / KTO. The old experimental path still works and emits a FutureWarning (removal in v2.0.0).

# Before
from trl.experimental.distillation import DistillationConfig, DistillationTrainer
Now
from trl import DistillationConfig, DistillationTrainer

Also lands a trl distillation CLI and moves the tests to tests/. The full refactor arc: switched signature columns to prompt (deprecated messages-format), pinned the generation stack to GRPO's, wired the chunked JSD loss and deleted the full-logit path, cleaned the Liger path to share extraction with the chunked path, adopted GRPO's log() / training_step timing / _save_checkpoint, rebuilt the test suite to GRPO shape, and much more.

by @​qgallouedec across ~30 PRs (#6479, #6480, #6481, #6482, #6484, #6487, #6497, #6508, #6509, #6510, #6511, #6512, #6513, #6521, #6522, #6523, #6524, #6525, #6526, #6530, #6537, #6604, #6605, #6606, #6607, #6609, #6610, #6611, #6612, #6613, #6614, #6629, #6630, #6632, #6633, #6634, #6639, #6640, #6641, #6642, #6643, #6644, #6645, #6647, #6653).

DistillationTrainer supports Vision Language Models

Alongside the graduation, VLMs work end-to-end in DistillationTrainer.

by @​qgallouedec in huggingface/trl#6650

AsyncGRPO: loop-owning agent training (OpenEnv harness)

New experimental loop-owning (black-box) path for AsyncGRPOTrainer — for training external agents like opencode that run their own tool loop, rather than TRL driving each turn as in the environment_factory white-box path.

The agent runs in an OpenEnv session in transparent_proxy mode; an in-sandbox proxy captures each turn's token ids and logprobs. On completion, TRL reads the trace, rebuilds per-turn training rows, and scores the workspace with the session's verify(). Ships with three caller-supplied policy hooks (rollout_reward_fn, train_turn_fn, agent_turn_fn) so you can drop framework aux calls (title generator, context summarizer) and reinforce only the turns you want.

Includes a self-contained examples/scripts/openenv/opencode.py (local subprocess sandbox + DeepCoder held-out stdin/stdout verifier), validated end-to-end on Qwen3-4B.

by @​AmineDiro in huggingface/trl#6420, plus HF-sandbox variant by @​sergiopaniego in huggingface/trl#6565

AsyncGRPO: observability + sampling controls

• Record rollout traces to trackio by @​AmineDiro in huggingface/trl#6491
• Step-time metric by @​AmineDiro in huggingface/trl#6490
• Fix queue wait time metric by @​AmineDiro in huggingface/trl#6489
• top_p / top_k / min_p / repetition_penalty in AsyncGRPOConfig by @​AmineDiro in huggingface/trl#6608
• Fork-independent epoch counting — landed in v1.9 window
• Close in-flight harness sessions concurrently on shutdown by @​sergiopaniego in huggingface/trl#6564

DiffusionGemma block-diffusion SFT example

A new SFT example for google/diffusiongemma-26B-A4B-it implementing the reference recipe: one response block per step, uniform random token corruption with t ~ U(eps, 1), two-pass self-conditioning, flat cross-entropy over the whole canvas plus an autoregressive co-loss on the encoder. LoRA targets attention + dense MLP linears. Requires transformers >= 5.12.0.

Ships with diffusion_gemma.jinja / diffusion_gemma_training.jinja chat templates (with {% generation %} markers) so assistant_only_loss=True works out of the box.

... (truncated)

Commits

• a7be897 Release: v1.10 (#6724)
• 0ee92b9 [DistillationTrainer refactor] Docs: announce the DistillationTrainer promoti...
• c16d094 [DistillationTrainer refactor] Support Vision Language Models (#6650)
• 0b5709a [DistillationTrainer refactor] Re-point ServerDistillationTrainer at the stab...
• 3acf31b [DistillationTrainer refactor] Update the example script for the stable API (...
• 5087db7 [DistillationTrainer refactor] Docs: de-namespace DistillationTrainer for the...
• 11250f0 [DistillationTrainer refactor] Move the distillation tests to tests/ (#6647)
• aacb260 [DistillationTrainer refactor] Add the trl distillation CLI (#6642)
• 2f2f0a8 [DistillationTrainer refactor] Move DistillationTrainer to the Stable telemet...
• ef0a079 [DistillationTrainer refactor] Promote DistillationTrainer to the stable API ...
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)