Skip to content

[AIGTWY-4710] Keep managed models across config transitions AIGTWY-4786 - #709

Merged
david-siqi-liu merged 2 commits into
mainfrom
david/aigtwy-4710-model-lifecycle
Sep 18, 2026
Merged

david-siqi-liu merged 2 commits into
mainfrom
david/aigtwy-4710-model-lifecycle

Conversation

@david-siqi-liu

@david-siqi-liu david-siqi-liu commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

Keeps a coding agent's managed model settings in sync with the managed config across workspace transitions (AIGTWY-4710). Switching between managed configs reconciles each agent's model settings to the current config, and configuring a workspace with no managed config clears ug's managed model settings so an unmanaged workspace never enforces a stale list inherited from a previous managed workspace.

Lifecycle exercised, per agent (functional in-process tests plus end-to-end integration journeys):

  • no managed config: no picker (Claude) / no catalog (Codex).
  • static config A: the picker/catalog lists exactly A's models.
  • static config B: reconciles to B; models A listed that B omits are pruned.
  • model discovery via a Model Provider Service: the picker/catalog is cleared, since the Databricks-Model-Provider-Service header routes and there is no static list.
  • no managed config again: the picker/catalog is cleared, not inherited.

Changes:

  • Claude clears the managed picker (availableModels, enforceAvailableModels, modelPicker) whenever ug is not writing a static list this run (an unmanaged workspace or a provider switch). It always clears the picker in its own ucode-settings.json. In the OS managed-settings.json the picker is shared with the administrator, so ug treats it as one unit and reverts it to its pre-ucode baseline only when the live picker is unchanged since ug last wrote it (a three-way merge of baseline, ug's last write, and the live file). That way ug removes only what it introduced and never an administrator's own picker, a picker they added after ug cleared its own, or one they later edited (the enforce flag included).
  • Codex clears its generated catalog and model_catalog_json pointer on an unmanaged configure or a provider switch, so an unmanaged workspace uses its own discovery. A managed static config writes them; the launch-time model pin follows Codex's existing clear-when-unmanaged behavior.
  • Prettifies the Claude /model picker labels: the enforced allow-list still shows one row per model (it can include multiple versions per family and non-Claude models), but each label is now title-cased with a dotted version and known acronyms upper-cased (system.ai.claude-haiku-4-5 shows as Claude Haiku 4.5, system.ai.glm-5-3 as GLM 5.3) instead of the raw id. The routable id still shows on the right of each row.
  • The UCODE_MANAGED_CONFIG_STUB test hook accepts an explicit JSON null to reproduce a workspace with no managed config, distinct from an unset or unreadable stub (which still falls through to the real HTTP read).
  • Fixes a pre-existing bug where a managed config's per-family default models were dropped. managed_resolve._manifest_models read a stale models key instead of the normalized default_models_by_model_family, so managed_claude_family_models returned nothing and no ANTHROPIC_DEFAULT_<family>_MODEL was written (only the overall default and the picker survived, on separate paths). Unit fixtures that hand-built the stale shape masked it; the fix corrects them and adds a regression test that builds through normalize_managed_config so the shape can't drift again.
  • Reconciles the per-family default env keys the same three-way way as the picker so a workspace switch does not carry stale defaults over. Each ANTHROPIC_DEFAULT_<family>_MODEL resolves as config, then an administrator-authored value (one that differs from ug's last write, preserved verbatim), then discovery; a value ug itself wrote is re-derived from the new workspace rather than preserved, and any family default whose model is not in the enforced availableModels list is dropped (e.g. a discovered fable default when the admin's list omits fable).
  • Adds functional lifecycle tests (both agents) covering the full sequence above, including the per-family default env keys landing (opus and sonnet carry ug's [1m] context tag), per-agent managed_fixture integration journeys running it through ug configure against the managed e2e workspace, and coverage-matrix updates. The integration MPS states point at real provider services on that workspace: main.default.ci_e2e_anthropic_mps (Claude) and main.default.ci_e2e_openai_mps (Codex).

What Claude Code writes for models, by managed model source:

  • static (model_services): sets availableModels, enforceAvailableModels, and modelPicker in ~/.claude/ucode-settings.json and the OS managed-settings.json. The managed default is applied at launch as env.ANTHROPIC_MODEL.
  • model discovery via MPS (model_provider_service): writes no model list and clears ug's static picker; the Databricks-Model-Provider-Service header routes and Claude shows its own model names. A Bedrock-backed provider pins ANTHROPIC_DEFAULT_{OPUS,SONNET,HAIKU}_MODEL from the manifest's family slots; an Anthropic provider pins nothing.
  • model discovery via unity_catalog_location: not consumed on main (schema only; no fields written).

What Codex writes for models, by managed model source:

  • static (model_services): writes the catalog file ~/.ucode/codex-model-catalog.json and points ucode.config.toml's model_catalog_json at it. The managed default is pinned as model in ucode.config.toml.
  • model discovery via MPS (model_provider_service): writes no catalog and clears any static catalog; the Databricks-Model-Provider-Service header routes and Codex discovers models at launch.
  • model discovery via unity_catalog_location: not consumed on main (schema only; no fields written).

Known limitations (out of scope here):

  • Clearing on an unmanaged configure also resets in the rare case where the same workspace's managed config genuinely disappeared, since ug does not distinguish that from an intentional switch to an unmanaged workspace.
  • On a non-interactive configure (no sudo, managed_writes_allowed() False), ug cannot rewrite the root-owned OS managed-settings.json, so a stale picker there is not cleared in that path; the private ucode-settings.json still is. The interactive flow clears both.
  • If ug's managed-file snapshots (its pre-ucode baseline or last write) are unreadable, ug leaves the managed-file picker in place rather than risk clobbering an admin's entry.

This pull request and its description were written by Isaac.

@david-siqi-liu david-siqi-liu added the quicktree-assisted PR created with quicktree-assisted workflow label Sep 17, 2026
@david-siqi-liu
david-siqi-liu marked this pull request as ready for review September 17, 2026 19:06
@david-siqi-liu
david-siqi-liu force-pushed the david/aigtwy-4710-model-lifecycle branch 12 times, most recently from 181ae42 to d5eb211 Compare September 17, 2026 23:36
Comment thread src/ucode/managed_files.py Outdated
Comment thread src/ucode/managed_files.py Outdated
Comment thread src/ucode/agents/claude.py Outdated
Comment thread src/ucode/agents/claude.py Outdated
@lilly-luo

lilly-luo commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

picker + available model switching is working well for cc. however it doesn't look like the cc family defaults are getting propagated. see https://eng-ml-agent-platform.staging.cloud.databricks.com/ .

is this a known limitation ?

Screenshot 2026-09-17 at 9 54 59 PM Screenshot 2026-09-17 at 9 55 36 PM

i see this in my managed settings but it doesn't match what's in the config, except for the default. moreover, i didn't define FABLE in the managed config

    "ANTHROPIC_DEFAULT_OPUS_MODEL": "system.ai.claude-opus-4-8[1m]",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "system.ai.claude-sonnet-4-6[1m]",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "system.ai.claude-haiku-4-5",
    "ANTHROPIC_DEFAULT_FABLE_MODEL": "system.ai.claude-fable-5-1",
    "ANTHROPIC_MODEL": "system.ai.claude-sonnet-5"

@david-siqi-liu
david-siqi-liu force-pushed the david/aigtwy-4710-model-lifecycle branch 2 times, most recently from 26caa1e to adc7d84 Compare September 18, 2026 02:49
… workspace

Keeps each agent's managed model settings in sync with the managed config:
switching between managed configs reconciles to the current one, and
configuring a workspace with no managed config clears ug's managed model
settings so an unmanaged workspace never enforces a stale list carried over
from a previous managed workspace.

- Claude clears the managed picker (availableModels, enforceAvailableModels,
  modelPicker) when not writing a static list. It clears the whole picker in
  its own ucode-settings.json. In the OS managed-settings.json the picker is
  shared with the administrator, so ucode reverts it to its pre-ucode baseline
  only when the live picker is unchanged since ucode last wrote it (a three-way
  merge of baseline, last write, and live, via a ManagedFileSnapshots value).
- Fixes a pre-existing bug where a managed config's per-family default models
  were dropped: managed_resolve._manifest_models read a stale `models` key
  instead of the normalized `default_models_by_model_family`, so no
  ANTHROPIC_DEFAULT_<family>_MODEL was written. Corrects the unit fixtures that
  masked it and adds a regression test through normalize_managed_config.
- Reconciles the per-family default env keys three-way like the picker so a
  workspace switch drops stale defaults: config, then an admin-authored value
  (differs from ucode's last write, preserved), then discovery; a value ucode
  itself wrote is re-derived from the new workspace, and a default whose model
  is not in the enforced availableModels list is dropped (e.g. a discovered
  fable default when the admin's list omits fable).
- Codex clears its catalog and model_catalog_json pointer on an unmanaged
  configure or a provider switch; a managed static config writes them.
- The UCODE_MANAGED_CONFIG_STUB test hook accepts an explicit null to
  reproduce a workspace with no managed config.
- Adds functional lifecycle tests (both agents) and per-agent managed_fixture
  integration journeys covering no config -> static A -> static B -> MPS ->
  no config, plus per-family default coverage and coverage-matrix updates.

Co-authored-by: Isaac <no-reply@databricks.com>
@david-siqi-liu
david-siqi-liu force-pushed the david/aigtwy-4710-model-lifecycle branch from adc7d84 to 42f241a Compare September 18, 2026 03:10
@lilly-luo

lilly-luo commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

lgtm!

Screenshot 2026-09-17 at 11 17 38 PM Screenshot 2026-09-17 at 11 18 06 PM

@david-siqi-liu
david-siqi-liu enabled auto-merge (squash) September 18, 2026 03:18
@lilly-luo lilly-luo changed the title [AIGTWY-4710] Keep managed models across config transitions [AIGTWY-4710] Keep managed models across config transitions AIGTWY-4786 Sep 18, 2026
@david-siqi-liu
david-siqi-liu merged commit 141846f into main Sep 18, 2026
20 checks passed
@david-siqi-liu
david-siqi-liu deleted the david/aigtwy-4710-model-lifecycle branch September 18, 2026 03:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

quicktree-assisted PR created with quicktree-assisted workflow

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants