Skip to content

feat(requesty): add automated model catalog sync#3233

Open
mrubens wants to merge 4 commits into
anomalyco:devfrom
mrubens:codex/requesty-model-sync-v2
Open

feat(requesty): add automated model catalog sync#3233
mrubens wants to merge 4 commits into
anomalyco:devfrom
mrubens:codex/requesty-model-sync-v2

Conversation

@mrubens

@mrubens mrubens commented Jul 13, 2026

Copy link
Copy Markdown

Summary

  • add Requesty to the automated aggregator sync using its public model catalog
  • factor supported models against canonical metadata while syncing Requesty-specific pricing, limits, and capabilities
  • expose GPT-5.4, GPT-5.5, and GPT-5.6 routes through Requesty's native Responses API so reasoning and function tools work together
  • preserve the Gemini image model's intrinsic lack of tool support and omit the non-agent Nemotron content-safety classifier
  • retain curated Requesty entries that are absent from the public response

This follows the catalog-sync approach from #2805 and incorporates its review feedback: inherited modalities are preserved, intrinsic capabilities are not disabled by incomplete provider flags, reasoning models declare provider controls, and normal model comparison allows future syncs to update existing files.

The GPT chat-completions routes were replaced with their exact openai-responses/... catalog IDs because Requesty's chat endpoint rejects function tools when reasoning effort is enabled. The Responses endpoint supports both together.

Sources

Validation

  • bun test packages/core/test/sync.test.ts (48 passed)
  • bun ./packages/core/script/validate.ts
  • bun ./packages/core/script/sync-models.ts requesty --dry-run (85 unchanged)
  • cd packages/web && bun run build
  • OpenCode live checks against the built catalog:
    • all 46 added tool-capable models completed a Bash tool call
    • the non-tool Gemini image model completed a tool-free request
    • all six openai-responses/... additions completed tool calls with both default and high reasoning effort

@mrubens mrubens closed this Jul 13, 2026
@mrubens mrubens reopened this Jul 13, 2026
@mrubens mrubens marked this pull request as ready for review July 13, 2026 11:59
@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [low] [possible mistake] packages/core/src/sync/providers/requesty.ts:80 - Check: parseModels must pass openai-responses/ models through to translateModel for the sync to manage them automatically. Why: The api filter (model.api ?? "chat") !== "chat" drops any model whose api field is neither "chat" nor absent. The six openai-responses/... routes are served via Requesty's Responses API, not its Chat Completions endpoint; if the /v1/models response tags those IDs with api: "responses" (or any non-"chat" value), they are silently excluded before requestyBaseModel or buildRequestyModel ever runs. With deleteMissing: false the hand-authored TOMLs would be retained but never updated by future syncs, and no test calls requesty.parseModels or requesty.translateModel to verify the responses models survive the filter—all Requesty tests invoke buildRequestyModel directly. Action: Confirm the Requesty API sets api to "chat" (or omits it) for the openai-responses/... IDs. If it sets a different value, broaden the filter (e.g., allow "responses") or remap those IDs before the api check. Add a parseModels test that feeds a responses-model payload and asserts it is included in the parsed set.

@mrubens

mrubens commented Jul 13, 2026

Copy link
Copy Markdown
Author

Confirmed against Requesty’s current /v1/models payload: all six openai-responses/... IDs added here are returned with api: "chat", so the existing parseModels filter includes them. I kept the filter unchanged and added a focused regression test in 482dc5283 that feeds openai-responses/gpt-5.5 through parseModels and asserts it survives the filter. The focused suite now passes 48/48, and full catalog validation also passes.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [medium] [possible mistake] providers/requesty/models/google/gemma-4-31b-it.toml:5-10 (sync source: packages/core/src/sync/providers/requesty.ts:294-308) - Check: reasoning_options must reflect controls the inference provider actually exposes for the specific model, not just the model family's general capability. Why: The new reasoningOptions function returns effort + budget_tokens for every google/* base model, and gemma-4-31b-it.toml inherits those options. However, the provider.toml reasoning documentation (providers/requesty/provider.toml:5-12) only describes Vertex thinkingConfig translation for Gemini models ("Flash/Flash Lite or Pro"). Gemma is a separate model family that does not support thinkingConfig on Vertex, so reasoning_effort may be silently dropped. Unlike the fallback branch for other providers (existing?.reasoning_options ?? []), the google branch ignores existing, so a future hand-authored reasoning_options = [] for Gemma would be overwritten on the next sync run. Action: Verify that Requesty meaningfully accepts and translates reasoning_effort for google/gemma-4-31b-it (accepted and applied, not accepted and ignored). If it does not, exclude Gemma from the google-family reasoning heuristic so the file can use reasoning_options = [].

@mrubens

mrubens commented Jul 13, 2026

Copy link
Copy Markdown
Author

Confirmed. Requesty’s reasoning documentation scopes configurable Google effort to Gemini, and live Gemma checks did not show a meaningful effort control: none still produced 2,064 reasoning tokens while high produced 1,804. In e25168f0f, I narrowed the Google heuristic to google/gemini-*, force google/gemma-* to reasoning_options = [], and added regression coverage that clears the previously generated controls. The rebuilt catalog keeps Gemma reasoning and tool support enabled, its OpenCode Bash tool call passes, the focused suite passes 48/48, validation passes, and the Requesty dry run is idempotent.

@github-actions

Copy link
Copy Markdown
Contributor

No actionable findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant