Skip to content

automate Cloudflare AI Gateway model sync - #5659

Closed
superhighfives wants to merge 3 commits into
anomalyco:devfrom
superhighfives:automate-cloudflare-ai-gateway
Closed

automate Cloudflare AI Gateway model sync#5659
superhighfives wants to merge 3 commits into
anomalyco:devfrom
superhighfives:automate-cloudflare-ai-gateway

Conversation

@superhighfives

@superhighfives superhighfives commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Automates the Cloudflare AI Gateway catalog through the shared hourly provider sync. The live catalog is authoritative for proxied third-party availability, pricing, context limits, and supported reasoning controls, while local curation covers tested overrides.

  • Uses Cloudflare’s REST API model naming and routing and model catalog as source references.
  • Adds strict pagination and retry handling.
  • Keeps generated provider TOMLs override-only while preserving authored base-model overrides.
  • Requires manual review for deletions and rejects unsafe sync paths.
  • Adds focused tests and updates the sync documentation.

bun validate passes, and the live check reports all 43 generated models unchanged.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] .github/workflows/sync-models.yml:88 - Check: Sync credentials must be dedicated/read-scoped and must not collide with deploy secrets (sync.md Workers AI guidance; patch 1→2 credential change). Why: The hourly job now injects CLOUDFLARE_API_TOKEN / CLOUDFLARE_ACCOUNT_ID from those secret names. Deploy already uses secrets.CLOUDFLARE_API_TOKEN for SST, and Workers AI deliberately uses separate CLOUDFLARE_WORKERS_AI_SYNC_* secrets so catalog sync does not share deploy credentials. Reusing the deploy token name risks privileged token exposure on every hourly run and/or the wrong account catalog if CLOUDFLARE_ACCOUNT_ID is not a catalog-read sandbox. Patch 1 originally wired the production AI Gateway sandbox secrets, then patch 2 switched to the generic names. Action: Use dedicated sync secret names (e.g. CLOUDFLARE_AI_GATEWAY_SYNC_API_TOKEN + CLOUDFLARE_AI_GATEWAY_SYNC_ACCOUNT_ID), wire only those in the workflow, and read them in the provider module (keep local aliases if needed). Do not share the deploy CLOUDFLARE_API_TOKEN.
  • [high] [violation] packages/core/src/sync/providers/cloudflare-ai-gateway.ts (deriveReasoningOptions + translateModel header) - Check: Every toggle reasoning control requires a leading top-of-file wire-path comment (AGENTS.md / audit skill); peer syncs synthesize headers when they emit toggles. Why: Schema walk sets hasToggle for enable_thinking / thinking and can emit { type = "toggle" } without curation. header is only noteHeader(curated.note), and authoritativeHeaders: true replaces any existing header with that value (often empty). OpenRouter / Merge Gateway / LLMGateway always attach a toggle wire header when they emit toggles. Automated sync can therefore create or rewrite reasoners with toggle and no wire comment, or wipe a previously correct header. Action: When final reasoning_options include toggle, always emit a leading wire-path header (from curation note, or a derived default from the matched property names). Fail closed if toggle is present and no wire path can be documented. Add a regression test for schema-derived toggle + missing curation note.
  • [medium] [violation] packages/core/src/sync/providers/cloudflare-ai-gateway.ts (deriveReasoningOptions) - Check: Do not pair toggle with effort that already includes none; derive real budget_tokens when the host schema exposes a reasoning budget. Why: Derived options unconditionally append toggle then effort with no none/toggle exclusivity check, and never map thinking_budget (or similar) to budget_tokens. For uncurated schema-driven models this can invent invalid toggle+none shapes or drop a real budget control in favor of effort-only/toggle-only. Alibaba paths are curated today, so live files may be fine, but hourly automation will hit any new schema-only reasoner. Action: After derivation, apply AGENTS toggle rules (none ∈ effort ⇒ no toggle; separate on/off + graded effort ⇒ toggle + effort without none). Detect reasoning-budget schema fields and emit budget_tokens only for those. Cover both with unit tests.

@rekram1-node

Copy link
Copy Markdown
Collaborator

had some conflicts and I couldnt push for some reason, had the agent port it to a separate pr (tagged u as coauthor):
59fa8dc

Also merged it.

@superhighfives
superhighfives deleted the automate-cloudflare-ai-gateway branch August 31, 2026 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants