Skip to content

Make daily Rig decomposition benchmark resilient to structured-output and typecheck failures - #501

Merged
pelikhan merged 4 commits into
mainfrom
copilot/design-relational-schema-hospital
Aug 28, 2026
Merged

Make daily Rig decomposition benchmark resilient to structured-output and typecheck failures#501
pelikhan merged 4 commits into
mainfrom
copilot/design-relational-schema-hospital

Conversation

Copilot AI commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

The benchmark produced empty solutions when free-form SQL was forced through object-schema validation, while standalone skill typechecking failed because required Node and Copilot SDK types were unavailable.

  • Output handling

    • Require s.string for free-form solver results, preserving complete SQL and Markdown fences.
    • Reserve object schemas for structured metadata.
  • Standalone skill packaging

    • Add @github/copilot-sdk and @types/node dependencies to the skill package.
    • Pin the benchmark workflow to the updated skill revision.

Example:

const singleSolver = agent({
  model: "medium",
  instructions: "...",
  output: s.string,
});

Copilot AI and others added 3 commits August 28, 2026 04:38
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Design and implement relational schema for hospital appointment system Make daily Rig decomposition benchmark resilient to structured-output and typecheck failures Aug 28, 2026
Copilot AI requested a review from pelikhan August 28, 2026 04:43
@pelikhan
pelikhan marked this pull request as ready for review August 28, 2026 11:00
@pelikhan
pelikhan merged commit 716ca95 into main Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[rig-decomposition-bench] Daily rig decomposition benchmark — 2026-08-27 — tie

2 participants