feat(api): generate and migrate Dataset REST resources

[Abhijeet Prasad (AbhiPrasad)]

AI Summary

Implements the Dataset slice of #683 by publishing all operations tagged Datasets and moving eligible high-level SDK workflows onto the public REST resources. Shared generated models are repartitioned across Projects, Experiments, and Datasets as their reachable schema graph expands.

Migration flow

Before
  init_dataset() -----> APP /api/dataset/register
  Dataset.fetch() ----> API /btql
  Dataset.summarize() -> API /dataset-summary

After
                         +--> POST /v1/project --------+
  init_dataset() -------|                              +--> POST /v1/dataset
                         +--> GET /v1/project/{id} -----+

  Dataset.fetch() ------> POST /v1/dataset/{id}/fetch --> backend BTQL
  Dataset.summarize() --> GET  /v1/dataset/{id}/summarize

SDK workflow	Previous wire call	Public REST call
Resolve/create project	Combined dataset registration	POST /v1/project or GET /v1/project/{id}
Resolve/create dataset	POST /api/dataset/register	POST /v1/dataset
Fetch ordinary records	POST /btql	POST /v1/dataset/{id}/fetch
Fetch filtered records	POST /btql	Unchanged for _internal_btql
Summarize	GET /dataset-summary	GET /v1/dataset/{id}/summarize
Devserver ID lookup	Raw GET /v1/dataset/{id}	Generated GET /v1/dataset/{id}
Insert/update/delete rows	Batched /logs3 ingestion	Unchanged
Resolve environments	GET /environment-object/...	Unchanged

Behavior	Result
Dataset name omitted	Send logs, matching the legacy registration UDF default
Existing project or dataset	Public POST operations preserve get-or-create behavior
Registration atomicity	Project and dataset creation are now two requests, matching bt
Ordinary fetch semantics	REST builds the same dataset BTQL query with cursor, limit, and version
Custom BTQL semantics	Continue through BTQL for filters, sampling, and total limits
Legacy output conversion	Continue converting expected to output after REST fetches
Retry behavior	Fetch is a safe read; project and dataset creation are idempotent writes

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c131e9bc45

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".