(7/n) Complete declarative task runtime and shared pairwise orchestration

[kargibora]

Summary

This PR completes the declarative task architecture by connecting packaged task YAML definitions to dataset loading,
baseline selection, benchmark execution, judge configuration, and scoring.

Previously, task definitions described benchmark settings, but parts of the runtime still selected behavior through
hard-coded branches, repeated registry lookups, or direct function calls. Adding another task could therefore require
changes across several unrelated files.

After this change, a task is resolved once and passed through the complete evaluation pipeline. Existing components can
be reused by referencing their registered names from task YAML.

Before and after

   Before                                                       After
  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
   Task information was partly defined in YAML and partly       Stable task behavior is defined in task YAML and
   selected in Python.                                          validated by typed schemas.
  ───────────────────────────────────────────────────────────  ───────────────────────────────────────────────────────────
   Runners and dataset loaders could resolve the same task      The dispatcher resolves the task once and passes the
   repeatedly.                                                  resulting specification downstream.
  ───────────────────────────────────────────────────────────  ───────────────────────────────────────────────────────────
   Pairwise tasks relied on benchmark-specific routing and      Compatible tasks share the same pairwise runner and
   duplicated setup logic.                                      dataset orchestration.
  ───────────────────────────────────────────────────────────  ───────────────────────────────────────────────────────────
   Scoring configuration was validated but not always used      The runner resolves and executes the scoring adapter
   by the runtime.                                              declared by the task.
  ───────────────────────────────────────────────────────────  ───────────────────────────────────────────────────────────
   Adding fields to one large schema made task definitions      Schemas are separated by responsibility and protocol-
   harder to maintain.                                          specific fields remain isolated.
  ───────────────────────────────────────────────────────────  ───────────────────────────────────────────────────────────
   Supported component names could be duplicated between        Validation derives available runners, datasets, and
   validation and runtime registries.                           scorers from their owning registries.

Architecture

The runtime now follows this path:

  Task YAML
      ↓
  Schema validation and inheritance
      ↓
  Resolved task specification
      ↓
  Benchmark and dataset registries
      ↓
  Shared or protocol-specific runner
      ↓
  Generation, judging, scoring, and artifacts

The new schema package separates datasets, sources, baselines, pairwise protocols, MT-Bench protocols, and resolved task
models. This prevents a single schema file from growing whenever a new benchmark family introduces specialized fields.

Most new tasks do not require a new schema or runner. A task using existing components only needs a YAML definition
selecting its dataset adapter, runner, baseline policy, judge protocol, and scorer.

A new Python component is only necessary when a benchmark introduces genuinely different behavior:

• A dataset adapter handles benchmark-specific downloading and normalization.
• A benchmark runner handles a different evaluation workflow.
• A protocol schema validates new task-owned algorithm settings.
• A scoring adapter handles a different result interpretation.

Executable behavior remains in Python rather than YAML; YAML only selects registered and validated components.

Changes

• Split the task schema into focused modules.
• Added shared pairwise dataset and runner orchestration.
• Passed resolved task specifications through the runtime instead of repeating lookups.
• Made task-defined scoring adapters operational.
• Centralized ownership of runner, dataset, scorer, and baseline registrations.
• Added declarative fluency tasks for Finnish, French, German, Spanish, and Swedish.
• Added documentation for defining tasks and extending the available components.
• Preserved runtime overrides while keeping stable benchmark identity in task YAML.

[kargibora]

Note that with new fluency adaptation from David's PR I will change this to adjust it so everything is implemented exactly as it is.