Skip to content

Implement the allocation-local Data Designer client worker #876

Description

@nabinchha

Priority Level

High

Task Summary

Implement the allocation-local Data Designer client environment and worker that validates pinned inputs, invokes the public generation contract, and emits semantic attempt results for the Slurm runtime.

Technical Details & Implementation Plan

  • Build the client environment from pinned, immutable package and artifact references.
  • Reject editable installs, mutable source references, and distribution conflicts with the selected client image.
  • Discover and validate required plugins and Data Designer configuration inside the allocation before generation starts.
  • Invoke the public generation boundary from Define the public Data Designer contract used by the optional Slurm package #851 without importing engine internals.
  • Write bounded, structured progress and semantic client-result records for the runtime/state lane.
  • Resolve managed assets through approved paths without serializing credentials.

Acceptance criteria

  • The worker runs from built artifacts without a repository checkout.
  • Dependency and plugin failures surface before generation and produce stable failure classifications.
  • Generation uses only the public Data Designer contract and emits a validated semantic result.
  • Client logs and persisted records contain no credentials or unredacted environment values.
  • Tests cover dependency conflicts, missing plugins/assets, success, partial output, interruption, and normalized failures.

Out of scope

  • Scheduler submission or allocation orchestration.
  • Model-server startup and readiness.
  • Winner publication, retry selection, collection, or benchmark analysis.

Investigation / Context

#851 defines the core public generation invocation boundary, #868 owns allocation sequencing, and #872 supplies deterministic test doubles. This task owns the production client environment and worker between those boundaries.

Agent Plan / Findings

Keep environment validation separate from generation invocation, and emit package-owned result records rather than leaking library or subprocess exceptions.

Dependencies

Depends on #851, the shared records in #873, and the fake infrastructure in #872. Integrates with #868 and #869.

Metadata

Metadata

Assignees

Labels

taskInternal development task

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions