[hanhainebula]

Summary

Add 11 self-contained task packages spanning dynamic vector indexes, constrained retrieval, reranking, embedding adaptation, protein/text retrieval, molecular search, and training-record retrieval. Each package provides fixed inputs, a submission contract, agent and verifier environments, resource limits, grading code, optional baseline tools where applicable, and instructions for restoring assets and running the task.

All packages live under tasks/<task-name>. The author of all 11 tasks is jianlyu.

Task	Version / hardware	Objective	Reference score / 100
incremental-vector-search	0.10.2 / CPU	Maintain recall through incremental inserts under stage-specific query budgets.	0.0
deletion-resilient-vector-search	0.1.6 / CPU	Preserve retrieval quality and reclaim memory through 100 deletion stages.	7.9
resource-bounded-multivector-search	0.3.2 / CPU	Reduce MaxSim query time while retaining Recall@100 ≥ 0.99.	17.5
large-scale-formula-retrieval	0.4.2 / CPU	Search the complete ARQMath corpus with 34 MiB prepared artifacts and 50 ms queries.	22.2
budgeted-bimetric-retrieval	0.2.2 / CPU	Improve NQ retrieval using at most 20 expensive distances per query.	21.8
budgeted-candidate-reranking	0.2.2 / one A100 40GB	Rerank BM25 candidates using a metered, fixed Qwen3-1.7B model.	13.6
instruction-following-embedding	0.7.1 / one A100 40GB	Adapt an instruction-sensitive E5 query encoder within a 512 KiB deployment budget.	23.3
multilingual-vector-postprocessing	0.1.2 / one A100 40GB	Improve multilingual retrieval by transforming fixed embeddings.	20.1
protein-text-bidirectional-retrieval	0.6.1 / one A100 40GB	Learn compact embeddings for reciprocal protein/text retrieval.	19.8
constrained-molecular-search	0.3.3 / one A100 40GB	Recover feasible top-10,000 molecules under a two-second query deadline.	16.9
training-error-tracing	0.6.2 / one A100 40GB	Reduce training-record retrieval cost while meeting quality gates.	0.0

Reference results: Scores above come from prior Codex + OpenRouter openai/gpt-6-luna evaluations with reasoning effort max and are provided for reference only.

Package structure and resources

• Align raw-instruction.md with the existing structured task format and keep the agent-facing instruction.md consistent with it.
• Provide exactly two agent docs per task: available_resources.md and environment.md. Keep required license notices in dedicated locations.
• Include only current task specifications, runnable baseline/starter code, required inputs and runtime metadata. Authoring notes, calibration ledgers, experiment histories and reference-training recipes stay outside the packages.
• Define fixed scoring constants and phase limits in dedicated runtime modules. Keep data/model integrity checks, resource accounting and failure handling with their consumers.
• All 11 tasks include an independent integrity judge. Complete audit coverage for both budgeted tasks, multivector retrieval, formula retrieval and molecular search with protected trajectories, submission-provenance review and fail-closed reward publication; legitimate indexes, public-data parameters and zero-call strategies remain allowed.
• Clear formula build-stage IPC by both owner and creator and verify that no state survives; account for open-unlinked and mapped tracing files across preparation services.
• Use separate verifier environments and task-specific submission transfer, filesystem/process restrictions and network policies. GPU tasks request one device in each phase.

The structured contracts follow the existing retrieval-task conventions: memory-constrained-dense-retrieval and sparse-retrieval-pruning for index/resource interfaces; long-document-reranking for candidate-pool contracts; code-embedding-finetuning and query-encoder-alignment for model/vector submissions; and agentic-search and scientific-paper-qa for retrieval protocols and validation sections. Each new task defines its own data, interfaces, metrics and budgets.

Assets

The manifests contain 218 entries: 119 official dataset assets, 71 immutable upstream assets, and 28 bundled inputs/metadata files. Official assets are pinned to search-swe/Search-SWE at 3635ca4269c1a1122c2da50187d0db42f3f73006. Each assets.json records exact sizes, hashes, revisions and restoration paths.

Bundled verifier inputs and small runtime descriptors use the existing source.local_path restoration mechanism. Runtime-hidden data remains outside agent mounts; published benchmark inputs are isolated during execution, rather than claimed to be private datasets. Source-specific usage terms and attribution are documented in the task packages.

Downloaded data/model caches, credentials, job outputs and author experiments are excluded from the commit. The packages do not depend on external author workspaces.

Validation evidence

• All 11 package structure checks and asset size/SHA-256 checks passed.
• All 22 agent/verifier images built successfully. Both Compose configurations were rendered for each task, and container checks verified the two-document layout and required license mounts.
• 175 unit/kernel checks passed, including three compiled-kernel checks executed in task containers. A further 12 molecular-search container protocol checks passed, covering full/partial Recall, request budgets, invalid output, timeouts, isolation and storage accounting.
• Scoring-parameter equivalence, protected baseline code/library hashes, mirrored runtime/docs, license bytes and model metadata integrity chains passed.
• The six changed verifier images built successfully. Targeted regressions cover owner-transferred IPC cleanup, ordinary index reload, open-unlinked and mapped storage, persistent services, capability isolation and audit health. The five completed integrity pipelines passed 55 container entrypoint scenarios using stubbed evaluators/judges.
• Six live DeepSeek audit probes passed for the two budgeted tasks: actual fixed-test answer tables were rejected, while zero-call baselines and permitted public-data parameters were accepted. Production finalizers were checked with those audit outputs and explicitly synthetic numeric inputs; these are integrity checks, not benchmark-score measurements.
• Submission checks passed: python scripts/check_tasks.py --merge-ready --base cb26fc3f519a49d79edf6f281932b0251fc90259 (0 errors) and python scripts/check_release.py (0 errors, 0 warnings).

Contributor checklist

• All 11 packages use canonical paths and descriptive task names.
• Actual authors are recorded in task.toml.
• Immutable asset references, bundled restoration and source terms are documented.
• Task packages exclude credentials, downloaded caches, jobs and author-only experiments.
• Local validation and reference model results are recorded.
• Enable Allow edits from maintainers, or coordinate contributor-applied review fixes.

Maintainer merge checklist

• Review task objectives, scoring, resource enforcement and verifier isolation.
• Check task names against the current base and queued PRs.
• Review source-specific terms and verify immutable asset references.
• Confirm required CI/review checks against the final PR head and current base.
• Merge with a merge commit to preserve contributor history.