Skip to content

Latest commit

 

History

4 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Recursive Task Optimizer (EvoFlow)

An agent-agnostic, evaluator-driven outer loop for repeatedly improving a repository, prompt, workflow, generator, or other computable artifact with Claude Code, Codex CLI, Hermes, or any compatible headless CLI agent.

EvoFlow turns a one-shot coding agent into an experimental search process:

select a parent
    -> create an isolated child branch and worktree
    -> let a CLI agent modify the solution and its future improvement procedure
    -> validate the result with an external evaluator
    -> keep the candidate and its lineage in an archive
    -> repeat

Important

EvoFlow is universal at the orchestration layer, not at the quality-definition layer. Every real use case still needs a task-specific evaluator that can tell the system whether one candidate is better than another.

Warning

EvoFlow executes untrusted, model-generated code and commands. Run unattended experiments only inside a disposable container or VM with restricted credentials, network access, CPU, memory, process count, and disk usage. The Python runner is not an operating-system sandbox.

Install

npx skills add laruss/recursive-task-optimizer

The skills CLI installs the skill into whichever agents it detects - Claude Code, Codex, Cursor, Gemini CLI, OpenCode, and many others. Add -g to install at user level instead of into the current project:

npx skills add laruss/recursive-task-optimizer -g

The installed directory holds SKILL.md, references/, and scripts/evoflow.py. Point EVOFLOW at the runner and verify it:

export EVOFLOW="$HOME/.claude/skills/recursive-task-optimizer/scripts/evoflow.py"
python3 "$EVOFLOW" self-test

Installation is optional. The runner is a single dependency-free script, so a clone works just as well:

git clone https://github.com/laruss/recursive-task-optimizer
export EVOFLOW="$PWD/recursive-task-optimizer/skills/recursive-task-optimizer/scripts/evoflow.py"

Every python3 "$EVOFLOW" ... command below assumes one of these two setups. See references/native-install.md for manual installation into a specific agent.

Research basis

This project is an independent, constrained implementation inspired by the HyperAgents research project:

The paper introduces self-referential agents that combine task-solving behavior with a modifiable meta-level procedure for generating future improvements. It instantiates this idea as DGM-Hyperagents (DGM-H), which repeatedly selects parents, creates modified descendants, evaluates them, and accumulates an archive of useful stepping stones.

EvoFlow adapts those ideas to a portable CLI workflow. It is not a port, benchmark reproduction, or drop-in replacement for the official HyperAgents repository. No source files from the upstream repository are included here.

What this repository contains

This repository has two complementary layers:

  1. The Agent Skill - skills/recursive-task-optimizer/SKILL.md teaches a compatible agent how to design, configure, run, inspect, and troubleshoot an EvoFlow experiment.
  2. The deterministic runner - skills/recursive-task-optimizer/scripts/evoflow.py implements Git worktrees, candidate branches, protected-path checks, evaluation, archive management, parent selection, resume, export, and held-out final testing.

The skill layer is optional at runtime. You can install the skill with the skills CLI, ask an agent to read SKILL.md, or call scripts/evoflow.py directly.

The mental model

Think of EvoFlow as an automated research laboratory:

  • The target repository is the object being improved.
  • The CLI agent is the researcher that proposes and implements a mutation.
  • TASK.md is the local job specification.
  • META.md is the inherited research method: how the next descendant should diagnose problems and search for improvements.
  • MEMORY.md is a compact lineage notebook containing durable lessons.
  • The evaluator is an external measuring instrument.
  • A candidate branch is one experiment.
  • The archive is a family tree of experiments, scores, failures, diffs, and logs.
  • The outer runner is the lab supervisor that enforces rules and prevents candidates from rewriting their own exam.

A normal retry loop asks an agent to improve the same result again and again. EvoFlow also lets descendants improve the procedure used to generate later improvements, while retaining multiple branches instead of replacing everything with the latest attempt.

The core idea

Each candidate inherits four things from its parent:

candidate repository
├── project files and artifacts
└── .hyperflow/genome/
    ├── TASK.md
    ├── META.md
    └── MEMORY.md

Task-level improvement

The agent can modify the actual solution: source code, prompts, templates, orchestration logic, configuration, tests, documentation generators, or other allowed files.

Meta-level improvement

The agent can rewrite META.md. The runner loads this file into the prompt used to create future descendants. A better META.md can change how later generations:

  • analyze failed cases;
  • select hypotheses;
  • prioritize metrics;
  • create regression tests;
  • balance exploration and focused repair;
  • avoid repeating failed approaches;
  • update durable memory.

This is the practical metacognitive part of EvoFlow: the candidate can improve not only the solution, but also the inherited procedure that produces subsequent candidates.

Persistent lineage memory

MEMORY.md carries concise, reusable findings across descendants. Full logs and metrics stay in the immutable run archive, so memory does not need to grow without bound.

Open-ended archive

Valid lower-scoring candidates are not automatically discarded. They remain available as alternative stepping stones. Parent selection can therefore branch from earlier candidates rather than following one greedy chain forever.

Architecture

flowchart TB
    O[Stable objective and run configuration]
    A[Candidate archive]
    S[Parent selection]
    W[Disposable Git worktree and child branch]

    subgraph C[Mutable candidate]
        R[Solution files]
        T[TASK.md]
        M[META.md]
        D[MEMORY.md]
    end

    G[CLI mutation agent]
    P[Protected-path and size checks]
    Q[Optional gate: tests, lint, typecheck]
    E[External validation evaluator]
    H[Immutable logs, metrics, lineage, and report]

    O --> S
    A --> S
    S --> W
    W --> C
    C --> G
    G --> C
    C --> P
    P --> Q
    Q --> E
    E --> H
    H --> A
Loading

The mutable candidate and the trusted outer loop are intentionally separated.

Mutable by the candidate

Subject to your allow-list and limits, a candidate can modify:

  • project source files and artifacts;
  • .hyperflow/genome/TASK.md;
  • .hyperflow/genome/META.md;
  • .hyperflow/genome/MEMORY.md.

Protected by the runner

The following control surfaces are protected from candidate modification:

.hyperflow/config.toml
.hyperflow/evaluator.py
.hyperflow/state/**
.hyperflow/runtime/**

Additional paths can be protected in the configuration. The evaluator, archive, selection policy, safety rules, and held-out test remain outside the candidate's control.

This is a deliberate difference from a fully self-referential research system. EvoFlow keeps the operational boundary fixed so runs are easier to audit, compare, reproduce, and execute with different CLI providers.

What EvoFlow can optimize

EvoFlow works best when the work can be represented as repository changes and evaluated repeatedly.

Examples include:

  • improving an algorithm against a fixed correctness benchmark;
  • fixing a parser across hundreds of representative cases;
  • optimizing latency while preserving behavior and memory limits;
  • evolving the system prompt, tool policy, or orchestration code of another agent;
  • improving a document generator across many structured inputs;
  • refining a code review or migration workflow;
  • improving a simulation model against statistical and regression checks;
  • comparing Claude Code, Codex, Hermes, or other agents under the same budget and evaluator.

For repeated agent work, the target should be the procedure that handles a class of inputs, not one hand-picked answer. For example, optimize a report generator over 50 validation cases rather than optimizing one report until a judge likes it.

When EvoFlow is a good fit

Use EvoFlow when all of the following are true:

  • the objective can remain stable during a run;
  • multiple attempts are useful;
  • candidate changes can live in Git;
  • quality can be measured with tests, benchmarks, a rubric, an LLM judge, or a combination;
  • you can reserve private held-out cases for the final check;
  • you can run the agent and generated code in an isolated environment.

It is usually a poor fit when:

  • the task is a one-off request with no reusable procedure;
  • there is no credible way to compare candidates;
  • success is entirely subjective and no stable rubric can be built;
  • the candidate needs unrestricted production credentials or infrastructure access;
  • a failed experiment could cause irreversible external side effects.

Requirements

  • Python 3.11 or newer. The runner uses tomllib from the standard library.
  • Git.
  • A target Git repository with at least one commit.
  • An installed and authenticated non-interactive CLI agent, or a custom command that can edit its current working directory.
  • A task-specific evaluator.
  • A disposable container, VM, or similarly isolated environment for unattended execution.

The EvoFlow runner itself has no third-party Python dependencies.

Quick start

1. Verify the runner

python3 "$EVOFLOW" self-test

The self-test creates an isolated temporary repository, evaluates a seed, generates several descendants with a mock agent, resumes the run, executes a final test, and exports the best patch.

2. Initialize a target project

Choose an adapter preset:

python3 "$EVOFLOW" init \
  --project /path/to/target-repository \
  --adapter claude

Available presets:

claude
codex
hermes
generic

Initialization creates:

.hyperflow/
├── config.toml
├── evaluator.py
└── genome/
    ├── TASK.md
    ├── META.md
    └── MEMORY.md

3. Configure the experiment

Edit these files before the first run:

.hyperflow/config.toml
.hyperflow/genome/TASK.md
.hyperflow/evaluator.py

Replace the evaluator placeholder and remove the HYPERFLOW_EVALUATOR_TODO marker.

4. Configure real isolation

Run the project in a disposable environment. Restrict credentials, network access, CPU, memory, process count, disk usage, and writable mounts.

Only after doing that, set:

[safety]
acknowledge_untrusted_code = true
allow_network = false

allow_network = false is a policy signal included in the agent prompt. It does not create a firewall. Enforce network restrictions outside EvoFlow.

5. Commit the seed

cd /path/to/target-repository
git add .hyperflow .gitignore
git commit -m "chore: configure EvoFlow"

EvoFlow requires a clean working tree before starting or resuming a run.

6. Validate and run

python3 "$EVOFLOW" doctor \
  --project /path/to/target-repository

python3 "$EVOFLOW" run \
  --project /path/to/target-repository \
  --iterations 12

7. Inspect the result

python3 "$EVOFLOW" status \
  --project /path/to/target-repository

python3 "$EVOFLOW" best \
  --project /path/to/target-repository \
  --json

No candidate is merged automatically. Review the branch, diff, logs, metrics, dependencies, and held-out performance before promotion.

Configure the objective

The immutable run objective lives in .hyperflow/config.toml:

name = "parser-accuracy"
objective = """
Maximize parser correctness on the validation suite while preserving the public API,
keeping p95 latency below 50 ms, and introducing no new high-severity vulnerabilities.
"""
iterations = 20
seed = 42
selection = "score_child_prop"
higher_is_better = true

Write the objective in terms of observable behavior. Avoid goals such as "make the project better" because the agent and evaluator cannot infer a stable optimization target from them.

The objective is snapshotted at run creation and remains fixed during resume. Start a new run when you change the objective, evaluator, agent command, or safety policy.

Configure TASK.md

TASK.md describes the work contract visible to the mutation agent:

# Task contract

## Goal

Increase correctness on the parser validation set.

## Required deliverable

Modify the implementation and add legitimate regression tests when useful.

## Constraints

- Preserve the public API.
- Do not modify benchmark fixtures or the evaluator.
- Keep p95 latency below 50 ms.
- Avoid unrelated refactors.

## Validation notes

Validation reports correctness, timeout rate, and p95 latency.

A candidate may clarify useful execution details in TASK.md, but it must not redefine the fixed objective to make the task easier.

Configure META.md

META.md is the inherited mutation procedure. The default version tells descendants to inspect evidence, form one concrete hypothesis, make real changes, run checks, and update memory.

A later descendant might improve it with domain-specific strategy, for example:

1. Cluster failed validation cases by parser stage and error signature.
2. Select one high-frequency cluster that has not been attempted in the current lineage.
3. Reproduce it with the smallest local regression case.
4. Fix the root cause without special-casing validation file names or expected outputs.
5. Compare correctness and latency against the parent.
6. Record why the hypothesis worked or failed in MEMORY.md.

Because descendants inherit the updated file, this change affects future search behavior rather than only the current patch.

Configure MEMORY.md

Keep memory compact and durable:

# Durable lineage memory

- The tokenizer incorrectly treats escaped delimiters as separators.
- A regex-only fix improved simple cases but regressed nested input.
- The next useful direction is a stateful scan with a bounded allocation budget.

Do not copy complete logs, prompts, or benchmark output into memory. Those already live in the run archive.

Design the evaluator first

The evaluator is the center of the flow. If it measures the wrong thing, EvoFlow will efficiently optimize the wrong thing.

The evaluator command must write a JSON object to {result_file}:

{
  "score": 0.934,
  "metrics": {
    "correctness": 0.97,
    "p95_ms": 42.1,
    "memory_mb": 118.0
  },
  "summary": "97/100 cases; p95 42.1 ms",
  "eligible": true
}

Rules:

  • score is required and must be a finite number.
  • metrics is an optional JSON object.
  • summary is an optional human-readable string.
  • eligible defaults to true.
  • Use eligible = false for hard constraints that a high score must not compensate for, such as an API break, security failure, forbidden dependency, or memory-limit violation.
  • The evaluator must inspect the final artifact independently. Never trust candidate-reported metrics.
  • The evaluator must not modify the candidate worktree.

A minimal evaluator can look like this:

#!/usr/bin/env python3
from __future__ import annotations

import argparse
import json
import subprocess
from pathlib import Path


def evaluate(workspace: Path) -> dict:
    completed = subprocess.run(
        ["python3", "-m", "pytest", "-q"],
        cwd=workspace,
        text=True,
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT,
        timeout=300,
        check=False,
    )
    passed = completed.returncode == 0
    return {
        "score": 1.0 if passed else 0.0,
        "metrics": {"tests_passed": int(passed)},
        "summary": "test suite passed" if passed else "test suite failed",
        "eligible": passed,
    }


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--workspace", required=True)
    parser.add_argument("--output", required=True)
    parser.add_argument("--mode", default="validation")
    args = parser.parse_args()

    payload = evaluate(Path(args.workspace).resolve())
    output = Path(args.output).resolve()
    output.parent.mkdir(parents=True, exist_ok=True)
    output.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")


if __name__ == "__main__":
    main()

For meaningful search, use a continuous or multi-level score when possible. A binary pass/fail signal gives the archive very little information.

Gate versus evaluator

Use gate_command for cheap binary prerequisites such as unit tests, lint, or type checking:

[evaluation]
gate_command = ["bash", "-lc", "npm test && npm run typecheck"]
gate_timeout_seconds = 900

Use the evaluator for the richer comparison signal: correctness rate, latency, token use, cost, judge scores, robustness, or other domain metrics.

Prevent evaluator gaming

  • Keep evaluator code and fixtures outside the mutable allow-list.
  • Mount validation inputs read-only when possible.
  • Check the artifact directly instead of reading claims from README files or logs.
  • Include adversarial and metamorphic cases.
  • Reject deleted tests, changed APIs, fixture tampering, and hidden external side effects.
  • Manually investigate unexpected score jumps.
  • Keep a private held-out set that never participates in parent selection.

Choose an agent adapter

EvoFlow does not import a provider SDK. It launches an argv array in the candidate worktree and captures stdout, stderr, exit code, duration, and file changes.

The CLI must:

  • run without an interactive TTY;
  • accept a complete prompt through a file or stdin;
  • edit the current working directory;
  • terminate with a meaningful exit status;
  • operate within the configured timeout.

Claude Code

python3 "$EVOFLOW" init --project /path/to/project --adapter claude

The generated preset uses non-interactive claude -p, stdin, explicit tools, and JSON output. Review the command against the version installed in your environment.

Claude Code documentation:

Codex CLI

python3 "$EVOFLOW" init --project /path/to/project --adapter codex

The generated preset uses codex exec and instructs Codex to read the generated prompt file before editing the current repository.

Codex documentation:

Hermes

python3 "$EVOFLOW" init --project /path/to/project --adapter hermes

The generated preset uses Hermes non-interactively with a query file. Review any automatic-approval option carefully and run it only inside a sandbox.

Hermes documentation:

Any other CLI

Use the generic adapter and configure an argv array:

[agent]
adapter = "generic"
prompt_mode = "file"
command = [
  "my-agent",
  "run",
  "--non-interactive",
  "--prompt-file", "{prompt_file}"
]
timeout_seconds = 3600
require_zero_exit = true
inherit_environment = false
pass_env = ["MY_AGENT_API_KEY"]

Or send the complete prompt through stdin:

[agent]
adapter = "generic"
prompt_mode = "stdin"
command = ["my-agent", "--non-interactive"]

Available placeholders include:

{prompt_file}
{workspace}
{candidate_id}
{parent_id}
{run_dir}
{project}
{mode}

Prefer argv arrays over shell interpolation. If a shell is unavoidable, invoke it explicitly and keep all interpolated input under your control.

Restrict mutable paths

An empty mutable list means every non-protected path may be changed:

[paths]
mutable = []

For tighter experiments, use an allow-list:

[paths]
mutable = [
  "src/**",
  "tests/regression/**",
  ".hyperflow/genome/**"
]
protected = [
  ".hyperflow/config.toml",
  ".hyperflow/evaluator.py",
  ".hyperflow/state/**",
  ".hyperflow/runtime/**",
  ".gitignore"
]

Also configure change budgets:

[limits]
max_changed_files = 50
max_diff_bytes = 500000
max_genome_bytes = 100000

The runner checks the full diff relative to the parent commit, including commits created by the agent itself.

What happens in one generation

For generation g0007, EvoFlow performs the following sequence:

  1. Select an eligible parent from the archive.
  2. Create a new candidate branch from the parent commit.
  3. Create a disposable Git worktree for that branch.
  4. Build a prompt from the stable objective, parent metrics, archive summary, TASK.md, META.md, MEMORY.md, and immutable rules.
  5. Launch the configured CLI agent with the worktree as its current directory.
  6. Inspect all changed files and commits relative to the parent.
  7. Reject protected-path changes, mutable-scope violations, oversized diffs, and oversized genome files.
  8. Commit valid uncommitted candidate changes.
  9. Run the optional gate.
  10. Run the external validation evaluator.
  11. Append the candidate, metrics, logs, lineage, and status to the run archive.
  12. Keep valid candidate branches even when they are not the current best.

A failed generation is recorded for auditability. Failed branches are removed by default unless configured otherwise.

Parent selection strategies

Configure selection in .hyperflow/config.toml:

selection = "score_child_prop"

Available strategies:

  • score_child_prop - default; combines normalized performance with a novelty penalty based on how many descendants a candidate already produced.
  • best - always select the best eligible candidate; simple but prone to premature convergence.
  • latest - continue the newest eligible lineage; closest to a conventional sequential agent loop.
  • random - sample uniformly from eligible candidates.
  • ucb - combine quality with an exploration bonus.

Set higher_is_better = false for metrics that should be minimized, such as latency or error count.

Run state and artifacts

Each run is stored under:

.hyperflow/state/runs/<run-id>/
├── manifest.json
├── config.snapshot.toml
├── archive.jsonl
├── report.md
├── prompts/
├── logs/
├── results/
└── final-tests/

Important artifacts:

  • archive.jsonl - append-only record of candidates, parents, commits, scores, metrics, changed files, failures, and process metadata.
  • report.md - generated validation leaderboard and best-candidate summary.
  • logs/ - agent, gate, and evaluator stdout/stderr.
  • prompts/ - exact prompt used for each generation.
  • candidate branches - inspectable Git history for every valid variant.

The run configuration is snapshotted at creation. Resume uses the snapshot rather than silently adopting edited control files.

Command reference

# Initialize control files
python3 "$EVOFLOW" init --project /path/to/project --adapter claude

# Validate configuration, Git state, evaluator, adapter, and safety prerequisites
python3 "$EVOFLOW" doctor --project /path/to/project

# Start a run
python3 "$EVOFLOW" run --project /path/to/project --iterations 12

# Resume an existing run to a total budget of 30 generations
python3 "$EVOFLOW" run \
  --project /path/to/project \
  --run-id <run-id> \
  --resume \
  --iterations 30

# Print the latest report
python3 "$EVOFLOW" status --project /path/to/project

# Show the best validation candidate
python3 "$EVOFLOW" best --project /path/to/project --json

# Export a binary-capable Git patch
python3 "$EVOFLOW" export \
  --project /path/to/project \
  --candidate best \
  --output best.patch

# Run a private held-out evaluator
python3 "$EVOFLOW" final-test \
  --project /path/to/project \
  --candidate best \
  --final-config /secure/final.toml

# Remove leftover worktrees
python3 "$EVOFLOW" clean --project /path/to/project

# Remove leftover worktrees and candidate branches for a run
python3 "$EVOFLOW" clean \
  --project /path/to/project \
  --run-id <run-id> \
  --delete-branches

Held-out final testing

Validation feedback is used repeatedly during search, so candidates can overfit to it. Select the candidate using validation results, then run a private final evaluator exactly as a final check.

Keep the final evaluator and data outside the candidate repository:

# /secure/final.toml

[final]
command = [
  "python3",
  "/secure/private_final_evaluator.py",
  "--workspace", "{workspace}",
  "--output", "{result_file}",
  "--mode", "test"
]
timeout_seconds = 3600

Run it with:

python3 "$EVOFLOW" final-test \
  --project /path/to/project \
  --candidate best \
  --final-config /secure/final.toml

Final-test results are stored separately and are not fed back into parent selection. Do not continue the same search after inspecting private test feedback; doing so turns the private set into another validation set.

Install as a native Agent Skill

Native skill installation is optional. It improves discovery and lets an agent configure or operate EvoFlow using SKILL.md and the bundled references.

The skills CLI covers every supported agent in one command and is the recommended route:

npx skills add laruss/recursive-task-optimizer            # into the current project
npx skills add laruss/recursive-task-optimizer -g         # at user level
npx skills add laruss/recursive-task-optimizer -a codex   # into one specific agent

The manual equivalents below copy the same directory by hand.

Claude Code

Project-local installation:

mkdir -p /path/to/project/.claude/skills
cp -R /path/to/recursive-task-optimizer/skills/recursive-task-optimizer \
  /path/to/project/.claude/skills/recursive-task-optimizer

User-level installation:

mkdir -p ~/.claude/skills
cp -R /path/to/recursive-task-optimizer/skills/recursive-task-optimizer \
  ~/.claude/skills/recursive-task-optimizer

Invoke it with natural language or:

/recursive-task-optimizer Configure an evaluator-driven loop for this repository using Claude Code.

Codex CLI

export CODEX_HOME="${CODEX_HOME:-$HOME/.codex}"
mkdir -p "$CODEX_HOME/skills"
cp -R /path/to/recursive-task-optimizer/skills/recursive-task-optimizer \
  "$CODEX_HOME/skills/recursive-task-optimizer"

Restart Codex after installation, then ask it to use the recursive-task-optimizer skill.

Hermes

mkdir -p ~/.hermes/skills
cp -R /path/to/recursive-task-optimizer/skills/recursive-task-optimizer \
  ~/.hermes/skills/recursive-task-optimizer
hermes skills list

Invoke it with:

hermes chat -q "/recursive-task-optimizer Configure EvoFlow for the current repository."

CLI tools without native skill discovery

Point the agent at the entrypoint explicitly:

Read /opt/recursive-task-optimizer/SKILL.md and use it to configure EvoFlow for the current repository.

Or skip the skill layer and run the Python runner directly.

Safety model

EvoFlow provides application-level safeguards, not host isolation.

The runner provides:

  • disposable Git worktrees;
  • protected paths and optional mutable allow-lists;
  • diff, file-count, and genome-size limits;
  • process timeouts;
  • a minimal environment by default;
  • external gates and evaluators;
  • append-only run records;
  • separate held-out testing;
  • no automatic merge or deployment.

The runner cannot reliably prevent:

  • reads or writes outside the worktree;
  • symlink or path escapes;
  • network access;
  • access to host keychains, browser sessions, SSH agents, or Docker sockets;
  • destructive external API calls;
  • supply-chain attacks through downloaded dependencies;
  • resource exhaustion beyond what the host enforces;
  • model-generated child processes that outlive unusual termination paths.

For unattended runs:

  • use a disposable clone, container, or VM;
  • never mount the Docker socket;
  • do not forward an SSH agent;
  • expose only short-lived, low-limit credentials;
  • deny network access unless the task requires it;
  • mount benchmarks and expected outputs read-only;
  • enforce CPU, RAM, PID, file-size, and disk quotas;
  • pin CLI, runtime, image, and dependency versions;
  • manually review the complete winning diff before promotion.

Read references/safety.md before enabling unattended execution.

Practical evaluation advice

Repeated task workflows

When optimizing an agent or generator, evaluate it over a set of cases:

validation cases
├── case-001
├── case-002
├── ...
└── case-100

For every case, run the candidate with the same model, tool policy, token budget, time limit, and environment. Aggregate success rate, cost, latency, tool errors, formatting correctness, or judge scores.

This evaluates the procedure, not one memorized output.

LLM-as-a-judge

An LLM judge can be useful for writing, design, architecture, and other partially subjective tasks, but stabilize it:

  • use a fixed rubric with independent criteria;
  • blind candidate identifiers and lineage;
  • treat candidate text as untrusted data, not judge instructions;
  • fix model, version, temperature, and budget;
  • randomize A/B order;
  • repeat judgments and aggregate robustly;
  • combine judge scores with deterministic checks;
  • keep private test prompts outside the candidate repository.

Noisy benchmarks

  • fix random seeds where possible;
  • warm up caches;
  • repeat measurements;
  • report median, variance, and sample count;
  • use equal budgets;
  • re-evaluate top candidates before final selection;
  • consider a score such as mean_quality - uncertainty_penalty.

Limitations

  • EvoFlow does not guarantee improvement. It is a search framework whose behavior depends on the agent, evaluator, budget, task distribution, and safety boundary.
  • A weak evaluator can be gamed or can reward superficial changes.
  • Visible validation data can be overfit.
  • The external runner and selection policy do not evolve during a run.
  • META.md changes future mutation prompts, but this is more constrained than allowing a candidate to rewrite the entire executable outer loop.
  • CLI flags and permission models can change between provider versions; inspect generated presets and run each provider command manually before a long experiment.
  • Automated scores do not replace code review, security review, license review, or domain expertise.

Relationship to HyperAgents

EvoFlow carries over these high-level ideas from the paper and public repository:

  • a combined task-level and meta-level editable candidate;
  • metacognitive modification of the future improvement procedure;
  • repeated parent selection, mutation, evaluation, and archive accumulation;
  • population-style exploration instead of one greedy chain;
  • persistent memory and performance history;
  • validation-driven selection with a separate final test;
  • sandboxing and human oversight as necessary deployment boundaries.

EvoFlow intentionally differs in these ways:

  • the executable outer runner is not candidate-editable;
  • the evaluator, protected paths, archive, and selection policy stay trusted;
  • meta-level self-modification is represented by inherited META.md rather than unrestricted infrastructure rewriting;
  • any compatible CLI is invoked as an external process;
  • promotion is always manual;
  • the implementation uses the Python standard library and does not include upstream source code.

These constraints trade some research openness for portability, reproducibility, and operational auditability.

Repository layout

recursive-task-optimizer/
├── README.md
├── LICENSE
├── NOTICE.md
├── agents/
│   └── openai.yaml
└── skills/
    └── recursive-task-optimizer/     # the installable skill
        ├── SKILL.md
        ├── LICENSE
        ├── scripts/
        │   ├── evoflow.py
        │   └── mock_agent.py
        └── references/
            ├── adapters.md
            ├── evaluator-contract.md
            ├── examples.md
            ├── native-install.md
            ├── protocol.md
            ├── research-basis.md
            ├── safety.md
            └── setup.md

Additional documentation

The main README is self-contained. More detailed references are bundled with the skill:

Development and validation

After changing the runner:

python3 -m py_compile skills/recursive-task-optimizer/scripts/*.py
python3 "$EVOFLOW" self-test

The self-test should finish with:

SELF-TEST PASSED

When comparing providers, create separate runs from the same base commit with the same objective, evaluator, seed, generation budget, model budget, and isolation policy. Compare held-out performance, validation-to-test gap, cost, duration, failure rate, and lineage diversity rather than validation score alone.

Citation

When referring to the original research, cite the HyperAgents paper rather than this implementation:

@misc{zhang2026hyperagents,
  title={Hyperagents},
  author={Jenny Zhang and Bingchen Zhao and Wannan Yang and Jakob Foerster and Jeff Clune and Minqi Jiang and Sam Devlin and Tatiana Shavrina},
  year={2026},
  eprint={2603.19461},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2603.19461}
}

Original sources:

License and attribution

The original EvoFlow code in this repository is provided under the MIT License.

The HyperAgents paper, official repository, datasets, models, trademarks, and related materials remain subject to their own terms. See NOTICE.md and references/research-basis.md.

About

Agent-agnostic, evaluator-driven recursive improvement loop for Claude Code, Codex CLI, Hermes, and other headless CLI agents

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages