Redesign VeRO as a generic program optimization harness - #41

Closed

varunursekar wants to merge 70 commits into

mainfrom

Commits on Jul 15, 2026

Build canonical evaluation and optimization kernel
varunursekar
committed
35d3b2c
Add generic optimization runtime and coding-agent gateway
varunursekar
committed
631f1c2
Migrate coding agents to canonical runtime
varunursekar
committed
1279b78
Add canonical program optimization CLI
varunursekar
committed
e27988e
Extract Python task protocol from legacy core
varunursekar
committed
6365e8a
Enforce external Python evaluation boundaries
varunursekar
committed
f4da52b
Migrate benchmarking to canonical runtime
varunursekar
committed
8cb3fba
Add Harbor evaluation backend
varunursekar
committed
997ae68
Add canonical Harbor sidecar and verifier
varunursekar
committed
44e8192
Expose Harbor sidecar HTTP and CLI surfaces
varunursekar
committed
73d983d
Compile canonical Harbor optimization tasks
varunursekar
committed
818e3cc
Exclude generated Harbor compiler caches
varunursekar
committed
50f95e4
Refresh benchmarking lock for Harbor extra
varunursekar
committed
95cd51d
Remove paper-era implementation from v0.5
varunursekar
committed
bfaea83
Harden generic program optimization workflow
varunursekar
committed
f38dfba
Resolve concurrency and persistence review findings
varunursekar
committed
b0ae83d
Support isolated sandbox execution
varunursekar
committed
68f58a6
Separate candidate and evaluation limits
varunursekar
committed
34982ce
Harden evaluation persistence and process cleanup
varunursekar
committed
5dd4f26
Support nested Harbor evaluation on Modal
varunursekar
committed
9c95d96
Document nested Modal Harbor workflow
varunursekar
committed
4b3f2c7

Commits on Jul 16, 2026

Commits on Jul 17, 2026

Implement multi-evaluation optimization protocol
varunursekar
committed
e9b45f6
Restore failed-case evaluation semantics
varunursekar
committed
3db69df
Pass Harbor objectives to coding agents
varunursekar
committed
aa7744a
Add circle packing optimization benchmark
varunursekar
committed
5255f5d
Add self-contained experiment reports
varunursekar
committed
272c772
Add Harbor-native GAIA optimization benchmark
varunursekar
committed
aef04eb
Rename benchmark suite to program-opt-bench
varunursekar
committed
55609a0
Add durable Harbor optimization workflows
varunursekar
committed
5df8360
Drain in-flight evaluations before Harbor finalization
varunursekar
committed
1e3d93c
Add durable Harbor evaluation jobs
varunursekar
committed
e467616
Allow GPT-5.5 for GAIA optimization
varunursekar
committed
df700a7
harbor: lock the admin token to the trusted verifier (0o400 file, 0o700 dir)

varunursekar
and
claude
committed
3a3031d
harbor: offload verifier finalize's atomic write to a thread

varunursekar
and
claude
committed
9533e99

Commits on Jul 18, 2026

harbor: ${...} run-time parameters in build YAML (no rebuild to vary knobs)

varunursekar
and
claude
committed
96ffbb4
harbor: document the soft fixed-target guarantee at the gateway allow-list

varunursekar
and
claude
committed
4918ea3
native: remove token-limit hook; bump openai-agents 0.13.6 -> 0.18.3

varunursekar
and
claude
committed
bb9930a
native: rebuild VeroAgent on SandboxAgent; drop run_agent_with_json_sanitization

varunursekar
and
claude
committed
87c3ccb
native: delete replaced tools + filesystem ACL; strip ClaudeCodeAgent hooks

varunursekar
and
claude
committed
5e149f7
native: drive VeroAgent via the OpenAI Responses API (SandboxAgent needs it)

varunursekar
and
claude
committed
98d7303
native: single model-agnostic path — function tools over a sandbox session

varunursekar
and
claude
committed
616e8db
optimization: GenerationBackend seam over candidate production

varunursekar
and
claude
committed
219a5fb
optimization: EvolutionaryStrategy (population-based, N offspring/round)

varunursekar
and
claude
committed
5bd572f
runtime: stream agent activity to the event bus (observability)

varunursekar
and
claude
committed
0a4a95c
cleanup: remove vestigial classes/functions orphaned by the refactor

varunursekar
and
claude
committed
e78f8b8
chore: gitignore harbor jobs/ output directory

varunursekar
and
claude
committed
975caa8
chore: move matmul-{kernel,eval} examples to vero-tasks

varunursekar
and
claude
committed
837c855
examples: add a Harbor outer-loop / simple inner-loop example

varunursekar
and
claude
committed
1a642e3
docs: rewrite the agent setup guide for the current architecture

varunursekar
and
claude
committed
41e0eb7
docs: refresh README for the modernized native runner

varunursekar
and
claude
committed
473d5b8
docs: frame VeRO as optimizing programs, text, AND agents

varunursekar
and
claude
committed
629063c
docs: add badges, mermaid figures, highlights, and a callout to the READMEs

varunursekar
and
claude
committed
69ece41
docs: stop advertising `pip install scale-vero` (squatted PyPI name)

varunursekar
and
claude
committed
36c190e
chore: add gitignored runs/ dir for local GAIA/harbor run outputs

varunursekar
and
claude
committed
7719ec0

Commits on Jul 19, 2026

Add Claude optimizer support + injectable insights bundle to Harbor

varunursekar
and
claude
committed
12a99b9
README: lead with Harbor and break up the text walls

varunursekar
and
claude
committed
fe92b38
README: break up the remaining text walls

varunursekar
and
claude
committed
88cc42f
Unblock tau3 (and other env-declaring tasks) via non-interactive sub-runs

varunursekar
and
claude
committed
0932d2d
Add DarwinGodelStrategy: open-ended archive evolution

varunursekar
and
claude
committed
6968e7a
tau3: collapse to a single host-side agent; delete runner.py + cred forward

varunursekar
and
claude
committed
282f5c8
harbor: add disclose_budget toggle for budget-blind optimization

varunursekar
and
claude
committed
8bd5655
harbor: route task-owned eval LLM services to upstream (fix tau3 user-sim/grader)

varunursekar
and
claude
committed
3c106b1
harbor: give the trusted eval-sidecar the upstream for task_services_use_upstream

varunursekar
and
claude
committed
f53ce9b
program-opt-bench: add ALE-Bench (program optimization) task

varunursekar
and
claude
committed
2cdd60e
program-opt-bench: add OfficeQA (grounded-reasoning agent benchmark)

varunursekar
and
claude
committed
d285d63

Commits on Jul 20, 2026

Reserve a finalization inference budget so re-scoring cannot be starved

varunursekar
and
claude
committed
2ff7e95