-
Notifications
You must be signed in to change notification settings - Fork 5
fix(benchmark): install the authorities the observation path needs #723
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -25,6 +25,50 @@ use tracedecay_sessions::runtime::{codex, cursor, hermes, kiro}; | |
| use tracedecay_usecases::host_admission::HostAdmissionScope; | ||
| use tracedecay_usecases::observation::ObservationCancellation; | ||
|
|
||
| /// Installs the process-wide background CPU authority these benchmarks need. | ||
| /// | ||
| /// They drive the real observation pipeline, and host admission refuses every | ||
| /// capture with `background_cpu_unavailable` when no authority is installed. | ||
| /// Production installs it during daemon bootstrap, which a benchmark never | ||
| /// runs, and `tracedecay-global-db`'s equivalent harness helper is | ||
| /// `#[cfg(test)] pub(crate)`, so it cannot be reused from here. | ||
| fn ensure_background_cpu_authority() { | ||
| use std::num::NonZeroUsize; | ||
| use tracedecay_runtime_core::background_cpu::{ | ||
| install_process_background_cpu, process_background_cpu, | ||
| }; | ||
|
|
||
| use std::sync::{Arc, OnceLock}; | ||
| use tracedecay_runtime_core::resident_memory::{ | ||
| DEFAULT_PROCESS_RESIDENT_MEMORY_LIMIT_V1, ProcessResidentMemoryV1, | ||
| }; | ||
|
|
||
| static BENCHMARK_RESIDENT_MEMORY: OnceLock<Arc<ProcessResidentMemoryV1>> = OnceLock::new(); | ||
|
|
||
| if process_background_cpu().is_none() { | ||
| // A sibling benchmark can win the process-wide installation race at a | ||
| // different canonical width; reuse that authority rather than making | ||
| // success depend on execution order. | ||
| if install_process_background_cpu(NonZeroUsize::MIN).is_err() { | ||
| assert!( | ||
| process_background_cpu().is_some(), | ||
| "background CPU authority is neither installable nor already installed" | ||
| ); | ||
| } | ||
| } | ||
|
|
||
| // The Codex provider path additionally refuses with | ||
| // `BackgroundResourceUnavailable { resource: "process resident-memory | ||
| // authority" }` until preparation resources are configured. | ||
| let memory = Arc::clone(BENCHMARK_RESIDENT_MEMORY.get_or_init(|| { | ||
| Arc::new(ProcessResidentMemoryV1::new( | ||
| DEFAULT_PROCESS_RESIDENT_MEMORY_LIMIT_V1, | ||
| )) | ||
| })); | ||
| let _ = tracedecay_sessions::runtime::codex::CodexDiscoveryHub::default() | ||
| .configure_preparation_resources(memory); | ||
|
Comment on lines
+68
to
+69
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When a nonignored benchmark fixture runs before daemon bootstrap tests in the same AGENTS.md reference: AGENTS.md:L82-L84 Useful? React with 👍 / 👎. |
||
| } | ||
|
|
||
| use super::artifact::{ | ||
| attest_build, command_output, git_snapshot, validate_git_snapshots, workload_identity, | ||
| }; | ||
|
|
@@ -81,6 +125,7 @@ impl Fixture { | |
| .expect("create Claude benchmark fixture tree"); | ||
| fs::create_dir_all(&profile).expect("create benchmark profile root"); | ||
| write_records(&transcript, &session_id); | ||
| ensure_background_cpu_authority(); | ||
| let runtime = HostAdmissionTestRuntimeV1::profile(&profile) | ||
| .await | ||
| .expect("open registered benchmark runtime"); | ||
|
|
@@ -397,6 +442,7 @@ impl ProviderFixture { | |
| let project_id = enroll_provider_benchmark_project(&project); | ||
| let source_path = | ||
| write_provider_fixture(kind, temp.path(), &home, &project, repetition).await; | ||
| ensure_background_cpu_authority(); | ||
| let runtime = if matches!(kind, ProviderKind::Claude) { | ||
| HostAdmissionTestRuntimeV1::profile(&profile) | ||
| .await | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On a fresh run of the documented benchmark command, no daemon authority exists, so this permanently installs width 1. Codex derives
shared_jsonl_preparation_workers()from that width and acquires those permits around its Rayon frame-preparation work, whereas production installs the worker plan'seffective_workers; the provider benchmark therefore serializes Codex preparation and records non-production latency and throughput. Initialize the canonical production worker plan rather than using the minimum width.AGENTS.md reference: AGENTS.md:L7-L12
Useful? React with 👍 / 👎.