Variant Configs & Sweep Tools - #6901
Conversation
3b6a9ef to
9345124
Compare
|
/cc @pmandewalkar |
| return sorted(all_metrics) | ||
|
|
||
|
|
||
| def get_median(values): |
There was a problem hiding this comment.
nit: i see that in sweep.py we import statistics and use that median, is there a reason why that's being redefined here?
There was a problem hiding this comment.
removed the custom get_median function and updated the script to use Python's built-in statistics.median().
| f"kubectl delete sandboxclaims --all -n {namespace} --ignore-not-found=true", | ||
| check=False, | ||
| ) | ||
| for _ in range(100): |
There was a problem hiding this comment.
Can we use the built in kubernetes wait function here?
There was a problem hiding this comment.
done, kubectl wait --for=delete.
pmandewalkar
left a comment
pmandewalkar
left a comment
There was a problem hiding this comment.
I see a good amount of YAML duplication across configs like baseline.yaml, c4a_arm.yaml, c4d_amd.yaml, larger_vm.yaml, etc., with mostly identical benchmark definitions. I think for overall easier maintenance and longevity could we use templating similar to how it is done in sched_tuner_daemonset.yaml.j2?
9345124 to
5d0ef01
Compare
5d0ef01 to
92472bb
Compare
@pmandewalkar PKB's |
3 participants
Files: ~20 new files
perfkitbenchmarker/data/k8s_agents/config/agentic_benchmark_config.yamlperfkitbenchmarker/data/k8s_agents/config/variants/baseline.yamlperfkitbenchmarker/data/k8s_agents/config/variants/kubelet_pulls.yamlperfkitbenchmarker/data/k8s_agents/config/variants/overlay_none.yamlperfkitbenchmarker/data/k8s_agents/config/variants/larger_vm.yamlperfkitbenchmarker/data/k8s_agents/config/variants/hyperdisk_200gb.yamlperfkitbenchmarker/data/k8s_agents/config/variants/thp.yamlperfkitbenchmarker/data/k8s_agents/config/variants/sched_tuning.yamlperfkitbenchmarker/data/k8s_agents/config/variants/swap.yamlperfkitbenchmarker/data/k8s_agents/config/variants/c4d_amd.yamlperfkitbenchmarker/data/k8s_agents/config/variants/c4a_arm.yamlperfkitbenchmarker/data/k8s_agents/config/variants/multi_node.yamlperfkitbenchmarker/data/k8s_agents/config/variants/variant_reference_guide.mdperfkitbenchmarker/data/k8s_agents/config/variants/tuning/kubelet_parallel_pulls.yamlperfkitbenchmarker/data/k8s_agents/config/variants/tuning/sched_tuner_daemonset.yaml.j2perfkitbenchmarker/data/k8s_agents/config/variants/tuning/swap_config.yamlperfkitbenchmarker/data/k8s_agents/config/variants/tuning/thp_config.yamlperfkitbenchmarker/scripts/agentic/sweep.pyperfkitbenchmarker/scripts/agentic/analyze.py.gitignoreadditionsDescription: Adds optimization sweep infrastructure:
k8s_snapshotkey including--enable-pod-snapshotsingke_additional_flags.agentic_benchmark_config.yaml) with YAML anchors for consistent cluster definitionssweep.py) — orchestrates multi-variant benchmark sweeps across all 7 benchmarks with warm pool drain between sweep levelsanalyze.py) — parses PKB NDJSON results, computes deltas vs baseline, generates markdown reports and CSV exports. Supports all 7 benchmarks via registry with auto-discovery fallback.