Python Density & Payload Benchmarks - #6897
Conversation
bdab832 to
445ff49
Compare
|
/cc @Nishanth29 |
|
|
||
| successful = result.get("successful_sessions", 0) | ||
| failed = result.get("failed_sessions", 0) | ||
| agg = result.get("aggregate", {}) |
There was a problem hiding this comment.
If the API returns aggregate as null on an error, agg.get(...) below might throw a NoneType error. maybe we can do agg = result.get("aggregate") or {} instead?
There was a problem hiding this comment.
updated across all benchmarks.
|
|
||
| successful = result.get("successful_sessions", 0) | ||
| failed = result.get("failed_sessions", 0) | ||
| agg = result.get("aggregate", {}) |
There was a problem hiding this comment.
same here, if aggregate comes back null on a failure, result.get("aggregate", {}) returns None. we can use result.get("aggregate") or {} here too
There was a problem hiding this comment.
yes, updated across all the benchmark scripts.
| "ms", | ||
| ns, | ||
| extra, | ||
| ) |
There was a problem hiding this comment.
Lots of the emit seems duplicates across benchmarks. Can you create some reusable utilities to simplify the code?
There was a problem hiding this comment.
Did some optimizations by creating two reusable utilities in k8s_benchmark_utils.py (EmitSampleIfPresent & EmitPercentileStats)
| # Build samples | ||
| run_id = str(uuid.uuid4())[:8] | ||
|
|
||
| extra = { |
There was a problem hiding this comment.
What does extra mean and what's it for? Please document
There was a problem hiding this comment.
It holds additional metadata key-value pairs (like run IDs and sweep parameters) that are appended to every pkb Sample (can be used for run correlation, downstream dashboard filtering, ...etc). I added a comment block above the extra dictionary in all benchmarks explaining that.
3 participants
Files: 2 new files
perfkitbenchmarker/linux_benchmarks/kubernetes/agentic/k8s_python_density_benchmark.pyperfkitbenchmarker/linux_benchmarks/kubernetes/agentic/k8s_payload_benchmark.pyDescription: Adds two Python-focused GKE Agent Sandbox benchmarks:
Python Density: Measures Code Execution Latency (CEL), TTFE, RSS memory growth, and per-type latency breakdown (compute, syscall, import) at varying concurrent sandbox counts. Designed for density saturation sweeps.
Payload Transfer: Measures the cost of returning large observation payloads from gVisor sandboxes. Breaks down generation, serialization, stdout write, and total transfer time with throughput metrics.