Shared Utilities & Infrastructure - #6896
Conversation
9708030 to
dc93fba
Compare
|
/gcbrun |
|
/cc @Nishanth29 |
|
Hi @george-kalisse-sada , it looks like the CI check Pull (bionic-baton-343) failed during execution. could you check the build failure and rerun/resolve the CI run so all checks pass? |
|
Per @george-kalisse-sada, the pull failure was likely a transient timeout issue. Zach synced to head and re-ran the check, which passed successfully(documenting here for visibility). |
| """Run a shell command, raising on failure.""" | ||
| logger.info(" CMD: %s", " ".join(cmd)) | ||
| env = os.environ.copy() | ||
| env["CLOUDSDK_AUTH_DISABLE_SSL_VALIDATION"] = "true" |
There was a problem hiding this comment.
Is CLOUDSDK_AUTH_DISABLE_SSL_VALIDATION needed here or was it left over from local testing? Can we drop this line?
| rendered = template.render(**kwargs) | ||
|
|
||
| # Write rendered YAML to tmp dir (RunKubectlCommand does not support stdin) | ||
| tmp_dir = os.path.join( |
There was a problem hiding this comment.
Doesn't writing rendered manifests to data.ResourcePath(...) creates temp files inside the repository source tree?? ig we use PKB's managed temp dir instead so it doesn't mess git or fail in read-only installs.
| import threading | ||
|
|
||
|
|
||
| _PID_FILE = "/tmp/pkb_portforward.pid" |
There was a problem hiding this comment.
Wondering if two sweeps running in parallel might overwrite this /tmp file? Maybe we can put it in PKB's temp directory or add the port to the filename.
| @@ -0,0 +1,54 @@ | |||
| # GKE Agent Sandbox Benchmarks | |||
There was a problem hiding this comment.
Replace "GKE" with "Kubernetes". Please also update other places that use "GKE"
|
|
||
| PKB benchmark modules for measuring GKE Agent Sandbox performance under | ||
| gVisor isolation. Each benchmark is an atomic single-point measurement | ||
| designed to be invoked repeatedly by the sweep runner with varying parameters. |
There was a problem hiding this comment.
Each benchmark is a standalone PKB benchmark.
| | Python Density | `k8s_python_density_benchmark.py` | Yes | CEL, TTFE, RSS, per-type latency (compute/syscall/import) | | ||
| | Payload Transfer | `k8s_payload_benchmark.py` | Yes | Generation, serialization, stdout write, throughput | | ||
| | Chromium Density | `k8s_chromium_density_benchmark.py` | Yes | Navigate, evaluate, click, fill, screenshot latency | | ||
| | QPS Saturation | `k8s_qps_benchmark.py` | Yes/No | TTFE at controlled request rates, warm pool drain detection | |
| flags.DEFINE_string( | ||
| "k8s_agentic_benchmark_note", | ||
| "", | ||
| "Arbitrary note string attached to every sample for tagging runs.", |
There was a problem hiding this comment.
Please elaborate. What does note mean here and what's is for?
| Delegates to PKB's native kubectl module which handles kubeconfig | ||
| and retries for transient connection errors automatically. | ||
| """ | ||
| return kubectl.RunKubectlCommand( |
There was a problem hiding this comment.
Why do we need to duplicate this? This method adds no value but complexity.
| ] | ||
| ) | ||
| _RunCmd( | ||
| ["git", "sparse-checkout", "set", "examples/chrome-sandbox"], |
There was a problem hiding this comment.
Are we simply building example code from https://github.com/kubernetes-sigs/agent-sandbox? Ideally this should be built by the agent-sandbox repo so we can simply reuse published images.
| "git", | ||
| "sparse-checkout", | ||
| "set", | ||
| "clients/python/agentic-sandbox-client/sandbox-router", |
There was a problem hiding this comment.
Same here. Are we simply building example code from https://github.com/kubernetes-sigs/agent-sandbox? Ideally this should be built by the agent-sandbox repo so we can simply reuse published images.
| @@ -0,0 +1,74 @@ | |||
| # GKE Agentic Benchmark Scripts | |||
There was a problem hiding this comment.
Replace "GKE" with "Kubernetes". You should scan all your code changes, find places that use "GKE" and update accordingly
|
|
||
| ### Sweep Runner | ||
|
|
||
| `sweep.py` orchestrates multi-variant benchmark sweeps. It calls PKB |
There was a problem hiding this comment.
I don't see sweep.py in this PR. Ideally we should remove this paragraph from this PR. We can discuss whether sweep.py belongs to PKB repo in a separate PR.
| logger.info("Snapshot bucket deleted.") | ||
|
|
||
|
|
||
| def teardown_images(project_id: str, region: str) -> None: |
There was a problem hiding this comment.
Do these teardown operations happen on-demand, or do they happen during each benchmark? I'd expect it to be the former (otherwise it will be very time-consuming to do the unnecessary re-build), but just want to confirm with you.
4 participants
Files: ~10 new files
perfkitbenchmarker/linux_benchmarks/kubernetes/__init__.pyperfkitbenchmarker/linux_benchmarks/kubernetes/agentic/__init__.pyperfkitbenchmarker/linux_benchmarks/kubernetes/agentic/README.mdperfkitbenchmarker/linux_benchmarks/kubernetes/agentic/k8s_benchmark_utils.pyperfkitbenchmarker/linux_benchmarks/kubernetes/agentic/gke_deploy_utils.pyperfkitbenchmarker/scripts/agentic/__init__.pyperfkitbenchmarker/scripts/agentic/README.mdperfkitbenchmarker/scripts/agentic/gke_image_build_utils.pyperfkitbenchmarker/scripts/agentic/gke_prerequisites.pyperfkitbenchmarker/scripts/agentic/gke_post_teardown.pyDescription: Adds shared infrastructure for GKE Agent Sandbox benchmarks:
k8s_benchmark_utils.py— Agent API interaction, kubectl helpers, warm pool management, port-forward manager, PKB sample constructiongke_deploy_utils.py— Idempotent deployment of Agent Sandbox ecosystem (CRDs, SandboxTemplates, WarmPools, Router, ADK Agent) onto pre-provisioned GKE clustersgke_image_build_utils.py— Container image builds (Chrome Sandbox, Sandbox Router) via Google Cloud Build with cross-architecture support (amd64/arm64)gke_prerequisites.py— One-time GCP project setup (APIs, Artifact Registry, Cloud Build SA, image builds including native ARM64 VM builds)gke_post_teardown.py— Cleanup of resources created by prerequisites