ADK Agent Container Image by george-kalisse-sada · Pull Request #6894 · GoogleCloudPlatform/PerfKitBenchmarker

george-kalisse-sada · 2026-07-17T03:12:13Z

Files: ~13 new files (no Python benchmark logic)

perfkitbenchmarker/data/docker/agentic/adk-agent/README.md
perfkitbenchmarker/data/docker/agentic/adk-agent/Dockerfile
perfkitbenchmarker/data/docker/agentic/adk-agent/__init__.py
perfkitbenchmarker/data/docker/agentic/adk-agent/.dockerignore
perfkitbenchmarker/data/docker/agentic/adk-agent/.gcloudignore
perfkitbenchmarker/data/docker/agentic/adk-agent/cloudbuild-arm64.yaml
perfkitbenchmarker/data/docker/agentic/adk-agent/requirements.txt
perfkitbenchmarker/data/docker/agentic/adk-agent/main.py
perfkitbenchmarker/data/docker/agentic/adk-agent/gke_performance_agent/__init__.py
perfkitbenchmarker/data/docker/agentic/adk-agent/gke_performance_agent/agent.py
perfkitbenchmarker/data/docker/agentic/adk-agent/sandboxed_apps/python_test_app/benchmark_density.py
perfkitbenchmarker/data/docker/agentic/adk-agent/sandboxed_apps/python_test_app/benchmark_payload.py
perfkitbenchmarker/data/docker/agentic/adk-agent/sandboxed_apps/python_test_app/benchmark_qps.py

Description: Adds the ADK Agent container image — a FastAPI service deployed inside the GKE cluster that the PKB benchmark modules interact with via HTTP. Includes:

FastAPI endpoints for density, payload, QPS, and Chromium benchmarks
Mock LLM agent definition using Google ADK with BenchmarkGkeCodeExecutor
3 benchmark scripts that execute inside gVisor sandboxes (density, payload, QPS)
Dockerfile, requirements, and Cloud Build config for ARM64 cross-compilation

Nothing in PKB imports these files — they are copied into a container image at build time.

roycaihw · 2026-07-22T23:14:47Z

@@ -0,0 +1,56 @@
+# ADK Agent Container
+
+FastAPI service deployed inside the GKE cluster that serves as the


Replace "GKE" with "Kubernetes"

roycaihw · 2026-07-22T23:16:26Z

+    sandbox-side metrics.
+    """
+    async with _benchmark_lock:
+        os.environ["BENCHMARK_MODE"] = "density"


Not urgent, but maybe for a followup PR:

Using os env to pass configs is not very intuitive, and also has the risk of race conditions. Consider using the prompt to pass the configs

POST /benchmark/python/density | v FastAPI service | | (prompt: {mode: "density", sample_count: 100...}) v ADK agent | | (prompt: {mode: "density", sample_count: 100...}) v MockLLM | | (script instruction based on the prompt) v ADK agent (execution)

@roycaihw Using os.environ is a design choice to keep the Python benchmark scripts static. If we passed parameters via the prompt, our MockLlm would have to perform string-replacement on the Python script's source code to inject the variables (e.g., changing SAMPLE_COUNT = 20 to 100) before sending it to the sandbox. With os.environ, the Python script remains static, and we cleanly inject the variables into the sandbox via standard Linux environment variables (/bin/sh -c 'SAMPLE_COUNT=100 python3 script.py').

Regarding race conditions: we explicitly use an asyncio.Lock() (_benchmark_lock) around the endpoint handlers to serialize benchmark requests, ensuring concurrent POSTs cannot clobber the shared environment variables.

roycaihw · 2026-07-22T23:16:41Z

+            req.concurrent_sessions,
+        )
+
+        prompt = "Please start the GKE performance benchmark workflow."


The prompt is not used, correct? Can we drop it?

You are correct that the content of the prompt is ignored by our MockLlm. However, the ADK Runner.run_async() method requires a non-empty string to initiate the conversation loop and trigger the first LLM generation. I can simplify the string to just "start" to make it clear that it is simply a trigger, but we cannot drop the variable entirely.

roycaihw · 2026-07-22T23:16:55Z

+    )
+
+
+class ChromiumBenchmarkRequest(BaseModel):


Right now the main.py combines all the endpoints and their Request class. For better readability, can you split each endpoint and its Request class into a separate file?

I'll refactor into a "modular" FastAPI structure with an api/routes/ directory, moving each endpoint and its models into separate files.

roycaihw · 2026-07-22T23:17:08Z

+_benchmark_lock = asyncio.Lock()
+
+
+def _percentile_stats(sorted_values: list, prefix: str) -> dict:


Can you turn this function into a util for reuse? You have two methods (_percentile_stats and _percentile). Make them consistent.

I'll consolidate the percentile logic into a shared api/utils.py file, sure.

roycaihw · 2026-07-22T23:17:18Z

+            logging.exception("Error shutting down ThreadPoolExecutor")
+
+
+app = FastAPI(title="GKE Benchmark Agent", version="0.2.0", lifespan=lifespan)


Let's replace GKE with Kubernetes

roycaihw · 2026-07-22T23:17:42Z

+  POST /benchmark/chromium/density   → run the Chromium density benchmark
+  POST /run                          → raw ADK agent interaction
+
+POST /benchmark/python/density — Request:


The documentation seems incomplete as it only has the request and response for POST /benchmark/python/density. I'd recommend putting the API endpoint documentation in a separate doc file.

I'll move the API doc out of main.py and into a new API_DOCS.md file.

roycaihw · 2026-07-22T23:20:38Z

+elapsed_ms = (time.perf_counter() - t0) * 1000
+
+print(json.dumps({
+    "sandbox_status": "ok",


Let's make this a "success" v.s. "fail" based on whether result matches the expectation (49,995,000-- the sum of all integers from 0 up to 9,999)

roycaihw · 2026-07-22T23:21:03Z

+print(json.dumps({
+    "sandbox_status": "ok",
+    "sandbox_qps_exec_ms": round(elapsed_ms, 3),
+    "sandbox_compute_result": result,


Let's drop the "sandbox_compute_result" since it's just confusing to people unless they read the actual source of what the sandbox does.

roycaihw · 2026-07-22T23:21:19Z

+
+print(json.dumps({
+    "sandbox_status": "ok",
+    "sandbox_qps_exec_ms": round(elapsed_ms, 3),


Let's drop "qps" from the metrics name since this metrics measures how long a simple python sandbox execution takes-- nothing related to qps here.

hubatish · 2026-07-24T16:52:56Z

Thanks for the review Roy. Give me a ping when Roy's lgtm'ed & I'll add the lgtm + tag which will start the merging process

roycaihw

Left some comments. LGTM otherwise

roycaihw · 2026-08-13T22:05:24Z

+router = APIRouter()
+
+class RunRequest(BaseModel):
+    prompt: str = "start"


nit: "start" seems unused. Consider removing it.

roycaihw · 2026-08-13T22:06:05Z

+        output = await run_agent(req.prompt)
+        return {"response": output}
+    except Exception as e:
+        logger.exception("Agent run failed")


Please also log the exception e

roycaihw · 2026-08-13T22:10:51Z

+  "sample_count": 100,
+  "sample_warmup": 5,
+  "concurrent_sessions": 1,
+  "sandbox_exec_timeout_s": 60


The file is titled API_DOCS but it seems to only contain some examples with no description? Please document the request and response API schema.

roycaihw · 2026-08-13T22:11:11Z

+### `POST /benchmark/python/qps`
+Runs the QPS saturation benchmark.
+**Request:**
+


Missing ```json

roycaihw · 2026-08-13T22:34:02Z

+app = App(
+    name=root_agent.name,
+    root_agent=root_agent,
+    # enable_tracing=True,


Remove unused code?

roycaihw · 2026-08-13T22:34:30Z

+            logging.info("SANDBOX_TIMINGS_DELETE: delete_ms=%.3f", delete_ms)
+
+
+gke_executor = BenchmarkGkeCodeExecutor(


Replace gke with kubernetes

roycaihw · 2026-08-13T22:34:52Z

+
+gke_executor = BenchmarkGkeCodeExecutor(
+    cluster_name=os.getenv("CLUSTER_NAME"),
+    location=os.getenv("GOOGLE_CLOUD_LOCATION"),


Drop the "GOOGLE_CLOUD_" prefix

roycaihw · 2026-08-13T22:35:12Z

+_SANDBOX_POOL = ThreadPoolExecutor(max_workers=_sandbox_io_workers())
+
+
+class BenchmarkGkeCodeExecutor(GkeCodeExecutor):


Drop "GkeCode"

roycaihw reviewed Jul 22, 2026

View reviewed changes

ADK Agent Container Image

0e7ea72

george-kalisse-sada force-pushed the SADA_1b_adk_agent_container branch from 20a9108 to 0e7ea72 Compare August 5, 2026 17:32

roycaihw reviewed Aug 13, 2026

View reviewed changes

		@@ -0,0 +1,56 @@
		# ADK Agent Container

		FastAPI service deployed inside the GKE cluster that serves as the

		_benchmark_lock = asyncio.Lock()


		def _percentile_stats(sorted_values: list, prefix: str) -> dict:

		logging.exception("Error shutting down ThreadPoolExecutor")


		app = FastAPI(title="GKE Benchmark Agent", version="0.2.0", lifespan=lifespan)

		logging.info("SANDBOX_TIMINGS_DELETE: delete_ms=%.3f", delete_ms)


		gke_executor = BenchmarkGkeCodeExecutor(

		_SANDBOX_POOL = ThreadPoolExecutor(max_workers=_sandbox_io_workers())


		class BenchmarkGkeCodeExecutor(GkeCodeExecutor):

Uh oh!

Conversation

george-kalisse-sada commented Jul 17, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hubatish commented Jul 24, 2026

Uh oh!

roycaihw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants