fix(api): include exhaustion reason in "Node exhausted" placement warning#3279
fix(api): include exhaustion reason in "Node exhausted" placement warning#3279AdaAibaby wants to merge 1 commit into
Conversation
PlaceSandbox logs "Node exhausted, trying another node" whenever the orchestrator returns codes.ResourceExhausted, but the gRPC error was silently dropped. The orchestrator has two distinct limits: - max running sandboxes per node (MaxSandboxesPerNode flag) - max concurrent sandbox starts (MaxStartingInstancesPerNode, default 3) Both return ResourceExhausted with different messages, and without the error there is no way to tell which limit was hit from the API logs. Add zap.Error(utils.UnwrapGRPCError(err)) to the Warn call, matching the pattern already used in the default (hard failure) branch below. Fixes: e2b-dev#3278
There was a problem hiding this comment.
Code Review
This pull request adds error logging to the sandbox placement warning when a node is exhausted by using utils.UnwrapGRPCError(err). Feedback indicates that utils.UnwrapGRPCError uses github.com/gogo/status instead of the standard google.golang.org/grpc/status, which prevents standard gRPC errors from being unwrapped correctly. It is recommended to update the helper function to use the standard gRPC status package.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Summary
Closes #3278
The
"Node exhausted, trying another node"Warn log inPlaceSandboxdiscarded the gRPC error from the orchestrator, making it impossible to tell from API logs which resource limit was hit.Change
One-line fix in
packages/api/internal/orchestrator/placement/placement.go:This matches the pattern already used in the
defaultbranch two lines below.Before / After
Before:
After:
or
Now you can tell at a glance whether the node hit its concurrent-start limit (
MaxStartingInstancesPerNode, default 3) or its total-sandbox limit (MaxSandboxesPerNode).Test plan
"Node exhausted"Warn log now includes the error field with the specific reason