Skip to content

docs: document GCP Batch infrastructure retry for Fusion Snapshots#1640

Open
justinegeffen wants to merge 6 commits into
masterfrom
docs/gcp-batch-snapshot-retry
Open

docs: document GCP Batch infrastructure retry for Fusion Snapshots#1640
justinegeffen wants to merge 6 commits into
masterfrom
docs/gcp-batch-snapshot-retry

Conversation

@justinegeffen

@justinegeffen justinegeffen commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Jira: EDU-1143

What

The Fusion Snapshots retry documentation only covered Spot reclamation (exit code 50001) and errorStrategy. Google Cloud Batch surfaces a broader set of infrastructure failures as exit codes in the 5000159999 range, and by default only 50001 is auto-retried. On Spot-based snapshot runs, the other codes cause avoidable failures, and this configuration requirement was undocumented.

Changes

  • fusion_docs/guide/snapshots/configuration.md: add a "Retrying Google Cloud Batch infrastructure failures" subsection under Retry handling — the exit-code range table, how to extend google.batch.autoRetryExitCodes (with maxSpotAttempts), guidance on which codes to retry, and a note on why errorStrategy keyed on task.exitStatus is unreliable for these codes.
  • fusion_docs/guide/snapshots/gcp.md: add a one-line pointer from the GCP snapshots page to the new section.

Verification

Confirmed against the current Nextflow source (nf-google, v26.04.3):

  • HPC/grid and local executors aside, Google Batch retry is governed by google.batch.maxSpotAttempts plus google.batch.autoRetryExitCodes (default [50001]); Fusion Snapshots set a non-zero maxSpotAttempts automatically.
  • Exit codes >= 50000 are read from the task exit file, so errorStrategy conditions on task.exitStatus do not reliably match infrastructure codes — hence the autoRetryExitCodes-first guidance.

🤖 Generated with Claude Code

The Fusion Snapshots retry docs only covered Spot reclamation and
errorStrategy. Google Cloud Batch surfaces a range of infrastructure
failures as exit codes 50001-59999, and by default only 50001 is
auto-retried. On Spot-based snapshot runs this leads to avoidable
failures.

Add a "Retrying Google Cloud Batch infrastructure failures" subsection
to the snapshots advanced-configuration page documenting the exit-code
range, how to extend google.batch.autoRetryExitCodes, and why
errorStrategy on task.exitStatus is unreliable for these codes. Add a
pointer from the GCP snapshots page.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@netlify

netlify Bot commented Jul 8, 2026

Copy link
Copy Markdown

Deploy Preview for seqera-docs failed. Why did it fail? →

Name Link
🔨 Latest commit f43896e
🔍 Latest deploy log https://app.netlify.com/projects/seqera-docs/deploys/6a4fdc856f5c94000870ee7b

@justinegeffen justinegeffen added 1. Dev/PM/SME Needs a review by a Dev/PM/SME deslop-done labels Jul 8, 2026

@alberto-miranda alberto-miranda left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM;

@justinegeffen justinegeffen added 3. Dev/PM/SME reviews complete SMEs have reviewed and approved. and removed 1. Dev/PM/SME Needs a review by a Dev/PM/SME labels Jul 8, 2026
@justinegeffen justinegeffen added the 1. Editor review Needs a language review label Jul 8, 2026
Comment thread fusion_docs/guide/snapshots/configuration.md Outdated
Comment thread fusion_docs/guide/snapshots/configuration.md Outdated
Signed-off-by: Justine Geffen <justinegeffen@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1. Editor review Needs a language review 3. Dev/PM/SME reviews complete SMEs have reviewed and approved. deslop-done

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants