feat(storage): native Google Cloud Storage support for self-hosting by waleedlatif1 · Pull Request #5728 · simstudioai/sim

waleedlatif1 · 2026-07-17T06:46:08Z

Summary

Adds Google Cloud Storage as a third object-storage backend alongside AWS S3 and Azure Blob, so Sim can be self-hosted entirely on GCP
Full parity with the existing providers: uploads, streaming downloads, deletes, head, bulk delete, V4 presigned URLs (single + batch), and large-file multipart uploads (browser direct + server streaming) via the GCS XML API
Auth via Application Default Credentials (GKE Workload Identity / GOOGLE_APPLICATION_CREDENTIALS) or inline service-account JSON (GCS_CREDENTIALS_JSON); signed URLs fall back to IAM signBlob when no private key is present
Per-context buckets (GCS_BUCKET_NAME, GCS_KB_BUCKET_NAME, GCS_EXECUTION_FILES_BUCKET_NAME, chat/copilot/profile-pictures/og-images/workspace-logos) matching the S3/Azure layout; selection precedence is Azure Blob > S3 > GCS > local disk
Replaces the hardcoded USE_BLOB_STORAGE ? 'blob' : 's3' serve-path literals with a shared getServeStoragePrefix() helper
Docs: GCS setup section in object-storage.mdx (bucket creation, IAM, CORS, Workload Identity), environment-variables.mdx, .env.example, helm values.yaml + a real storage section in values-gcp.yaml

Type of Change

New feature

Testing

29 new tests for the GCS provider client (auth modes, uploads, signed URLs, downloads with size caps, head/delete, XML multipart initiate/part/complete/abort)
Full uploads + files API suites pass (420 tests), typecheck clean, check:api-validation passes, helm lint + helm template with values-gcp.yaml render correctly

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

vercel · 2026-07-17T06:46:14Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview, Comment	Jul 17, 2026 7:14am

cursor · 2026-07-17T06:46:17Z

PR Summary

Medium Risk
Touches core file upload/serve/multipart paths and credential handling (ADC vs inline keys, IAM signBlob for signed URLs); mistakes could break uploads or mis-route serve URLs, but behavior mirrors proven S3/Blob patterns with new provider tests.

Overview
Adds Google Cloud Storage as a production object-storage backend (after Azure Blob and S3, before local disk), enabling full GCP self-hosting without S3 or Blob.

The app wires GCS through the same upload stack as existing providers: env-driven bucket config per file context, @google-cloud/storage for uploads/downloads/presigned URLs, and GCS XML API multipart for large browser uploads. Serve URLs use a new gcs/ prefix via getServeStoragePrefix(), replacing hardcoded blob vs s3 branching across presigned, export, view, multipart, and embed parsing.

Docs and ops: self-hosting guides, .env.example, and Helm values-gcp.yaml document buckets, CORS, Workload Identity, and optional inline GCS_CREDENTIALS_JSON.

^{Reviewed by Cursor Bugbot for commit b2dec5d. Configure here.}

greptile-apps · 2026-07-17T06:49:25Z

Greptile Summary

This PR adds native Google Cloud Storage support for self-hosted deployments. The main changes are:

GCS uploads, downloads, deletes, metadata, signed URLs, and multipart uploads.
Context-specific buckets with fallback to the general GCS bucket.
GCS support across file routes, authorization, and serve paths.
Environment, Helm, and self-hosting documentation updates.
Provider and route tests for the new storage paths.

Confidence Score: 5/5

This looks safe to merge.

The context bucket fallback is consistent across the reviewed GCS paths.
Browser multipart ETags are normalized before GCS completion.
Chat authorization now resolves the same bucket configuration as storage operations.
No blocking issue remains in the updated fixes.

Important Files Changed

Filename	Overview
apps/sim/lib/uploads/config.ts	Adds GCS provider selection and consistent context-specific bucket fallback.
apps/sim/lib/uploads/providers/gcs/client.ts	Implements GCS object operations, signed URLs, and XML multipart uploads.
apps/sim/app/api/files/multipart/route.ts	Adds GCS multipart initiation, part URL generation, completion, and abort handling.
apps/sim/app/api/files/authorization.ts	Uses the shared context resolver for chat storage authorization.

_{Reviews (4): Last reviewed commit: "fix(storage): validation pass — gcs serv..." | Re-trigger Greptile}

Adds GCS as a third object-storage backend with full parity with S3 and Azure Blob: uploads, streaming downloads, deletes, head, V4 signed URLs (single + batch), and browser/server multipart uploads via the GCS XML API. Selection precedence is Azure Blob > S3 > GCS > local disk. - new provider client at lib/uploads/providers/gcs (cached singleton, ADC/Workload Identity or inline GCS_CREDENTIALS_JSON auth) - per-context GCS_*_BUCKET_NAME config wired through getStorageConfig - shared getServeStoragePrefix() replaces hardcoded blob/s3 serve paths - docs (object-storage, environment-variables), .env.example, helm values.yaml + values-gcp.yaml storage section

… quote normalization - getGcsConfig falls back to the general bucket for every context (GCS bucket names are globally unique, so the S3-style sim-execution-files literal default would point at an unowned bucket; empty per-context buckets previously made uploads and downloads disagree) - completeGcsMultipartUpload restores quotes on ETags stripped by the shared browser upload client before building the completion XML - docs/.env.example/helm updated for the fallback behavior

waleedlatif1 · 2026-07-17T06:55:03Z

@greptile

waleedlatif1 · 2026-07-17T06:55:04Z

@cursor review

…ection through getStorageConfig - getChatStorageConfig delegates to getStorageConfig('chat') (identical for S3/Azure, picks up the GCS general-bucket fallback instead of reading the raw chat config and rejecting valid chat files) - parse route resolves the execution bucket via getStorageConfig('execution') for all providers, so GCS execution files in the fallback bucket are still recognized as our own objects

waleedlatif1 · 2026-07-17T07:03:22Z

@greptile

waleedlatif1 · 2026-07-17T07:03:23Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d6feccb. Configure here.}

…s + CORS doc fix - extractStorageKey, extractFilename, and extractEmbeddedFileRef now strip the gcs/ serve prefix like s3/ and blob/, so direct-uploaded files on GCS parse, delete, download, and embed correctly (previously only the serve route knew the prefix) - file-download storageProvider union includes 'gcs' - completeGcsMultipartUpload defensively rejects a 200 response carrying an XML error document - docs: CORS example lists concrete x-goog-meta-* header names (GCS matches responseHeader entries exactly; wildcards are only supported for origin)

waleedlatif1 · 2026-07-17T07:08:42Z

@greptile

waleedlatif1 · 2026-07-17T07:08:44Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit b2dec5d. Configure here.}

waleedlatif1 requested a review from a team as a code owner July 17, 2026 06:46

greptile-apps Bot reviewed Jul 17, 2026

View reviewed changes

Comment thread apps/sim/app/api/files/multipart/route.ts

Comment thread apps/sim/lib/uploads/config.ts

Comment thread apps/sim/app/api/files/multipart/route.ts

vercel Bot deployed to Preview July 17, 2026 06:50 View deployment

waleedlatif1 added 2 commits July 16, 2026 23:54

waleedlatif1 force-pushed the gcs-storage branch from b9e9c2a to 524722a Compare July 17, 2026 06:54

greptile-apps Bot reviewed Jul 17, 2026

View reviewed changes

Comment thread apps/sim/app/api/files/authorization.ts Outdated

cursor Bot reviewed Jul 17, 2026

View reviewed changes

Comment thread apps/sim/app/api/files/parse/route.ts

vercel Bot deployed to Preview July 17, 2026 06:59 View deployment

vercel Bot temporarily deployed to Preview July 17, 2026 07:03 Inactive

cursor Bot reviewed Jul 17, 2026

View reviewed changes

waleedlatif1 force-pushed the gcs-storage branch from 13a02bd to b2dec5d Compare July 17, 2026 07:08

cursor Bot reviewed Jul 17, 2026

View reviewed changes

vercel Bot deployed to Preview July 17, 2026 07:14 View deployment

waleedlatif1 merged commit f6bb8e6 into staging Jul 17, 2026
20 checks passed

waleedlatif1 deleted the gcs-storage branch July 17, 2026 07:15

waleedlatif1 mentioned this pull request Jul 17, 2026

v0.7.38: mothership improvements, clickup, rocketlane integrations, gitlab tools, deployment state machine #5712

Merged

Uh oh!

Conversation

waleedlatif1 commented Jul 17, 2026

Summary

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

greptile-apps Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Jul 17, 2026

Uh oh!

waleedlatif1 commented Jul 17, 2026

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Jul 17, 2026

Uh oh!

waleedlatif1 commented Jul 17, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

waleedlatif1 commented Jul 17, 2026

Uh oh!

waleedlatif1 commented Jul 17, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Jul 17, 2026 •

edited

Loading

cursor Bot commented Jul 17, 2026 •

edited

Loading

greptile-apps Bot commented Jul 17, 2026 •

edited

Loading