feat(storage): native Google Cloud Storage support for self-hosting (#5728)

* feat(storage): native Google Cloud Storage support for self-hosting

Adds GCS as a third object-storage backend with full parity with S3 and
Azure Blob: uploads, streaming downloads, deletes, head, V4 signed URLs
(single + batch), and browser/server multipart uploads via the GCS XML
API. Selection precedence is Azure Blob > S3 > GCS > local disk.

- new provider client at lib/uploads/providers/gcs (cached singleton,
  ADC/Workload Identity or inline GCS_CREDENTIALS_JSON auth)
- per-context GCS_*_BUCKET_NAME config wired through getStorageConfig
- shared getServeStoragePrefix() replaces hardcoded blob/s3 serve paths
- docs (object-storage, environment-variables), .env.example, helm
  values.yaml + values-gcp.yaml storage section

* fix(storage): review round 1 — GCS per-context bucket fallback + ETag quote normalization

- getGcsConfig falls back to the general bucket for every context (GCS bucket
  names are globally unique, so the S3-style sim-execution-files literal default
  would point at an unowned bucket; empty per-context buckets previously made
  uploads and downloads disagree)
- completeGcsMultipartUpload restores quotes on ETags stripped by the shared
  browser upload client before building the completion XML
- docs/.env.example/helm updated for the fallback behavior

* fix(storage): review round 2 — route chat authz and execution-URL detection through getStorageConfig

- getChatStorageConfig delegates to getStorageConfig('chat') (identical for
  S3/Azure, picks up the GCS general-bucket fallback instead of reading the
  raw chat config and rejecting valid chat files)
- parse route resolves the execution bucket via getStorageConfig('execution')
  for all providers, so GCS execution files in the fallback bucket are still
  recognized as our own objects

* fix(storage): validation pass — gcs serve-prefix parity in key parsers + CORS doc fix

- extractStorageKey, extractFilename, and extractEmbeddedFileRef now strip the
  gcs/ serve prefix like s3/ and blob/, so direct-uploaded files on GCS parse,
  delete, download, and embed correctly (previously only the serve route knew
  the prefix)
- file-download storageProvider union includes 'gcs'
- completeGcsMultipartUpload defensively rejects a 200 response carrying an
  XML error document
- docs: CORS example lists concrete x-goog-meta-* header names (GCS matches
  responseHeader entries exactly; wildcards are only supported for origin)

Author: waleedlatif1

apps/docs/content/docs/en/platform/self-hosting/environment-variables.mdx

@@ -72,7 +72,7 @@ import { Callout } from 'fumadocs-ui/components/callout'
 
 ## File Storage
 
-By default Sim writes uploads to local disk. For production, point it at AWS S3 or Azure Blob. See [Object Storage](/platform/self-hosting/object-storage) for the full setup, bucket layout, and IAM policy.
+By default Sim writes uploads to local disk. For production, point it at AWS S3, Azure Blob, or Google Cloud Storage. See [Object Storage](/platform/self-hosting/object-storage) for the full setup, bucket layout, and IAM policy.
 
 | Variable | Description |
 |----------|-------------|
@@ -82,6 +82,9 @@ By default Sim writes uploads to local disk. For production, point it at AWS S3
 | `S3_BUCKET_NAME` | General workspace files bucket — set with `AWS_REGION` to enable S3 |
 | `AZURE_STORAGE_CONTAINER_NAME` | General files container — set with Azure credentials to enable Blob (takes precedence over S3) |
 | `AZURE_CONNECTION_STRING` | Azure connection string, or use `AZURE_ACCOUNT_NAME` + `AZURE_ACCOUNT_KEY` |
+| `GCS_BUCKET_NAME` | General workspace files bucket — enables GCS when neither Azure Blob nor S3 is configured |
+| `GCS_PROJECT_ID` | GCP project ID. Omit to infer from credentials/ADC |
+| `GCS_CREDENTIALS_JSON` | Inline service-account JSON. Omit to use Application Default Credentials (Workload Identity, `GOOGLE_APPLICATION_CREDENTIALS`) |
 
 ## Email Providers
 

apps/docs/content/docs/en/platform/self-hosting/object-storage.mdx

@@ -1,23 +1,24 @@
 ---
 title: Object Storage
-description: Configure where Sim stores uploaded files — local disk, AWS S3, or Azure Blob
+description: Configure where Sim stores uploaded files — local disk, AWS S3, Azure Blob, or Google Cloud Storage
 ---
 
 import { Tab, Tabs } from 'fumadocs-ui/components/tabs'
 import { Callout } from 'fumadocs-ui/components/callout'
 import { Step, Steps } from 'fumadocs-ui/components/steps'
 import { FAQ } from '@/components/ui/faq'
 
-Sim stores every uploaded file — knowledge base documents, chat attachments, execution outputs, profile pictures, and more — in object storage. Three backends are supported:
+Sim stores every uploaded file — knowledge base documents, chat attachments, execution outputs, profile pictures, and more — in object storage. Four backends are supported:
 
 | Backend | When to use |
 |---------|-------------|
 | **Local disk** | Single-node Docker, local development, evaluation |
 | **[AWS S3](https://aws.amazon.com/s3/)** | Production, especially when running more than one app replica |
 | **[Azure Blob](https://learn.microsoft.com/azure/storage/blobs/)** | Production on Azure |
+| **[Google Cloud Storage](https://cloud.google.com/storage)** | Production on GCP |
 
 <Callout type="warning">
-  Local disk writes to the container's `/uploads` directory. Files are lost when the container is recreated unless that path is on a persistent volume, and they are **not** shared across replicas. For any multi-replica or production deployment, use S3 or Azure Blob.
+  Local disk writes to the container's `/uploads` directory. Files are lost when the container is recreated unless that path is on a persistent volume, and they are **not** shared across replicas. For any multi-replica or production deployment, use S3, Azure Blob, or Google Cloud Storage.
 </Callout>
 
 ## How the backend is selected
@@ -26,9 +27,10 @@ Sim picks the backend automatically from environment variables — there is no e
 
 1. **Azure Blob** — used if `AZURE_STORAGE_CONTAINER_NAME` is set **and** either (`AZURE_ACCOUNT_NAME` + `AZURE_ACCOUNT_KEY`) or `AZURE_CONNECTION_STRING` is set.
 2. **AWS S3** — used if `S3_BUCKET_NAME` **and** `AWS_REGION` are set (and Azure is not configured).
-3. **Local disk** — the fallback when neither is configured.
+3. **Google Cloud Storage** — used if `GCS_BUCKET_NAME` is set (and neither Azure nor S3 is configured).
+4. **Local disk** — the fallback when none is configured.
 
-If both Azure and S3 are configured, **Azure wins**. Set only the variables for the backend you intend to use.
+If more than one backend is configured, the first match in that order wins. Set only the variables for the backend you intend to use.
 
 ## Set up AWS S3
 
@@ -201,6 +203,143 @@ AZURE_STORAGE_WORKSPACE_LOGOS_CONTAINER_NAME=workspace-logos
 
 A full Helm example lives at `helm/sim/examples/values-azure.yaml`.
 
+## Set up Google Cloud Storage
+
+<Steps>
+
+<Step>
+
+### Create the buckets
+
+GCS uses one bucket per purpose, mirroring the S3 layout:
+
+```bash
+export PROJECT_ID=your-project-id
+export LOCATION=us-central1
+
+# Create buckets (names must be globally unique — prefix with your org)
+for name in workspace-files knowledge-base execution-files chat-files \
+            copilot-files profile-pictures og-images workspace-logos; do
+  gcloud storage buckets create "gs://myorg-sim-$name" \
+    --project "$PROJECT_ID" \
+    --location "$LOCATION" \
+    --uniform-bucket-level-access
+done
+```
+
+Keep all buckets **private** (no `allUsers` bindings). Sim serves files through short-lived V4 signed URLs, so the buckets never need public read access.
+
+Because uploads are sent **directly from the browser** via signed `PUT` requests, each bucket needs a CORS policy that allows your Sim origin:
+
+```bash
+cat > /tmp/cors.json <<'EOF'
+[
+  {
+    "origin": ["https://your-sim-domain.com"],
+    "method": ["GET", "PUT"],
+    "responseHeader": [
+      "Content-Type",
+      "ETag",
+      "x-goog-meta-originalname",
+      "x-goog-meta-uploadedat",
+      "x-goog-meta-purpose",
+      "x-goog-meta-userid",
+      "x-goog-meta-workspaceid",
+      "x-goog-meta-workflowid",
+      "x-goog-meta-executionid"
+    ],
+    "maxAgeSeconds": 3600
+  }
+]
+EOF
+
+for name in workspace-files knowledge-base execution-files chat-files \
+            copilot-files profile-pictures og-images workspace-logos; do
+  gcloud storage buckets update "gs://myorg-sim-$name" --cors-file=/tmp/cors.json
+done
+```
+
+<Callout type="info">
+  Header names must be listed individually — GCS CORS matches `responseHeader` entries exactly and does not support wildcards like `x-goog-meta-*`. `ETag` is required because large-file multipart uploads read each part's `ETag` from the browser, and CORS hides the header otherwise.
+</Callout>
+
+</Step>
+
+<Step>
+
+### Grant access
+
+Create a service account (or reuse the one your workload runs as) and grant it object access on the buckets:
+
+```bash
+gcloud iam service-accounts create sim-storage --project "$PROJECT_ID"
+
+for name in workspace-files knowledge-base execution-files chat-files \
+            copilot-files profile-pictures og-images workspace-logos; do
+  gcloud storage buckets add-iam-policy-binding "gs://myorg-sim-$name" \
+    --member "serviceAccount:sim-storage@$PROJECT_ID.iam.gserviceaccount.com" \
+    --role roles/storage.objectAdmin
+done
+```
+
+You then have two ways to supply credentials:
+
+- **Application Default Credentials (recommended on GCP)** — run Sim with the service account via GKE Workload Identity (or attach it to the GCE instance) and leave `GCS_CREDENTIALS_JSON` unset. Because there is no private key in this mode, generating signed URLs uses the IAM `signBlob` API — grant the service account `roles/iam.serviceAccountTokenCreator` **on itself**:
+
+  ```bash
+  gcloud iam service-accounts add-iam-policy-binding \
+    "sim-storage@$PROJECT_ID.iam.gserviceaccount.com" \
+    --member "serviceAccount:sim-storage@$PROJECT_ID.iam.gserviceaccount.com" \
+    --role roles/iam.serviceAccountTokenCreator
+  ```
+
+- **Inline key (for Docker Compose or non-GCP hosts)** — create a JSON key for the service account and set `GCS_CREDENTIALS_JSON` to its contents. With a private key present, signed URLs are generated locally and no extra IAM role is needed.
+
+</Step>
+
+<Step>
+
+### Configure environment variables
+
+```bash
+# Credentials — omit both when using Workload Identity / ADC
+GCS_PROJECT_ID=your-project-id            # optional; inferred from credentials when unset
+GCS_CREDENTIALS_JSON='{"type":"service_account","client_email":"...","private_key":"..."}'
+
+# Buckets (per purpose)
+GCS_BUCKET_NAME=myorg-sim-workspace-files
+GCS_KB_BUCKET_NAME=myorg-sim-knowledge-base
+GCS_EXECUTION_FILES_BUCKET_NAME=myorg-sim-execution-files
+GCS_CHAT_BUCKET_NAME=myorg-sim-chat-files
+GCS_COPILOT_BUCKET_NAME=myorg-sim-copilot-files
+GCS_PROFILE_PICTURES_BUCKET_NAME=myorg-sim-profile-pictures
+GCS_OG_IMAGES_BUCKET_NAME=myorg-sim-og-images
+GCS_WORKSPACE_LOGOS_BUCKET_NAME=myorg-sim-workspace-logos
+```
+
+Only `GCS_BUCKET_NAME` is strictly required to switch Sim into GCS mode. Every purpose-specific bucket falls back to the general bucket when unset — add the others so each file type lands in its own bucket.
+
+</Step>
+
+</Steps>
+
+### GCS bucket reference
+
+| Variable | Stores | Required |
+|----------|--------|----------|
+| `GCS_BUCKET_NAME` | General workspace files | **Yes** (enables GCS) |
+| `GCS_PROJECT_ID` | GCP project ID | No (inferred from credentials/ADC) |
+| `GCS_CREDENTIALS_JSON` | Inline service-account JSON | No (uses Application Default Credentials if unset) |
+| `GCS_KB_BUCKET_NAME` | Knowledge base documents | Recommended (falls back to `GCS_BUCKET_NAME`) |
+| `GCS_EXECUTION_FILES_BUCKET_NAME` | Workflow execution files | Recommended (falls back to `GCS_BUCKET_NAME`) |
+| `GCS_CHAT_BUCKET_NAME` | Deployed chat assets | Recommended (falls back to `GCS_BUCKET_NAME`) |
+| `GCS_COPILOT_BUCKET_NAME` | Copilot attachments | Recommended (falls back to `GCS_BUCKET_NAME`) |
+| `GCS_PROFILE_PICTURES_BUCKET_NAME` | User avatars | Recommended (falls back to `GCS_BUCKET_NAME`) |
+| `GCS_OG_IMAGES_BUCKET_NAME` | OpenGraph preview images (falls back to `GCS_BUCKET_NAME`) | Optional |
+| `GCS_WORKSPACE_LOGOS_BUCKET_NAME` | Workspace logos (falls back to `GCS_BUCKET_NAME`) | Optional |
+
+A full Helm example (Workload Identity, GKE) lives at `helm/sim/examples/values-gcp.yaml`.
+
 ## Set up an S3-compatible provider (R2, MinIO, B2)
 
 Sim works with any S3-compatible store by pointing the S3 client at a custom endpoint. Configure it exactly like AWS S3 (buckets, access key, secret), then add `S3_ENDPOINT` — and `S3_FORCE_PATH_STYLE` where the provider requires path-style addressing. Verified with [Cloudflare R2](https://developers.cloudflare.com/r2/), [MinIO](https://min.io/), [Backblaze B2](https://www.backblaze.com/cloud-storage), and [RustFS](https://rustfs.com/).
@@ -280,10 +419,11 @@ After restarting with the new configuration:
 If uploads fail, check the app logs for credential or permission errors (see [Troubleshooting](/platform/self-hosting/troubleshooting)).
 
 <FAQ items={[
-  { question: "What happens if I do not configure any storage variables?", answer: "Sim falls back to local disk, writing files to the /uploads directory inside the app container. This is fine for evaluation but not durable across container recreation and not shared across replicas — use S3 or Azure Blob for production." },
+  { question: "What happens if I do not configure any storage variables?", answer: "Sim falls back to local disk, writing files to the /uploads directory inside the app container. This is fine for evaluation but not durable across container recreation and not shared across replicas — use S3, Azure Blob, or Google Cloud Storage for production." },
   { question: "Do I have to create all eight S3 buckets?", answer: "No. Only AWS_REGION and S3_BUCKET_NAME are required to enable S3 mode. The purpose-specific buckets are recommended so each file type is isolated; og-images and workspace-logos fall back to the general bucket if their variables are unset." },
   { question: "How do I avoid storing AWS keys in plaintext?", answer: "On EC2/ECS/EKS, attach the IAM policy to the instance role, task role, or IRSA service-account role and leave AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY unset. Sim resolves credentials through the default AWS SDK provider chain automatically." },
-  { question: "Can I use both S3 and Azure Blob at the same time?", answer: "No. Sim selects a single backend. If both are configured, Azure Blob takes precedence. Set only the variables for the backend you want." },
+  { question: "Can I configure more than one cloud backend at the same time?", answer: "No. Sim selects a single backend, in the order Azure Blob, then S3, then Google Cloud Storage. Set only the variables for the backend you want." },
+  { question: "How do I use GCS without storing a service-account key?", answer: "Run Sim with GKE Workload Identity (or an attached GCE service account) and leave GCS_CREDENTIALS_JSON unset — credentials resolve through Application Default Credentials. Grant the service account roles/iam.serviceAccountTokenCreator on itself so signed URLs can be generated via the IAM signBlob API." },
   { question: "Are the buckets exposed publicly?", answer: "No, and they should not be. Keep them private with public access blocked. Sim serves files to users through short-lived presigned URLs, so the buckets never need public read permissions." },
   { question: "Can I use MinIO or Cloudflare R2?", answer: "Yes. Configure it like AWS S3, then set S3_ENDPOINT to your provider's endpoint. For R2, set AWS_REGION=auto and leave S3_FORCE_PATH_STYLE unset. For MinIO/Ceph, set S3_FORCE_PATH_STYLE=true. See the S3-compatible provider section above." },
 ]} />

apps/sim/.env.example

@@ -81,7 +81,7 @@ API_ENCRYPTION_KEY=your_api_encryption_key # Use `openssl rand -hex 32` to gener
 # CONTEXT_DEV_API_KEY_1=                         # Context.dev API key #1
 # CONTEXT_DEV_API_KEY_2=                         # Context.dev API key #2
 
-# File Storage (Optional - defaults to local disk; use S3 or Azure Blob for production)
+# File Storage (Optional - defaults to local disk; use S3, Azure Blob, or Google Cloud Storage for production)
 # AWS_REGION=us-east-1                            # Required with S3_BUCKET_NAME to enable S3. Use "auto" for Cloudflare R2
 # AWS_ACCESS_KEY_ID=                              # Omit to use the instance/IRSA credential chain
 # AWS_SECRET_ACCESS_KEY=                          # Omit to use the instance/IRSA credential chain
@@ -99,7 +99,7 @@ API_ENCRYPTION_KEY=your_api_encryption_key # Use `openssl rand -hex 32` to gener
 # Instagram OAuth (Optional - Instagram App ID/Secret from Meta App Dashboard > Instagram > API setup with Instagram login)
 # INSTAGRAM_CLIENT_ID=
 # INSTAGRAM_CLIENT_SECRET=
-# Instagram publish file uploads require S3 or Azure Blob above (Meta must fetch a public HTTPS URL).
+# Instagram publish file uploads require cloud storage above — S3, Azure Blob, or GCS (Meta must fetch a public HTTPS URL).
 # Gmail attachments do not need this.
 
 # TikTok OAuth (Optional - Client Key/Secret from the TikTok for Developers app)
@@ -119,6 +119,18 @@ API_ENCRYPTION_KEY=your_api_encryption_key # Use `openssl rand -hex 32` to gener
 # AZURE_STORAGE_OG_IMAGES_CONTAINER_NAME=         # OpenGraph preview images (falls back to AZURE_STORAGE_CONTAINER_NAME)
 # AZURE_STORAGE_WORKSPACE_LOGOS_CONTAINER_NAME=   # Workspace logos (falls back to AZURE_STORAGE_CONTAINER_NAME)
 
+# Google Cloud Storage (used when neither Azure Blob nor S3 is configured)
+# GCS_PROJECT_ID=                                 # GCP project ID (optional — inferred from credentials/ADC when unset)
+# GCS_CREDENTIALS_JSON=                           # Inline service-account JSON. Omit to use Application Default Credentials (Workload Identity, GOOGLE_APPLICATION_CREDENTIALS)
+# GCS_BUCKET_NAME=                                # General workspace files bucket (enables GCS; all other buckets fall back to it)
+# GCS_KB_BUCKET_NAME=                             # Knowledge base documents
+# GCS_EXECUTION_FILES_BUCKET_NAME=                # Workflow execution files
+# GCS_CHAT_BUCKET_NAME=                           # Deployed chat assets
+# GCS_COPILOT_BUCKET_NAME=                        # Copilot attachments
+# GCS_PROFILE_PICTURES_BUCKET_NAME=               # User profile pictures
+# GCS_OG_IMAGES_BUCKET_NAME=                      # OpenGraph preview images
+# GCS_WORKSPACE_LOGOS_BUCKET_NAME=                # Workspace logos
+
 # Admin API (Optional - for self-hosted GitOps)
 # ADMIN_API_KEY=  # Use `openssl rand -hex 32` to generate. Enables admin API for workflow export/import.
                   # Usage: curl -H "x-admin-key: your_key" https://your-instance/api/v1/admin/workspaces

apps/sim/app/api/files/authorization.test.ts

@@ -27,8 +27,7 @@ vi.mock('@/lib/uploads', () => ({
 }))
 
 vi.mock('@/lib/uploads/config', () => ({
-  BLOB_CHAT_CONFIG: {},
-  S3_CHAT_CONFIG: {},
+  getStorageConfig: vi.fn(() => ({})),
 }))
 
 vi.mock('@/lib/uploads/server/metadata', () => ({

apps/sim/app/api/files/authorization.ts

@@ -6,7 +6,6 @@ import { and, eq, isNull } from 'drizzle-orm'
 import { NextResponse } from 'next/server'
 import { getFileMetadata } from '@/lib/uploads'
 import type { StorageContext } from '@/lib/uploads/config'
-import { BLOB_CHAT_CONFIG, S3_CHAT_CONFIG } from '@/lib/uploads/config'
 import type { StorageConfig } from '@/lib/uploads/core/storage-client'
 import { getFileMetadataByKey } from '@/lib/uploads/server/metadata'
 import { inferContextFromKey } from '@/lib/uploads/utils/file-utils'
@@ -781,23 +780,6 @@ export async function assertToolFileAccess(
  * Get chat storage configuration based on current storage provider
  */
 async function getChatStorageConfig(): Promise<StorageConfig> {
-  const { USE_S3_STORAGE, USE_BLOB_STORAGE } = await import('@/lib/uploads/config')
-
-  if (USE_BLOB_STORAGE) {
-    return {
-      containerName: BLOB_CHAT_CONFIG.containerName,
-      accountName: BLOB_CHAT_CONFIG.accountName,
-      accountKey: BLOB_CHAT_CONFIG.accountKey,
-      connectionString: BLOB_CHAT_CONFIG.connectionString,
-    }
-  }
-
-  if (USE_S3_STORAGE) {
-    return {
-      bucket: S3_CHAT_CONFIG.bucket,
-      region: S3_CHAT_CONFIG.region,
-    }
-  }
-
-  return {}
+  const { getStorageConfig } = await import('@/lib/uploads/config')
+  return getStorageConfig('chat')
 }

apps/sim/app/api/files/export/[id]/route.ts

@@ -12,7 +12,7 @@ import { extractEmbeddedImageIds } from '@/lib/copilot/tools/server/files/embedd
 import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
 import { captureServerEvent } from '@/lib/posthog/server'
 import type { StorageContext } from '@/lib/uploads/config'
-import { USE_BLOB_STORAGE } from '@/lib/uploads/config'
+import { getServeStoragePrefix } from '@/lib/uploads/config'
 import { downloadFile } from '@/lib/uploads/core/storage-service'
 import { getFileMetadataById } from '@/lib/uploads/server/metadata'
 import { verifyFileAccess } from '@/app/api/files/authorization'
@@ -106,7 +106,7 @@ export const GET = withRouteHandler(
     }
 
     if (!isMarkdown(record.originalName, record.contentType)) {
-      const storagePrefix = USE_BLOB_STORAGE ? 'blob' : 's3'
+      const storagePrefix = getServeStoragePrefix()
       const servePath = `/api/files/serve/${storagePrefix}/${encodeURIComponent(record.key)}`
       auditExport('file', 0)
       return NextResponse.redirect(new URL(servePath, request.url), { status: 302 })