feat(storage): support S3-compatible endpoints (R2, MinIO, B2) for file storage#4865
Conversation
…le storage Add S3_ENDPOINT and S3_FORCE_PATH_STYLE env vars, wired into the shared upload S3 client so Cloudflare R2, MinIO, Backblaze B2, and other S3-compatible stores work for self-hosted file storage. The endpoint is trusted operator config (no SSRF/HTTPS gate). Makes the multipart Location fallback endpoint-aware, extends the S3 client unit tests, and documents the new vars in Helm values, .env.example, and the English self-hosting docs (incl. browser-reachability + CORS guidance).
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
PR SummaryMedium Risk Overview The shared upload Documentation adds an English Reviewed by Cursor Bugbot for commit f3c7b14. Configure here. |
Greptile SummaryThis PR adds
Confidence Score: 5/5Safe to merge — all S3 operations route through the updated client, and the addressing-mode branching is correct for each provider. The implementation correctly threads endpoint and forcePathStyle from env through config to the SDK client, with proper empty-string handling in Helm. The buildObjectFallbackUrl branching is correct, the envBoolean mock exactly mirrors the production helper, and existing AWS S3 behavior is unchanged. The only finding is a test that pairs an R2 URL with forcePathStyle: true — an invalid provider combination — but this does not affect production code paths. apps/sim/lib/uploads/providers/s3/client.test.ts — the new endpoint test uses an R2 URL with forcePathStyle: true, which is an invalid combination and reduces the test's value as a correctness check. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[S3_ENDPOINT set?] -->|No| B[AWS S3
virtual-hosted, region-derived host]
A -->|Yes| C[S3_FORCE_PATH_STYLE?]
C -->|true| D[MinIO / Ceph RGW
path-style: endpoint/bucket/key]
C -->|false| E[R2 / other
virtual-hosted: bucket.endpoint/key]
subgraph getS3Client
F[new S3Client
region + endpoint + forcePathStyle]
end
subgraph buildObjectFallbackUrl
A
B
C
D
E
end
B --> F
D --> F
E --> F
Reviews (2): Last reviewed commit: "fix(storage): address review feedback an..." | Re-trigger Greptile |
- Add envBoolean to the shared env test mock (createEnvMock) so config.ts's forcePathStyle coercion resolves — fixes failing knowledge/utils.test.ts - Declare S3_FORCE_PATH_STYLE as z.string() (every other env var's pattern); it's coerced via envBoolean at the consumption site, avoiding a boolean type that never matches the string process.env value - Log path-style from S3_CONFIG.forcePathStyle (envBoolean) instead of a separate isTruthy call, so the startup log can't disagree with the client - Make buildObjectFallbackUrl honor forcePathStyle: virtual-hosted-style URL (bucket as subdomain) for R2, path-style only when forcePathStyle is set
|
@greptile |
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit f3c7b14. Configure here.
…eph, B2, RustFS) and backends
Summary
S3_ENDPOINTandS3_FORCE_PATH_STYLEenv vars so self-hosters can use Cloudflare R2, MinIO, Backblaze B2, or any S3-compatible store for file storage — previously only real AWS S3 workedS3Client(getS3Client); every storage op (upload, download, presign, multipart) routes through it, so presigned URLs honor the custom endpoint automaticallyforcePathStyledefaults tofalse(AWS S3, R2) and is settruefor MinIO/CephLocationfallback endpoint-aware (the only hardcoded AWS hostname in the upload path)values.yaml+values-aws.yaml,.env.example, and the English self-hosting docs (newobject-storage.mdxpage), including the browser-reachability + CORS requirement for direct presigned uploadsType of Change
Testing
providers/s3/client.test.tsto assert the client is constructed withendpoint/forcePathStylewhen set, and absent/falsefor plain AWSbun run scripts/check-api-validation-contracts.tspasses (no route changes); biome cleanNotes
Checklist