feat(artwork): S3-compatible backend (phase 1.h.2) - #29
Merged
Conversation
Adds the second `ArtworkBackend` variant so a deploy can point at
AWS S3, MinIO, Cloudflare R2 or Backblaze B2 instead of (or in
addition to a future migration from) the LocalFileSystem path.
`ArtworkBackend::from_env` now resolves to:
- `Local { dir }` if `WAVEFLOW_ARTWORK_LOCAL_DIR` is set,
- `S3 { bucket, endpoint, region, key, secret, prefix }` if
`WAVEFLOW_ARTWORK_S3_BUCKET` is set (with required key + secret),
- `None` (feature off) if neither is set,
- error if both are set — silently picking one would let an
operator believe they're storing to S3 when the local path won.
The S3 builder reaches AWS by default; the optional endpoint
override + `with_allow_http(true)` lets the same builder reach
MinIO over plain HTTP. An optional `S3_PREFIX` wraps the inner
store in `object_store::prefix::PrefixStore` so the shared
`artwork/<hash>` key shape stays a single source of truth and
multiple workloads can share one bucket.
`Debug` on the new enum redacts the secret access key — same
hygiene the parent `Config` already applies to `database_url` and
`stream_secret`. The `ArtworkConfig` type alias keeps the 1.h.1
wire-up (`Config::artwork: Option<ArtworkConfig>`) compiling
without churning the test harness or `main.rs`.
Tests: serialised `from_env` round-trip across the four resolution
branches (clean, local-only, S3-missing-creds, full S3, both-set
conflict) plus a static S3 builder smoke test that doesn't need a
live endpoint. 28/28 unit tests pass.
Signed-off-by: InstaZDLL <github.105mh@8shield.net>
📝 WalkthroughWalkthroughCette PR étend le système de stockage d'artwork pour supporter les backends S3-compatibles (AWS/MinIO/R2/B2) en plus du système de fichiers local. Un nouvel enum ChangesS3 Backend Support for Artwork Storage
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Sprint 1 / Phase 1.h.2 — second
ArtworkBackendvariant so a deploy can store artwork on AWS S3, MinIO, Cloudflare R2, or Backblaze B2 instead of (or eventually migrating from) the LocalFileSystem path that landed in #28. SameObjectStoretrait, sameput/get/head, zero caller change inapi/artwork.rs.Env surface
ArtworkBackend::from_envresolution:None— feature off, endpoints 503 (unchanged from 1.h.1)WAVEFLOW_ARTWORK_LOCAL_DIRsetLocal { dir }— LocalFileSystem (unchanged from 1.h.1)WAVEFLOW_ARTWORK_S3_BUCKETsetS3 { … }— needs key + secret; defaults regionus-east-1S3 vars:
WAVEFLOW_ARTWORK_S3_BUCKET,_ACCESS_KEY_ID,_SECRET_ACCESS_KEY, optional_REGION,_ENDPOINT,_PREFIX.Highlights
api/artwork.rsis untouched; only the boot path branches.AmazonS3Builder::with_endpoint+with_allow_http(true)) reaches MinIO over plain HTTP for local-dev / on-prem deploys; AWS itself ignores the flag.S3_PREFIXwraps the inner store inobject_store::prefix::PrefixStoreso the sharedartwork/<hash>key shape stays a single source of truth and multiple workloads can share one bucket cleanly.Debugredacts the secret access key — same hygiene the parentConfigalready applies todatabase_urlandstream_secret.ArtworkConfigtype alias keeps the 1.h.1 wire-up compiling without churning the test harness ormain.rs.Test plan
cargo fmt --all --check✅ locallycargo clippy --all-targets --all-features -- -D warnings✅ locallycargo test --lib(28/28 pass; 2 new tests coverfrom_env5-branch resolution + static S3 builder smoke)cargo testintegration suite (every existing test stays untouched —ArtworkConfigis a type alias toArtworkBackend, sotests/support.rskeeps itsartwork: None)docker run -p 9000:9000 minio/minio server /data) — out of scope for CI (no MinIO container in the GH workflow), tracked as a follow-up checklist item.Follow-up
apalisPostgres-backed job queue, first job = artwork pipeline async.Part of the post-1.g sprint plan validated 2026-06-05.
Summary by CodeRabbit
Notes de version
New Features
Documentation