Skip to content

feat(artwork): S3-compatible backend (phase 1.h.2) - #29

Merged
InstaZDLL merged 1 commit into
mainfrom
feat/1-h-2-artwork-s3
Jun 6, 2026
Merged

feat(artwork): S3-compatible backend (phase 1.h.2)#29
InstaZDLL merged 1 commit into
mainfrom
feat/1-h-2-artwork-s3

Conversation

@InstaZDLL

@InstaZDLL InstaZDLL commented Jun 6, 2026

Copy link
Copy Markdown
Owner

Summary

Sprint 1 / Phase 1.h.2 — second ArtworkBackend variant so a deploy can store artwork on AWS S3, MinIO, Cloudflare R2, or Backblaze B2 instead of (or eventually migrating from) the LocalFileSystem path that landed in #28. Same ObjectStore trait, same put / get / head, zero caller change in api/artwork.rs.

Env surface

ArtworkBackend::from_env resolution:

Env state Resolution
Neither set None — feature off, endpoints 503 (unchanged from 1.h.1)
WAVEFLOW_ARTWORK_LOCAL_DIR set Local { dir } — LocalFileSystem (unchanged from 1.h.1)
WAVEFLOW_ARTWORK_S3_BUCKET set S3 { … } — needs key + secret; defaults region us-east-1
Both set Boot fails — silently picking one would let an operator believe they're storing to S3 when the local path won

S3 vars: WAVEFLOW_ARTWORK_S3_BUCKET, _ACCESS_KEY_ID, _SECRET_ACCESS_KEY, optional _REGION, _ENDPOINT, _PREFIX.

Highlights

  • Single trait surface in the handler — the upload / read code in api/artwork.rs is untouched; only the boot path branches.
  • Endpoint override (AmazonS3Builder::with_endpoint + with_allow_http(true)) reaches MinIO over plain HTTP for local-dev / on-prem deploys; AWS itself ignores the flag.
  • Optional S3_PREFIX wraps the inner store in object_store::prefix::PrefixStore so the shared artwork/<hash> key shape stays a single source of truth and multiple workloads can share one bucket cleanly.
  • Debug redacts the secret access key — same hygiene the parent Config already applies to database_url and stream_secret.
  • ArtworkConfig type alias keeps the 1.h.1 wire-up compiling without churning the test harness or main.rs.

Test plan

  • cargo fmt --all --check ✅ locally
  • cargo clippy --all-targets --all-features -- -D warnings ✅ locally
  • cargo test --lib (28/28 pass; 2 new tests cover from_env 5-branch resolution + static S3 builder smoke)
  • cargo test integration suite (every existing test stays untouched — ArtworkConfig is a type alias to ArtworkBackend, so tests/support.rs keeps its artwork: None)
  • Manual smoke against MinIO (docker run -p 9000:9000 minio/minio server /data) — out of scope for CI (no MinIO container in the GH workflow), tracked as a follow-up checklist item.

Follow-up

  • 1.h.3 — image-crate resize pipeline (3 sizes: thumbnail / preview / full) with per-variant BLAKE3.
  • 1.i.1apalis Postgres-backed job queue, first job = artwork pipeline async.

Part of the post-1.g sprint plan validated 2026-06-05.

Summary by CodeRabbit

Notes de version

  • New Features

    • Ajout du support du stockage d'artworks sur backends S3-compatibles (MinIO, R2, B2, etc.) en alternative au stockage local.
    • Les backends local et S3 sont maintenant configurables via variables d'environnement avec une règle d'exclusivité mutuelle.
  • Documentation

    • Documentation enrichie des endpoints de cache d'artworks (comportement idempotent, en-têtes de cache, ETag).
    • Guide de configuration du stockage précisant les variables d'environnement et les contraintes de backends.

Adds the second `ArtworkBackend` variant so a deploy can point at
AWS S3, MinIO, Cloudflare R2 or Backblaze B2 instead of (or in
addition to a future migration from) the LocalFileSystem path.

`ArtworkBackend::from_env` now resolves to:
- `Local { dir }` if `WAVEFLOW_ARTWORK_LOCAL_DIR` is set,
- `S3 { bucket, endpoint, region, key, secret, prefix }` if
  `WAVEFLOW_ARTWORK_S3_BUCKET` is set (with required key + secret),
- `None` (feature off) if neither is set,
- error if both are set — silently picking one would let an
  operator believe they're storing to S3 when the local path won.

The S3 builder reaches AWS by default; the optional endpoint
override + `with_allow_http(true)` lets the same builder reach
MinIO over plain HTTP. An optional `S3_PREFIX` wraps the inner
store in `object_store::prefix::PrefixStore` so the shared
`artwork/<hash>` key shape stays a single source of truth and
multiple workloads can share one bucket.

`Debug` on the new enum redacts the secret access key — same
hygiene the parent `Config` already applies to `database_url` and
`stream_secret`. The `ArtworkConfig` type alias keeps the 1.h.1
wire-up (`Config::artwork: Option<ArtworkConfig>`) compiling
without churning the test harness or `main.rs`.

Tests: serialised `from_env` round-trip across the four resolution
branches (clean, local-only, S3-missing-creds, full S3, both-set
conflict) plus a static S3 builder smoke test that doesn't need a
live endpoint. 28/28 unit tests pass.

Signed-off-by: InstaZDLL <github.105mh@8shield.net>
@coderabbitai

coderabbitai Bot commented Jun 6, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Cette PR étend le système de stockage d'artwork pour supporter les backends S3-compatibles (AWS/MinIO/R2/B2) en plus du système de fichiers local. Un nouvel enum ArtworkBackend remplace la configuration précédente, la résolution des variables d'environnement applique une exclusivité mutuelle, et l'initialisation est intégrée au bootstrap applicatif.

Changes

S3 Backend Support for Artwork Storage

Layer / File(s) Summary
Backend configuration contract and environment resolution
Cargo.toml, src/storage.rs
ArtworkBackend enum (Local/S3) remplace ArtworkConfig. Logique from_env() lit env vars, impose l'exclusivité local vs S3, valide les identifiants S3, fournit un défaut pour la région. from_backend() route vers les implémentations spécifiques.
S3 backend implementation
src/storage.rs
ArtworkStorage::s3() construit un AmazonS3Builder avec bucket/région/credentials, gère l'override d'endpoint optionnel avec allow_http(true) pour MinIO/R2, encapsule optionnellement avec PrefixStore.
Application bootstrap integration
src/main.rs
Remplace l'initialisation local-only par une sélection polymorphe : lit config.artwork, route via from_backend(), met à jour les messages de log pour refléter les deux variables d'activation possibles.
Tests and configuration documentation
src/storage.rs, .env.example, CLAUDE.md
Test from_env_resolves_backend_choice() couvre Local, S3, violation d'exclusivité, validation S3, défauts. Smoke test pour endpoint override. .env.example et CLAUDE.md documentent le schéma S3, les contraintes et l'override d'endpoint.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • InstaZDLL/waveflow-server#28: Implémentation initiale du cache d'artwork avec ArtworkConfig/ArtworkStorage::local() que ce PR refactorise pour ajouter le support S3.

Poem

🎨 Du disque local aux nuages sans fin,
S3 arrive, MinIO en voisin,
Mutual exclusivity veille,
Un enum sagace se réveille,
L'artifact danse, de backends, sans chagrin ! ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed Le titre décrit précisément le changement principal : l'ajout d'un backend S3 pour le stockage d'artwork et identifie correctement la phase de développement (1.h.2).
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/1-h-2-artwork-s3

Comment @coderabbitai help to get the list of available commands and usage tips.

@InstaZDLL InstaZDLL self-assigned this Jun 6, 2026
@InstaZDLL
InstaZDLL merged commit 592a399 into main Jun 6, 2026
8 checks passed
@InstaZDLL
InstaZDLL deleted the feat/1-h-2-artwork-s3 branch June 6, 2026 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant