Skip to content

Proposal: expand media-storage cloud backends — Google Cloud Storage first #333

Description

@regiscamimura

Proposal

Add more cloud backends to the existing MediaStorage system, starting with Google Cloud
Storage
. MediaStorage already ships Local and S3 backends; this adds GCSMediaStorage
as the next one — and I'm happy to follow with others (e.g. Azure Blob) after.

from piccolo_api.media.gcs import GCSMediaStorage

storage = GCSMediaStorage(
    column=Movie.poster,
    bucket_name="my-bucket",
    folder_name="movie_posters",
)

Design

Mirrors S3MediaStorage method-for-method (async + _sync worker via ThreadPoolExecutor,
reusing the base's generate_file_key), using google-cloud-storage. Optional dependency as
a gcs extra (pip install 'piccolo_api[gcs]'), like s3.

No new framework — MediaStorage's six abstract methods are already the provider interface,
so each new backend is just one module + extra + docs.

One GCS doc note: v4 signing needs a private key. Locally that's a service-account JSON;
on GCP compute (Cloud Run) the ADC has no private key, so signing goes through the IAM
signBlob API (grant the runtime SA roles/iam.serviceAccountTokenCreator).

Questions before I PR

  1. Happy for GCS to live in piccolo_api, extending the existing backends?
  2. google-cloud-storage + a gcs extra — agreeable?
  3. Interested in further providers after GCS? If so, I could factor a small shared
    CloudMediaStorage mixin once there's a second one to share it with.

The GCS backend is implemented and tested (faked storage.Client, no live GCS) — happy to
open the PR once you confirm the direction. Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions