fix: plaintext password issue via sealed secrets#495
Merged
Conversation
- Add SOPS + Age encryption for Helm chart secrets (datamate, label-studio, milvus) - Remove plaintext passwords from values.yaml files - Replace hardcoded secrets in docker-compose.yml with env vars - Add .env.example template for Docker deployment path - Add scripts/secrets.sh helper for encrypt/decrypt/helm-install - Add docs/SECRETS_SETUP.md setup guide - Update .gitignore: exclude .sops-keys/, allow .env.example and .env.enc Deployment: - K8s/Helm: helm secrets install -f secrets.yaml - Docker: cp .env.example .env && edit && docker compose up
- K8s install: use helm secrets upgrade with -f secrets.yaml for datamate, label-studio, milvus - Docker install: add pre-check for .env file, exit with helpful message if missing - Set SOPS_AGE_KEY_FILE env var for helm secrets decryption
- JwtUtils.java: remove hardcoded default "datamate-secret-key-for-jwt-token-generation" Change property from jwt.secret to datamate.jwt.secret (aligned with JwtConfig) - application.yml: add datamate.jwt.secret mapping from JWT_SECRET env var - application.yml: remove hardcoded defaults for DB_PASSWORD and REDIS_PASSWORD - docker-compose.yml: add JWT_SECRET env var to datamate-backend service - Helm values.yaml: add JWT_SECRET secretKeyRef to backend env - Helm secrets.yaml: add JWT_SECRET to public.secrets.data
- Reject plaintext private keys on startup (exit with error) - Require CERT_PASS when key is encrypted - Set chmod 600 on decrypted key for restrictive access
… from Docker builds - Create .dockerignore to prevent runtime/datamate-python/.env from being copied into Docker images (it contained localhost:15432 telepresence debug settings) - config.py: remove hardcoded defaults for pgsql_password, mysql_password, label_studio_password, label_studio_user_token - docker-compose.yml: add explicit PGSQL_HOST/PGSQL_PORT for backend-python - Helm values.yaml: add explicit PGSQL_HOST/PGSQL_PORT for backend-python
… dependency - scripts/secrets.sh: add check_tools(), ensure_key() with auto-generation New helm-upgrade command: decrypts secrets.yaml and runs helm upgrade --install - Makefile: K8s install targets now call "bash scripts/secrets.sh helm-upgrade" instead of direct "helm secrets upgrade". No helm-secrets plugin needed. - Users only need sops + age (brew install), key auto-generated on first run. Docker users unaffected - still use .env file.
- docker-compose.yml: add LABEL_STUDIO_USER_TOKEN, LABEL_STUDIO_PASSWORD env vars - Helm values.yaml: add secretKeyRef for both to backend-python env - secrets.yaml: add encrypted LABEL_STUDIO_USER_TOKEN, LABEL_STUDIO_PASSWORD Fixes 500 error "Label Studio API token is required" when creating annotation tasks after config.py defaults were removed.
The loginAnnotationUsingGet flow requires label_studio_username to auto-login to Label Studio. Without it, clicking "edit" on an annotation task redirects to Label Studio login page instead of auto-authenticating.
The loginAnnotationUsingGet API requires label_studio_username to auto-login to Label Studio. Docker got this in 638931b, but the Helm values.yaml was missing it, causing K8s deployments to show Label Studio login prompt.
Replace SOPS + Age encryption with Bitnami Sealed Secrets. No key distribution needed - secrets are encrypted with cluster public key, decrypted automatically by the in-cluster controller. Changes: - Add 3 SealedSecret YAMLs (datamate, label-studio, milvus) under deployment/kubernetes/sealed-secrets/ (safe to commit to Git) - Update Makefile: apply SealedSecrets before helm install - Helm charts: support existingSecret for label-studio, secrets.create flag to skip Helm-managed Secret creation - Remove SOPS artifacts: .sops.yaml, scripts/secrets.sh, secrets.yaml files User workflow: make install INSTALLER=k8s No tools needed on user machine. Controller decrypts automatically.
When existingSecret is configured, the pgbouncer sidecar was still reading POSTGRE_PASSWORD from .Values.env (empty after SOPS migration), causing "password authentication failed" when connecting to PostgreSQL.
Helm must not create the datamate-conf Secret since it is managed by the SealedSecret controller. Without this, fresh installs fail with "conflict with controller" error.
Gateway was still reading JWT_SECRET from values.yaml (empty string), causing "JWT secret is required" startup error. Changed to secretKeyRef to match the SealedSecret-managed datamate-conf Secret.
…e filter
When DATAMATE_JWT_ENABLE=true, the Python backend's _apply_data_scope
filter adds WHERE created_by IN ('user', 'system') to all BaseEntity
queries. Seed data had empty created_by, causing templates, annotation
templates, and operator categories to return 0 results.
- t_clean_template: add created_by/updated_by='system' to INSERT
- t_dm_annotation_templates: add COALESCE fallback in ON CONFLICT
- t_operator_category: add UPDATE to fix empty created_by
All three tables also get a safety-net UPDATE at the end.
The gateway's application.yml had password: ${DB_PASSWORD:password} which
falls back to 'password' when DB_PASSWORD env var is not set. Removed the
default so it fails fast if the variable is missing, consistent with the
main-application config.
- Add Secret Management section to README-zh.md and README.md - Document Sealed Secrets Controller installation (Helm) - Add air-gapped/offline environment instructions (image download) - Add kubeseal CLI usage for updating secrets - Add make download-sealed-secrets target for offline image download
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.