You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Integrate DataOps with the shared Cognito browser login
Status: in progress
Tags: enhancement, portal, frontend, backend, infra, testing, P1
Depends on: None
Blocks: None
Scope
Replace the production DataOps browser gate at ops.dtcdev.click with the shared serverless authentication service at https://auth.dtcdev.click. The DataOps Lambda remains the relying party: it starts Cognito authorization-code + S256 PKCE, handles https://ops.dtcdev.click/auth/callback, validates the returned identity, maps it to an existing enabled DataOps user by normalized email, and creates an opaque server-side DataOps session carried in a secure cookie.
The shared stack is already deployed as dtcdev-shared-auth in us-east-1. Its DataOps public client is registered for:
callback: https://ops.dtcdev.click/auth/callback
logout return: https://ops.dtcdev.click/
flow/scopes: authorization code with openid email profile
issuer: the UserPoolId/IssuerUrl output of that stack
The DataOps application and stack remain in this repository and deploy as dataops-v1 in eu-west-1 through .github/workflows/deploy-dataops-v1.yml with GitHub Actions OIDC. Shared Cognito ownership remains in ../aws-infra/sandbox/auth; this issue only consumes its non-secret contract.
This migration replaces browser Basic auth and the password-derived dtc_auth cookie. It must not weaken or silently replace the independent credentials used by machine-to-machine intake, integrations, webhooks, or existing bearer-session clients. Route-specific layered authorization remains route-specific.
Acceptance Criteria
GET /login starts authorization at https://auth.dtcdev.click/oauth2/authorize using the configured DataOps client, exact callback URL, response_type=code, openid email profile, an unpredictable state and nonce, and an S256 PKCE challenge. OAuth transaction state is short-lived, one-time, integrity-protected, and never stored in browser localStorage.
GET /auth/callback rejects missing, expired, replayed, or mismatched state; OAuth provider errors; missing codes; failed token exchanges; and invalid tokens without creating a DataOps session.
The callback exchanges the code at /oauth2/token with the original PKCE verifier and validates the ID token signature against the issuer JWKS plus issuer, DataOps client audience, expiry/not-before, nonce, email, and email_verified. JWKS caching handles key rotation safely. Tokens, authorization codes, PKCE verifiers, and session identifiers are absent from response bodies, URLs after callback completion, and application logs.
A successful callback normalizes the verified email and maps it to exactly one existing DataOps user. Unknown or disabled users receive a non-sensitive 403 response and are not auto-provisioned. The real local user ID is propagated to /api/me, role checks, assignment/notification filtering, and audit fields instead of the current portal-admin identity.
A successful callback creates an opaque, cryptographically random, server-side session with a bounded absolute expiry. The browser receives only an HttpOnly; Secure; SameSite=Lax; Path=/ cookie; expired, deleted, fabricated, and revoked sessions are rejected. OAuth-created expiry enforcement must not silently invalidate or change the contract of existing non-browser bearer sessions.
An authenticated browser session serves the portal, docs/content routes, and same-origin /api/* calls without Basic credentials or a browser-managed bearer token. Unauthenticated page requests redirect to /login; unauthenticated data/API requests return 401 without HTML or protected data.
GET or POST /logout deletes the local server-side session, expires the cookie, and uses the Cognito logout endpoint with the registered DataOps logout URI. A logged-out cookie cannot be reused.
Production no longer serves the DataOps username/password form, accepts Basic auth as a browser-session substitute, or requires BASIC_AUTH_USERNAME / the Basic-auth password secret at Lambda runtime. The legacy /api/auth/login endpoint remains unavailable in production portal mode. Deleting historical Secrets Manager resources is not required.
Existing dedicated machine credentials and entry points keep their current isolation and authorization semantics, including email-document intake, bookkeeping ingestion/layered authorization, webhook/integration credentials, and existing bearer-session clients. A machine credential alone never becomes a browser session, and browser OAuth alone does not bypass a route's additional credential.
infra/template.full.yaml exposes the shared auth base URL, issuer/user-pool contract, DataOps public client ID, callback URL, logout URL, and session-lifetime configuration as non-secret deployment configuration. The Lambda gains only permissions actually required by the implementation; no Cognito client secret or Google secret is copied into this repository or GitHub Actions.
.github/workflows/deploy-dataops-v1.yml passes the auth configuration and continues to validate/build/deploy dataops-v1 exclusively through GitHub Actions OIDC. The workflow does not deploy or mutate dtcdev-shared-auth, and the application is not manually deployed for this issue.
Automated tests cover authorization parameter construction, PKCE/state/nonce lifecycle and replay rejection, token/JWKS validation failures and key refresh, local-user mapping (enabled/disabled/unknown/case normalization), session creation/expiry/logout, page-vs-API unauthenticated behavior, and authenticated /api/me/role behavior. Token and JWKS HTTP calls are stubbed; automated tests do not require a live Cognito account.
Regression tests prove each existing machine credential path and existing bearer-session path retains its prior allow/deny boundary, including invalid/missing credential cases and routes with layered credentials.
Tester runs npm --prefix backend test, npm --prefix backend run typecheck, npm --prefix backend run build, npm --prefix backend run test:e2e, make sam-validate, and make sam-build, reporting exact commands, exit codes, and test counts. Playwright captures and inspects screenshots for the signed-out entry/error state and the authenticated DataOps workspace; no screenshot may show a 404, error page, broken layout, overlap, or leaked credential.
[HUMAN] After deployment, an administrator-created Cognito username/password user can complete the real hosted-login flow, return to https://ops.dtcdev.click/auth/callback, use the portal as the matching DataOps user, log out, and fail to reuse the old session. Capture the tested identity, time, browser result, and screenshot without recording credentials or tokens.
[HUMAN] After the shared stack owner enables Google, a real @datatalks.club Google account can sign in and a non-@datatalks.club Google account is rejected. Record the result without exposing credentials or tokens. Google is enabled on the shared user pool; this real-account allow/deny check keeps the issue open with the human label until a human verifies both account classes.
Test Scenarios
Scenario: Signed-out operator starts secure login
Given: No valid DataOps session cookie exists
When: The operator opens a protected portal page
Then: DataOps redirects through /login to the shared authorize endpoint with fresh state, nonce, and S256 PKCE while an API request receives JSON 401
Scenario: Valid shared identity becomes the correct local operator
Given: Cognito returns a valid token for an enabled DataOps user email
When: The callback validates and exchanges the authorization code
Then: DataOps creates a bounded opaque session, redirects to a clean same-origin page, /api/me returns that local user, and authorization/audit behavior uses that user's ID and role
Scenario: Callback tampering or replay fails closed
Given: State, nonce, PKCE data, token claims, signature, or transaction expiry is invalid, or a completed callback is replayed
When: The callback is requested
Then: No session is created, a safe error is returned, and no code/token/verifier/session value is logged or rendered
Scenario: Unknown or disabled shared identity is denied
Given: A valid Cognito identity has no unique enabled DataOps user mapping
When: The callback completes
Then: DataOps returns 403 without auto-provisioning or revealing whether another account exists
Scenario: Session expiry and logout revoke access
Given: An operator has a valid OAuth-created DataOps session
When: The session expires or the operator logs out
Then: The server-side session and cookie no longer authorize portal or API access and Cognito logout returns to the registered DataOps URI
Scenario: Non-browser credentials retain their boundaries
Given: Existing intake, integration, webhook, layered, and bearer-session clients
When: They call their current routes with valid, missing, and invalid credentials
Then: Their pre-migration allow/deny behavior is preserved, additional credentials are still enforced, and none of those credentials grants browser access
Scenario: CI deploys the relying-party configuration
Given: The auth change reaches main
When: the DataOps deployment workflow runs
Then: tests and SAM validation/build pass, GitHub OIDC deploys dataops-v1 in eu-west-1, and the workflow does not mutate the shared auth stack in us-east-1
Out of Scope
Creating, redeploying, or changing the shared Cognito stack in ../aws-infra, including the Dapier, Datamailer, AI Engineering Gym, and AI System Design Studio clients.
Creating Google OAuth credentials or enabling the Google identity provider; those require the shared-stack owner and real external credentials.
Auto-provisioning DataOps users from Cognito, synchronizing Cognito groups to DataOps roles, or building user lifecycle administration across services.
Migrating existing DataOps password hashes into Cognito or deleting historical Basic-auth secrets from AWS Secrets Manager.
Replacing route-specific API keys, webhook signatures, integration credentials, or bearer-session APIs with browser OAuth.
Redesigning the DataOps workspace beyond the login, callback, logout, and authentication error states needed for this migration.
Dependencies
External contract (already available): CloudFormation stack dtcdev-shared-auth in us-east-1, auth base https://auth.dtcdev.click, its IssuerUrl/JWKS outputs, and the DataOps public client registered for https://ops.dtcdev.click/auth/callback.
Deployment contract: dataops-v1 remains in eu-west-1 and is deployed by .github/workflows/deploy-dataops-v1.yml through arn:aws:iam::817685572750:role/dataops-github-actions-deploy.
Local identity contract: DataOps users, roles, and disabled status in the existing users table remain the authorization source after Cognito proves identity.
Google federation is enabled on the shared user pool. The Dapier, DataOps, AI Engineering Gym, AI System Design Studio, and Datamailer clients all support both COGNITO and Google. Infrastructure enablement is complete; the remaining dependency is the unchecked [HUMAN] live allow/deny verification with real in-domain and out-of-domain Google accounts.
Integrate DataOps with the shared Cognito browser login
Status: in progress
Tags:
enhancement,portal,frontend,backend,infra,testing,P1Depends on: None
Blocks: None
Scope
Replace the production DataOps browser gate at
ops.dtcdev.clickwith the shared serverless authentication service athttps://auth.dtcdev.click. The DataOps Lambda remains the relying party: it starts Cognito authorization-code + S256 PKCE, handleshttps://ops.dtcdev.click/auth/callback, validates the returned identity, maps it to an existing enabled DataOps user by normalized email, and creates an opaque server-side DataOps session carried in a secure cookie.The shared stack is already deployed as
dtcdev-shared-authinus-east-1. Its DataOps public client is registered for:https://ops.dtcdev.click/auth/callbackhttps://ops.dtcdev.click/openid email profileUserPoolId/IssuerUrloutput of that stackThe DataOps application and stack remain in this repository and deploy as
dataops-v1ineu-west-1through.github/workflows/deploy-dataops-v1.ymlwith GitHub Actions OIDC. Shared Cognito ownership remains in../aws-infra/sandbox/auth; this issue only consumes its non-secret contract.This migration replaces browser Basic auth and the password-derived
dtc_authcookie. It must not weaken or silently replace the independent credentials used by machine-to-machine intake, integrations, webhooks, or existing bearer-session clients. Route-specific layered authorization remains route-specific.Acceptance Criteria
GET /loginstarts authorization athttps://auth.dtcdev.click/oauth2/authorizeusing the configured DataOps client, exact callback URL,response_type=code,openid email profile, an unpredictablestateandnonce, and an S256 PKCE challenge. OAuth transaction state is short-lived, one-time, integrity-protected, and never stored in browserlocalStorage.GET /auth/callbackrejects missing, expired, replayed, or mismatched state; OAuth provider errors; missing codes; failed token exchanges; and invalid tokens without creating a DataOps session./oauth2/tokenwith the original PKCE verifier and validates the ID token signature against the issuer JWKS plus issuer, DataOps client audience, expiry/not-before, nonce,email, andemail_verified. JWKS caching handles key rotation safely. Tokens, authorization codes, PKCE verifiers, and session identifiers are absent from response bodies, URLs after callback completion, and application logs./api/me, role checks, assignment/notification filtering, and audit fields instead of the currentportal-adminidentity.HttpOnly; Secure; SameSite=Lax; Path=/cookie; expired, deleted, fabricated, and revoked sessions are rejected. OAuth-created expiry enforcement must not silently invalidate or change the contract of existing non-browser bearer sessions./api/*calls without Basic credentials or a browser-managed bearer token. Unauthenticated page requests redirect to/login; unauthenticated data/API requests return 401 without HTML or protected data.GETorPOST /logoutdeletes the local server-side session, expires the cookie, and uses the Cognito logout endpoint with the registered DataOps logout URI. A logged-out cookie cannot be reused.BASIC_AUTH_USERNAME/ the Basic-auth password secret at Lambda runtime. The legacy/api/auth/loginendpoint remains unavailable in production portal mode. Deleting historical Secrets Manager resources is not required.infra/template.full.yamlexposes the shared auth base URL, issuer/user-pool contract, DataOps public client ID, callback URL, logout URL, and session-lifetime configuration as non-secret deployment configuration. The Lambda gains only permissions actually required by the implementation; no Cognito client secret or Google secret is copied into this repository or GitHub Actions..github/workflows/deploy-dataops-v1.ymlpasses the auth configuration and continues to validate/build/deploydataops-v1exclusively through GitHub Actions OIDC. The workflow does not deploy or mutatedtcdev-shared-auth, and the application is not manually deployed for this issue./api/me/role behavior. Token and JWKS HTTP calls are stubbed; automated tests do not require a live Cognito account.npm --prefix backend test,npm --prefix backend run typecheck,npm --prefix backend run build,npm --prefix backend run test:e2e,make sam-validate, andmake sam-build, reporting exact commands, exit codes, and test counts. Playwright captures and inspects screenshots for the signed-out entry/error state and the authenticated DataOps workspace; no screenshot may show a 404, error page, broken layout, overlap, or leaked credential.https://ops.dtcdev.click/auth/callback, use the portal as the matching DataOps user, log out, and fail to reuse the old session. Capture the tested identity, time, browser result, and screenshot without recording credentials or tokens.@datatalks.clubGoogle account can sign in and a non-@datatalks.clubGoogle account is rejected. Record the result without exposing credentials or tokens. Google is enabled on the shared user pool; this real-account allow/deny check keeps the issue open with thehumanlabel until a human verifies both account classes.Test Scenarios
Scenario: Signed-out operator starts secure login
Given: No valid DataOps session cookie exists
When: The operator opens a protected portal page
Then: DataOps redirects through
/loginto the shared authorize endpoint with fresh state, nonce, and S256 PKCE while an API request receives JSON 401Scenario: Valid shared identity becomes the correct local operator
Given: Cognito returns a valid token for an enabled DataOps user email
When: The callback validates and exchanges the authorization code
Then: DataOps creates a bounded opaque session, redirects to a clean same-origin page,
/api/mereturns that local user, and authorization/audit behavior uses that user's ID and roleScenario: Callback tampering or replay fails closed
Given: State, nonce, PKCE data, token claims, signature, or transaction expiry is invalid, or a completed callback is replayed
When: The callback is requested
Then: No session is created, a safe error is returned, and no code/token/verifier/session value is logged or rendered
Scenario: Unknown or disabled shared identity is denied
Given: A valid Cognito identity has no unique enabled DataOps user mapping
When: The callback completes
Then: DataOps returns 403 without auto-provisioning or revealing whether another account exists
Scenario: Session expiry and logout revoke access
Given: An operator has a valid OAuth-created DataOps session
When: The session expires or the operator logs out
Then: The server-side session and cookie no longer authorize portal or API access and Cognito logout returns to the registered DataOps URI
Scenario: Non-browser credentials retain their boundaries
Given: Existing intake, integration, webhook, layered, and bearer-session clients
When: They call their current routes with valid, missing, and invalid credentials
Then: Their pre-migration allow/deny behavior is preserved, additional credentials are still enforced, and none of those credentials grants browser access
Scenario: CI deploys the relying-party configuration
Given: The auth change reaches
mainWhen: the DataOps deployment workflow runs
Then: tests and SAM validation/build pass, GitHub OIDC deploys
dataops-v1ineu-west-1, and the workflow does not mutate the shared auth stack inus-east-1Out of Scope
../aws-infra, including the Dapier, Datamailer, AI Engineering Gym, and AI System Design Studio clients.Dependencies
dtcdev-shared-authinus-east-1, auth basehttps://auth.dtcdev.click, itsIssuerUrl/JWKS outputs, and the DataOps public client registered forhttps://ops.dtcdev.click/auth/callback.dataops-v1remains ineu-west-1and is deployed by.github/workflows/deploy-dataops-v1.ymlthrougharn:aws:iam::817685572750:role/dataops-github-actions-deploy.COGNITOandGoogle. Infrastructure enablement is complete; the remaining dependency is the unchecked[HUMAN]live allow/deny verification with real in-domain and out-of-domain Google accounts.