Skip to content

feat: AWS RDS/Aurora IAM authentication for the PostgreSQL sink - #1591

Open
dcoric wants to merge 38 commits into
finos:feat/postgresfrom
dcoric:feat/postgres-rds-iam-auth
Open

feat: AWS RDS/Aurora IAM authentication for the PostgreSQL sink#1591
dcoric wants to merge 38 commits into
finos:feat/postgresfrom
dcoric:feat/postgres-rds-iam-auth

Conversation

@dcoric

@dcoric dcoric commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Description

Draft, stacked on #1532 (PostgreSQL sink) and #1584 (connection configuration). The diff below currently includes those changes. Once they merge, this branch will be rebased onto main so only the IAM-auth changes remain, and the draft flag will be removed.

Adds AWS RDS/Aurora IAM token authentication to the PostgreSQL sink, the follow-up to #1497 / #1532 (the mongo backend already supports AWS auth via AWS_CREDENTIAL_PROVIDER; the postgres sink previously only accepted a static password).

  • A new awsIamAuth option ({ enabled, region }) on the postgres sink. When enabled, the pool is built from the discrete host / port / user fields and the static password is replaced by a token provider function. pg invokes it for every new connection, so each one receives a fresh (~15 min) IAM token and refresh is automatic - no password is stored.
  • The token is generated from the AWS SDK default credential chain via @aws-sdk/rds-signer, loaded lazily and declared as an optional dependency so non-AWS installs stay lean and still type-check. Enabling IAM auth without the dependency installed throws a clear, actionable error.
  • TLS is required by RDS for IAM auth, so ssl defaults to true when unset; a connectionString is ignored in this mode.
  • region resolves from awsIamAuth.region, then AWS_REGION / AWS_DEFAULT_REGION, then the SDK's default region resolution.
  • Unit tests cover token-as-password, TLS defaulting, explicit ssl preserved, connection-string ignored, region resolution (config / AWS_REGION / AWS_DEFAULT_REGION and their priority), the default port, the missing-dependency error, and token-generation failure propagation. There is no integration test - the IAM path needs a real RDS/Aurora instance and credentials.
{
  "type": "postgres",
  "host": "mydb.abc123.eu-west-2.rds.amazonaws.com",
  "port": 5432,
  "user": "gitproxy_iam",
  "database": "gitproxy",
  "awsIamAuth": { "enabled": true, "region": "eu-west-2" },
  "enabled": true
}

Known follow-ups (non-blocking)

  • tsc is not fully independent of the optional dependency: after a slim install (npm install --omit=optional) the dynamic-import specifier resolves to a TS2307. CI uses npm ci, which installs optional dependencies, so type checks and builds pass; runtime is unaffected. A follow-up could suppress the import-line diagnostic.
  • Pre-existing and out of scope: logConfiguration logs the full sink config, which in the non-IAM (static password) path serializes credentials. IAM mode is safe (the token lives only in the pool's password function and is never written back into the config), but redacting credentials in that log line is a worthwhile separate fix.

Related Issue

Resolves #1561

Checklist

General

Documentation

  • Documentation has been added/updated for any new features

Configuration

  • If configuration schema (config.schema.json) was modified: regenerated src/config/generated/config.ts

Tests

  • Tests have been added/updated for new functionality
  • Unit tests pass (npm test)
  • Linting and formatting pass (npm run lint and npm run format:check)
  • Type checks pass (npm run check-types)

dcoric added 10 commits June 8, 2026 13:41
# Conflicts:
#	src/config/generated/config.ts
Adds an `awsIamAuth` object ({ enabled, region }) to the PostgreSQL sink
schema and regenerates the config types. This is the configuration surface
for authenticating to Amazon RDS/Aurora with an IAM auth token instead of a
static password; the connection wiring follows in a later commit.
When `awsIamAuth.enabled` is set, the pool is built from the discrete
host/port/user fields and the static password is replaced by a provider
function. node-postgres calls it for every new connection, so each one gets
a fresh (~15 min) IAM token and refresh is automatic; no password is stored.

The token is generated with the AWS SDK default credential chain via
`@aws-sdk/rds-signer`, which is loaded lazily and declared as an optional
dependency so non-AWS installs stay lean and still type-check. TLS is
required by RDS for IAM auth, so `ssl` defaults to true when unset, and a
connection string is ignored in this mode.
Adds unit coverage for the IAM auth path: a generated token is used as the
password, TLS defaults on, an explicit ssl setting is preserved, a connection
string is ignored, the region falls back to AWS_REGION, a missing host/user
raises a clear error, and a token-generation failure propagates. The optional
@aws-sdk/rds-signer module is stubbed so no real AWS credentials are needed.
Adds an AWS RDS/Aurora IAM authentication section to the PostgreSQL backend
docs (config example, region/TLS behavior, the rds_iam grant, and the
optional @aws-sdk/rds-signer dependency) and drops the stale "no IAM auth
helper" limitation note.
Closes the coverage gaps the review surfaced on the IAM auth path:
- the loadRdsSigner catch branch (the actionable "install the optional
  @aws-sdk/rds-signer dependency" error a user hits after
  `npm install --omit=optional`), exercised by re-importing the helper
  against a registry where the dynamic import rejects;
- the AWS_DEFAULT_REGION fallback and the AWS_REGION-over-AWS_DEFAULT_REGION
  priority;
- the default IAM token port (5432) reaching the signer when none is set.
@netlify

netlify Bot commented Jun 12, 2026

Copy link
Copy Markdown

Deploy Preview for endearing-brigadeiros-63f9d0 ready!

Name Link
🔨 Latest commit 6ae7a99
🔍 Latest deploy log https://app.netlify.com/projects/endearing-brigadeiros-63f9d0/deploys/6a67319d3451a20007b1acc4
😎 Deploy Preview https://deploy-preview-1591.git-proxy.preview.finos.org
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@codecov

codecov Bot commented Jun 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.20215% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.49%. Comparing base (b44f5ee) to head (6ae7a99).
⚠️ Report is 190 commits behind head on feat/postgres.

Files with missing lines Patch % Lines
src/db/postgres/users.ts 92.98% 12 Missing ⚠️
src/service/index.ts 33.33% 8 Missing ⚠️
src/db/postgres/helper.ts 94.89% 6 Missing and 1 partial ⚠️
src/db/index.ts 42.85% 4 Missing ⚠️
src/db/postgres/repo.ts 95.55% 4 Missing ⚠️
src/db/postgres/pushes.ts 97.14% 3 Missing ⚠️
Additional details and impacted files
@@                Coverage Diff                @@
##           feat/postgres    #1591      +/-   ##
=================================================
+ Coverage          90.41%   90.49%   +0.08%     
=================================================
  Files                 87       92       +5     
  Lines               8661     9218     +557     
  Branches            1603     1729     +126     
=================================================
+ Hits                7831     8342     +511     
- Misses               799      844      +45     
- Partials              31       32       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread src/db/postgres/helper.ts
const config: PoolConfig = {};
const iamAuthEnabled = Boolean(db.awsIamAuth?.enabled);

if (iamAuthEnabled) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same silent-precedence point as in #1584: when awsIamAuth.enabled is set, connectionString is dropped without any signal. A console.warn noting it's being ignored in IAM mode would make it observable

@dcoric
dcoric changed the base branch from main to feat/postgres August 24, 2026 06:32
@dcoric
dcoric marked this pull request as ready for review August 24, 2026 10:21
@dcoric
dcoric requested a review from a team as a code owner August 24, 2026 10:21
dcoric added 4 commits August 24, 2026 13:03
…-connection-config

# Conflicts:
#	website/docs/architecture/architecture.md
…-iam-auth

# Conflicts:
#	test/db/postgres/helper.test.ts
#	website/docs/architecture/architecture.md
@jescalada
jescalada force-pushed the feat/postgres-rds-iam-auth branch 2 times, most recently from 06e8ef8 to 856c8ab Compare August 26, 2026 05:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PostgreSQL sink: AWS RDS IAM authentication support

2 participants