From 52433ac704b43f6b06ccf6df4f7acda3473a32e0 Mon Sep 17 00:00:00 2001 From: Aditya Pareek <119134401+LOGANBLUE1@users.noreply.github.com> Date: Tue, 4 Aug 2026 10:53:08 +0530 Subject: [PATCH] added july changes --- SnowflakeConnDest.properties | 12 +- SnowflakeConnSource.properties | 12 +- keypair-auth-setup.md | 199 +++++++++++++++++++++++++++++++++ readmeRun-Mig.md | 6 + 4 files changed, 217 insertions(+), 12 deletions(-) create mode 100644 keypair-auth-setup.md create mode 100644 readmeRun-Mig.md diff --git a/SnowflakeConnDest.properties b/SnowflakeConnDest.properties index 9034d908..b1d5947f 100644 --- a/SnowflakeConnDest.properties +++ b/SnowflakeConnDest.properties @@ -1,7 +1,7 @@ [Snowflake] -URL= -USER= -PASSWORD= -WAREHOUSE= -DB= -SCHEMA= \ No newline at end of file +URL=https://ugjmdjg-fp46687.snowflakecomputing.com +USER=ZINGGTEST1JUL +PASSWORD=zinggTest@1Jul +WAREHOUSE=compute_wh +DB=zingg +SCHEMA=public diff --git a/SnowflakeConnSource.properties b/SnowflakeConnSource.properties index 9034d908..3d3380e3 100644 --- a/SnowflakeConnSource.properties +++ b/SnowflakeConnSource.properties @@ -1,7 +1,7 @@ [Snowflake] -URL= -USER= -PASSWORD= -WAREHOUSE= -DB= -SCHEMA= \ No newline at end of file +URL=https://nxlgozv-rr15407.snowflakecomputing.com +USER=ZINGGTEST4JUN +PASSWORD=zinggTest@4Jun +WAREHOUSE=compute_wh +DB=zingg +SCHEMA=public diff --git a/keypair-auth-setup.md b/keypair-auth-setup.md new file mode 100644 index 00000000..3679051e --- /dev/null +++ b/keypair-auth-setup.md @@ -0,0 +1,199 @@ +# Snowflake Key‑Pair Authentication — Local Setup + +## Why this exists + +Snowflake is phasing out password authentication for automated / non‑interactive +workloads (its strong‑auth mandate). Our Snowflake test and CLI paths therefore +authenticate with an **RSA key pair** instead of a username + password. + +This guide gets a new developer connecting locally with key‑pair auth. The same +key mechanism is what CI uses (see [CI notes](#ci-notes)). + +> **FIPS constraint — read this first.** Our runtime uses the **FIPS** build of +> Snowpark (`snowpark-fips-*-with-dependencies.jar`). That jar does **not** bundle +> a BouncyCastle crypto provider, and the JDK in this environment cannot decrypt +> encrypted (PBES2) private keys — you'll hit `PBES2 SecretKeyFactory not +> available`. **Therefore we use an *unencrypted* PKCS#8 private key.** Protect it +> with file permissions (`chmod 600`), never commit it, and inject it from a +> secret in CI. Do **not** generate an encrypted key for this setup unless you are +> also prepared to add and register a certified `bc-fips` provider. + +--- + +## Prerequisites + +- `openssl` (ships with macOS / Linux). +- ACCOUNTADMIN access to the Snowflake account **or** someone who has it and can + register your public key. +- The user account you'll authenticate as (e.g. `ZINGGTEST1JUL`). + +--- + +## Step 1 — Generate an (unencrypted) key pair + +Keep keys out of the repo. Convention: `~/.snowflake/keys/`. + +```bash +KEYDIR="$HOME/.snowflake/keys" +mkdir -p "$KEYDIR" + +# unencrypted PKCS#8 private key +openssl genrsa 2048 2>/dev/null \ + | openssl pkcs8 -topk8 -nocrypt -inform PEM -out "$KEYDIR/rsa_key_plain.p8" +chmod 600 "$KEYDIR/rsa_key_plain.p8" + +# matching public key +openssl rsa -in "$KEYDIR/rsa_key_plain.p8" -pubout -out "$KEYDIR/rsa_key.pub" +chmod 644 "$KEYDIR/rsa_key.pub" +``` + +Print the public‑key body as a single line (no header/footer) for the next step: + +```bash +grep -v "PUBLIC KEY" "$KEYDIR/rsa_key.pub" | tr -d '\n'; echo +``` + +--- + +## Step 2 — Register the public key in Snowflake + +Run as **ACCOUNTADMIN** in a worksheet for the target account. Substitute your +user and the public‑key body from Step 1. + +```sql +-- Make it a non-interactive service account (this also disables password login) +ALTER USER SET TYPE = SERVICE; + +-- Register the PUBLIC key (paste the single-line body, no BEGIN/END lines) +ALTER USER SET RSA_PUBLIC_KEY = 'MIIBIjANBgkqh...'; +``` + +Register the **public** key only. Your private key never leaves your machine. + +> Order matters: register the public key **before** (or in the same session as) +> `SET TYPE = SERVICE`, so you don't lock out the old password before key‑pair +> auth is in place. + +--- + +## Step 3 — Verify the registration matches your key + +The metadata fingerprint must equal your key's fingerprint. + +In Snowflake: +```sql +DESC USER ; -- read the RSA_PUBLIC_KEY_FP row +``` + +Locally: +```bash +echo -n "SHA256:"; \ +openssl rsa -in "$HOME/.snowflake/keys/rsa_key_plain.p8" -pubout 2>/dev/null \ + | openssl rsa -pubin -outform DER 2>/dev/null \ + | openssl dgst -sha256 -binary | openssl base64 +``` + +The two `SHA256:...` values must match exactly. If `RSA_PUBLIC_KEY_FP` is empty or +different, the `ALTER USER` didn't take or the key was pasted wrong — redo Step 2. + +> **Propagation delay:** right after `SET RSA_PUBLIC_KEY`, `DESC USER` shows the +> fingerprint immediately, but Snowflake's auth cache can take a few minutes to +> pick up the new key. If the fingerprints match but login fails with +> `JWT token is invalid`, **wait a few minutes and retry** — it is not a config +> error. + +--- + +## Step 4 — Point the connection properties at your key + +Edit the connection properties file you use (e.g. +`snowflake/snowflakeConn.properties`). Remove any `PASSWORD=` line and set: + +```properties +[SNOWFLAKE] +URL=https://.snowflakecomputing.com +USER= +PRIVATE_KEY_FILE=/absolute/path/to/rsa_key_plain.p8 +PRIVATE_KEY_FILE_PWD= +WAREHOUSE=compute_wh +DB=zingg +SCHEMA=public +ACCOUNT= +``` + +Notes: +- `PRIVATE_KEY_FILE` must be an **absolute** path — the CLI (`SnowClient`) resolves + it relative to the current working directory, and `rsa_key.p8` alone won't be + found. +- `PRIVATE_KEY_FILE_PWD` is **empty** because the key is unencrypted. +- **Never commit real key material or a real path.** These local values live in an + untracked properties file. Committed properties files keep placeholders. + +--- + +## Step 5 — Run + +```bash +caffeinate ./scripts/zingg.sh \ + --phase train \ + --conf ./examples/febrl/config.json \ + --properties-file ./snowflakeConn.properties +``` + +Success looks like a JDBC log line `Opening session with server: ... password is +not provided ... private key file: /.../rsa_key_plain.p8` followed by the phase +running. Any errors after that are about data/tables/license, not authentication. + +--- + +## How the code consumes the key + +There are three connection paths and they behave slightly differently: + +| Path | File | Source of key path + passphrase | +| --- | --- | --- | +| Product CLI | `snowflake/client/.../SnowClient.java` (`configFile()`) | **Only** the properties file. Does *not* read env vars. | +| Integration tests | `snowflake/infraForTest/.../SnowflakeConnectionInjector.java` | Properties file, **overridden** by env `SNOWFLAKE_PRIVATE_KEY_FILE` / `SNOWFLAKE_PRIVATE_KEY_FILE_PWD`. | +| Python scripts | `snowflake/snowflake_connection.py` | Env `SNOWFLAKE_PRIVATE_KEY_FILE` / `SNOWFLAKE_PRIVATE_KEY_FILE_PWD`, falling back to the properties file. | + +For the CLI you must put the (absolute) key path in the properties file. For tests +and Python you can instead export env vars and leave the committed properties as +placeholders. + + +## CI notes + +- The private key is stored as a GitHub Actions **secret** (`SNOWFLAKE_PRIVATE_KEY`), + set by a repo/org admin. Because the key is unencrypted there is **no passphrase + secret**. +- A workflow step writes the secret to a temp file and exports the env var the + test/Python code reads, e.g.: + ```yaml + - name: Provision Snowflake key + run: | + echo "${{ secrets.SNOWFLAKE_PRIVATE_KEY }}" > "$RUNNER_TEMP/rsa_key.p8" + chmod 600 "$RUNNER_TEMP/rsa_key.p8" + echo "SNOWFLAKE_PRIVATE_KEY_FILE=$RUNNER_TEMP/rsa_key.p8" >> "$GITHUB_ENV" + ``` +- Never echo the key into logs beyond writing the file; GitHub masks secret values + but avoid printing them. + +--- + +## Troubleshooting + +| Symptom | Cause | Fix | +| --- | --- | --- | +| `JWT token is invalid` (fingerprints match) | Auth‑cache propagation delay after registering the key | Wait a few minutes, retry | +| `JWT token is invalid` (fingerprints differ / `RSA_PUBLIC_KEY_FP` empty) | Wrong or truncated public key registered | Re‑run Step 2 with the exact single‑line body | +| `NoSuchFileException: rsa_key...` | `PRIVATE_KEY_FILE` is relative / wrong | Use an absolute path | +| `PBES2 SecretKeyFactory not available` / `Could not extract private key` | Encrypted key in a FIPS runtime with no BC provider | Use an **unencrypted** PKCS#8 key (Step 1) | +| `password is not provided` then a data error | Auth succeeded; failure is downstream | Check the config, tables, warehouse, license | + +## Rotating a key + +1. Generate a new unencrypted pair (Step 1). +2. `ALTER USER SET RSA_PUBLIC_KEY = '';` (you can use + `RSA_PUBLIC_KEY_2` to stage a second key for zero‑downtime rotation). +3. Update the local properties file / the `SNOWFLAKE_PRIVATE_KEY` CI secret. +4. Once everything uses the new key, unset the old one. diff --git a/readmeRun-Mig.md b/readmeRun-Mig.md new file mode 100644 index 00000000..974fa31f --- /dev/null +++ b/readmeRun-Mig.md @@ -0,0 +1,6 @@ +This is specific to zingg usage +Make sure that you have created a new account +Create a new DB: zingg +create a new stage: zingg_stage (Client side encryption) +change the source and dest SFconn files +python3 run-migration.py