Skip to content

Commit 21df925

Browse files
authored
ci(sea): Kernel E2E workflow — build napi binding from KERNEL_REV + run tests/e2e/sea on a live warehouse (#418)
* ci(sea): add Kernel E2E workflow — build napi binding from KERNEL_REV, run tests/e2e/sea on a live warehouse Brings the SEA-backend end-to-end gate onto main (it previously lived only in the stacked sea-kernel-rev-pin branch). Mirrors the Python connector's kernel-e2e.yml. What it does: - Reads the pinned kernel SHA from KERNEL_REV, checks the kernel out via a GitHub App token (private repo), runs `npm run build:native` to compile the napi binding into native/sea/ — so the binary matches KERNEL_REV exactly. - Asserts the committed native/sea/index.{d.ts,js} contract matches the rebuilt binding (catches drift between the pin and the committed router/types). - Smoke-checks the binding loads, then runs `tests/e2e/sea/**` against the azure-prod warehouse (maps the secrets onto the DATABRICKS_PECOTESTING_* vars the SEA suite gates on). Gate semantics (same pattern as the Thrift integration + Python kernel-e2e): - Plain PR events post a synthetic-success "Kernel E2E" check so PRs that don't touch SEA aren't blocked; the real suite runs in the merge queue. - The `kernel-e2e` label triggers a preview run on the PR and is auto-removed on push (a labelled run can't be re-triggered with unreviewed code). - merge_group runs the real gate when SEA-relevant files changed, auto-passes otherwise. All referenced pieces already exist on main: the setup-jfrog composite (npm + the inline cargo proxy config), KERNEL_REV, the build:native script, the committed native/sea contract, and tests/e2e/sea/. One-time admin setup required before this gates anything (documented in the workflow header): the `kernel-e2e` label, the INTEGRATION_TEST_APP_ID / INTEGRATION_TEST_PRIVATE_KEY GitHub App (allowlisted for databricks-sql-kernel), and the azure-prod environment exposing DATABRICKS_HOST / TEST_PECO_WAREHOUSE_HTTP_PATH / DATABRICKS_TOKEN. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * ci(sea): inject E2E_* vars so tests/e2e/sea actually runs (config.ts process.exit fix) The tests/e2e/sea suite reads creds from two sources: most tests read DATABRICKS_PECOTESTING_* directly (already injected), but e2e-smoke.test.ts imports tests/e2e/utils/config.ts, which process.exit(1)s at module load if any E2E_* var is unset — aborting the whole mocha run before it starts. Mirror main.yml's e2e-test job mapping (E2E_HOST/PATH/ACCESS_TOKEN + CATALOG/SCHEMA/ VOLUME/TABLE_SUFFIX) so the suite loads and runs. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * ci(sea): cache ~/.npm in kernel-e2e (mirror main.yml) so npm ci doesn't hit unreachable dev proxy Two package-lock entries (flatbuffers etc.) resolve to npm-proxy.dev.databricks.com, which the protected runner can't reach (ECONNRESET). main.yml's e2e job only succeeds because it caches ~/.npm — those tarballs load from the warm cache, never fetched. kernel-e2e had no such cache, so npm ci failed deterministically. Add the identical actions/cache step (same key) before npm ci. The driver has a single root package-lock.json and the kernel checkout adds none, so hashFiles('**/package-lock.json') matches main.yml's key exactly → restores main's warmed cache. Confirmed end-to-end up to this point: App token + private-kernel checkout + JFrog now all pass; npm ci was the only remaining failure. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * ci(sea): run SEA e2e under nyc so ts-node/register loads .ts (fix ESM-loader race) Bare `mocha` has no ts-node registration, so it loaded the .ts specs via the ESM dynamic-import path and crashed with ERR_INTERNAL_ASSERTION (... not yet fully loaded ... Promise.all) before any test ran. main.yml's e2e job avoids this by running `nyc mocha` — nyc.config.js's `require: ['ts-node/register']` installs the CommonJS ts-node hook, loading specs synchronously. Mirror that: invoke `nyc ... mocha --config tests/e2e/.mocharc.js <glob>`, keeping the glob at argv index 4 (the mocharc derives spec from process.argv.slice(4)). Everything else in the pipeline is now confirmed green: App-token + private-kernel checkout, ~/.npm cache, npm ci, build:native, the KERNEL_REV contract-drift assert, and the binding smoke-check. This was the last failing step. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> --------- Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
1 parent 03936ec commit 21df925

1 file changed

Lines changed: 415 additions & 0 deletions

File tree

0 commit comments

Comments
 (0)