Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .changeset/slot-lookup-ratchet-enforcement.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
---

ci(tooling): enforce the #4251 slot-lookup sweep ratchet with a counted baseline

The grandfather list shipped as paths in `eslint.config.mjs` under a comment
reading "NEVER add an entry" — a promise nothing checked. ESLint cannot express
a ratchet: an ignored file is ignored completely, so a NEW `getService<any>(…)`
added to an already-listed file rode the old entry with `pnpm lint` green.

The list moves to `scripts/slot-lookup-baseline.json` (file → site count) and
becomes the single source: its keys are the config's `ignores`, its values are
what `pnpm check:slot-lookup` enforces. The script re-runs ESLint with the
grandfathering lifted and matches reports by the rule's exact message, so the
counter cannot drift from the rule. It fails on a new file, a grown count, a
stale entry, and — via a key-set comparison against the merge base with main —
on a file being ADDED to the list, which no count check can see.

Tooling and CI only; releases nothing.
15 changes: 15 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,12 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v7
with:
# The slot-lookup ratchet compares the baseline against its state at
# the merge base with main — the only way to see a file being ADDED
# to the grandfather list. A shallow clone has no merge base, and the
# check would degrade to "not verified" on every run.
fetch-depth: 0

- name: Setup Node.js
uses: actions/setup-node@v7
Expand Down Expand Up @@ -57,6 +63,15 @@ jobs:
- name: ESLint
run: pnpm lint

# Slot-lookup sweep ratchet (#4251). `pnpm lint` above bans erasing a
# service-lookup result to `any` across packages/, but the files still
# holding pre-existing sites are grandfathered by path — and an ignored
# file is ignored COMPLETELY, so new erasures added to one ride the old
# entry in silence. This re-measures those files with the grandfathering
# lifted and holds them to a per-file count, so the list can only shrink.
- name: Slot-lookup ratchet
run: pnpm check:slot-lookup

# Raw NUL guard (#3127): one literal U+0000 byte makes grep/ripgrep treat
# the whole file as binary and silently return ZERO matches — the file drops
# out of code search and out of every grep-based lint, with no error saying
Expand Down
89 changes: 31 additions & 58 deletions eslint.config.mjs
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.

import { readFileSync } from 'node:fs';

import tsParser from '@typescript-eslint/parser';

// Flat ESLint config — guards against memory-bloating import patterns.
Expand Down Expand Up @@ -73,7 +75,10 @@ const SLOT_LOOKUPS = ['resolveService', 'getService', 'getRequestKernelService']
// callers read only `getPort()`.
const UNCONTRACTED_SLOTS = ['protocol', 'mcp', 'kernel-resolver', 'scope-manager', 'http\\.server'].join('|');

const SLOT_LOOKUP_ANY_MESSAGE =
// Exported so `scripts/check-slot-lookup-ratchet.mjs` can identify THIS rule's
// reports among the other `no-restricted-syntax` rules, by exact message —
// the counter and the rule must never be able to disagree about what counts.
export const SLOT_LOOKUP_ANY_MESSAGE =
'Do not erase a service-lookup result to `any` (`: any`, `as any`, or a ' +
'`getService<any>(…)` type argument) — the lookup already returns the slot\'s ' +
'contract (#4168/#4176/#4202), and this switches that checking off ' +
Expand All @@ -86,62 +91,29 @@ const SLOT_LOOKUP_ANY_MESSAGE =
'note, so the exemption is reviewed once and visible in one place — see ' +
'issues #4127 and #4251.';

// [#4251] The sweep ratchet. These files hold pre-existing lookup-erasure
// sites — `getService<any>(…)`, `: any`, or `as any` — that predate the rule
// reaching them: the rule's scope was packages/runtime only until #4251
// widened it, and the type-argument selector did not exist. Enumerated by
// running this config with this list emptied: 180 sites in 44 files (the
// issue's 80 was the non-test `<any>` form alone; the annotation forms and
// test files the old selectors would have caught under a wider scope roughly
// double it). They are grandfathered BY FILE, here, for the same reason
// UNCONTRACTED_SLOTS is central: `--no-inline-config` means the escape must
// live in config, and a shrinking list in one place is the ratchet made
// visible. Batches remove entries as they sweep (see #4214 for the batch
// pattern and its yield — these sites are where the erased contracts live).
// NEVER add an entry: a new file starts covered, and a new violation in a
// listed file rides an existing entry only until its batch.
const SLOT_LOOKUP_UNSWEPT = [
'packages/cli/src/commands/migrate/files-to-references.ts',
'packages/cli/src/commands/migrate/value-shapes.ts',
'packages/cli/src/commands/serve.ts',
'packages/client/src/client.hono.test.ts',
'packages/cloud-connection/src/cloud-connection-plugin.ts',
'packages/cloud-connection/src/marketplace-install-local-plugin.ts',
'packages/core/examples/kernel-features-example.ts',
'packages/metadata-protocol/src/plugin.ts',
'packages/metadata/src/plugin.ts',
'packages/objectql/src/plugin.integration.test.ts',
'packages/objectql/src/plugin.ts',
'packages/plugins/plugin-approvals/src/approvals-plugin.ts',
'packages/plugins/plugin-approvals/src/status-mirror-cascade.integration.test.ts',
'packages/plugins/plugin-audit/src/audit-plugin.ts',
'packages/plugins/plugin-auth/src/auth-plugin.ts',
'packages/plugins/plugin-email/src/email-plugin.ts',
'packages/plugins/plugin-hono-server/src/current-user-endpoints.ts',
'packages/plugins/plugin-pinyin-search/src/pinyin-search-plugin.ts',
'packages/plugins/plugin-reports/src/reports-plugin.ts',
'packages/plugins/plugin-security/src/security-plugin.ts',
'packages/plugins/plugin-sharing/src/sharing-plugin.ts',
'packages/plugins/plugin-webhooks/src/webhook-outbox-plugin.ts',
'packages/qa/dogfood/test/showcase-agent-intersection.dogfood.test.ts',
'packages/qa/dogfood/test/showcase-bu-hierarchy-sharing.dogfood.test.ts',
'packages/qa/dogfood/test/showcase-d3-d4-capabilities.dogfood.test.ts',
'packages/qa/dogfood/test/showcase-permission-zoo.dogfood.test.ts',
'packages/rest/src/external-datasource-routes.ts',
'packages/rest/src/rest-api-plugin.ts',
'packages/services/service-datasource/src/admin-routes.ts',
'packages/services/service-job/src/job-service-plugin.ts',
'packages/services/service-messaging/src/messaging-service-plugin.test.ts',
'packages/services/service-messaging/src/messaging-service-plugin.ts',
'packages/services/service-queue/src/queue-service-plugin.ts',
'packages/services/service-realtime/src/realtime-service-plugin.ts',
'packages/services/service-settings/src/settings-service-plugin.ts',
'packages/services/service-sms/src/sms-plugin.ts',
'packages/services/service-storage/src/storage-service-plugin.ts',
'packages/triggers/trigger-record-change/src/formula-context.test.ts',
'packages/triggers/trigger-record-change/src/multilookup-context.test.ts',
'packages/triggers/trigger-record-change/src/record-change-integration.test.ts',
];
// [#4251] The sweep ratchet, read from `scripts/slot-lookup-baseline.json`.
//
// Those files hold pre-existing lookup-erasure sites — `getService<any>(…)`,
// `: any`, or `as any` — that predate the rule reaching them: the rule's scope
// was packages/runtime only until #4251 widened it, and the type-argument
// selector did not exist. 171 sites in 40 files at the widening; they are
// grandfathered BY FILE for the same reason UNCONTRACTED_SLOTS is central —
// `--no-inline-config` means the escape must live in config, and one shrinking
// list is the ratchet made visible. Batches remove entries as they sweep (see
// #4214 for the batch pattern and its yield — these sites are where the erased
// contracts live).
//
// The baseline is the SINGLE SOURCE: its keys are these ignores and its values
// are the per-file counts `pnpm check:slot-lookup` enforces. That coupling is
// the point (#4320 was found the same way — a promise nothing checked). A bare
// file list made three moves invisible: adding a file to silence lint, adding
// NEW violations to an already-listed file (they rode the entry silently), and
// clearing a file without dropping its entry (the list stops meaning anything).
// The counted baseline fails all three, and `--update` is the only way to move
// it — downward.
const SLOT_LOOKUP_UNSWEPT = Object.keys(JSON.parse(
readFileSync(new URL('./scripts/slot-lookup-baseline.json', import.meta.url), 'utf8'),
));

export default [
{
Expand Down Expand Up @@ -271,7 +243,8 @@ export default [
// held 77 of the 80 known sites, an unlinted majority that looked covered.
// Per-package curation would recreate that gap one package at a time, so the
// scope is total and the not-yet-swept files are grandfathered individually
// in SLOT_LOOKUP_UNSWEPT above — a shrinking list, not a silent boundary.
// in the counted baseline above — a shrinking list under `check:slot-lookup`,
// not a silent boundary.
//
// KNOWN RESIDUAL: a wrapper whose own return type is annotated
// (`const getEngine = async (): Promise<any> => …resolveService(…)`) erases
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
"check:role-word": "node scripts/check-role-word.mjs",
"check:org-identifier": "node scripts/check-org-identifier.mjs",
"check:authz-resolver": "node scripts/check-single-authz-resolver.mjs",
"check:slot-lookup": "node scripts/check-slot-lookup-ratchet.mjs",
"check:service-providers": "node scripts/check-service-providers.mjs",
"check:route-envelope": "node scripts/check-route-envelope.mjs --self-test && node scripts/check-route-envelope.mjs",
"check:error-code-casing": "node scripts/check-error-code-casing.mjs --self-test && node scripts/check-error-code-casing.mjs",
Expand Down
190 changes: 190 additions & 0 deletions scripts/check-slot-lookup-ratchet.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
#!/usr/bin/env node
// check-slot-lookup-ratchet — the #4251 sweep ratchet, enforced.
//
// `eslint.config.mjs` bans erasing a service-lookup result to `any`
// (#4127/#4214/#4251) across all of packages/, and grandfathers the files that
// still hold pre-existing sites by listing them in
// scripts/slot-lookup-baseline.json. ESLint's `ignores` alone cannot express a
// ratchet: an ignored file is ignored completely, so NEW erasures added to a
// listed file ride the existing entry in total silence — the same
// declared-but-unchecked shape this whole work line keeps finding (#4320's
// options configured a block that never ran; nothing checked the promise).
//
// So the baseline carries COUNTS, and this script is what makes them mean
// something. It fails when:
// • a file NOT in the baseline reports a site (that already fails `pnpm lint`
// — reported here too so one command explains the whole picture), or
// • a baselined file's count INCREASES (new erasure hiding behind an old
// entry — the invisible move), or
// • a baselined file's count DECREASED or the file is clean/gone (progress!)
// — run with --update to ratchet the baseline down and commit it.
//
// node scripts/check-slot-lookup-ratchet.mjs [--update]
//
// The counts are produced by running ESLint itself with the baseline's
// `ignores` lifted, and reports are matched by the rule's exact message
// (imported from the config). The counter therefore cannot drift from the
// rule: change the selectors and this re-measures against them.
//
// Sweeping a file means typing its lookups (pass the slot's contract), then
// `--update` to drop or shrink its entry. Entries only ever go down; a batch
// that adds one is doing the opposite of the job.
import { execFileSync } from 'node:child_process';
import { readFileSync, writeFileSync } from 'node:fs';
import { dirname, resolve, relative } from 'node:path';
import { fileURLToPath } from 'node:url';

import { ESLint } from 'eslint';

import eslintConfig, { SLOT_LOOKUP_ANY_MESSAGE } from '../eslint.config.mjs';

const __dirname = dirname(fileURLToPath(import.meta.url));
const repoRoot = resolve(__dirname, '..');
const BASELINE_PATH = 'scripts/slot-lookup-baseline.json';

const update = process.argv.includes('--update');
const baseline = JSON.parse(readFileSync(resolve(repoRoot, BASELINE_PATH), 'utf8'));
const baselinedFiles = new Set(Object.keys(baseline));

// The lint config with the grandfathering removed — every baselined file is
// measured as if it were already swept. Only the block that carries this rule
// is touched; every other config entry passes through untouched so the run
// stays byte-identical to `pnpm lint` in all other respects.
const carriesRule = (entry) => {
const rule = entry?.rules?.['no-restricted-syntax'];
return Array.isArray(rule) && rule.some((r) => r?.message === SLOT_LOOKUP_ANY_MESSAGE);
};

const measuringConfig = eslintConfig.map((entry) =>
carriesRule(entry)
? { ...entry, ignores: (entry.ignores ?? []).filter((p) => !baselinedFiles.has(p)) }
: entry,
);

if (!eslintConfig.some(carriesRule)) {
console.error(
'check-slot-lookup-ratchet: no config block carries the slot-lookup rule.\n' +
'The rule was renamed, removed, or its message changed without updating\n' +
'SLOT_LOOKUP_ANY_MESSAGE — refusing to report "clean" for a rule that is\n' +
'no longer being measured.',
);
process.exit(2);
}

const eslint = new ESLint({
cwd: repoRoot,
overrideConfigFile: true,
baseConfig: measuringConfig,
// Match the root `lint` script: this repo lints with --no-inline-config on
// purpose, so an eslint-disable comment must not shrink a count here either.
allowInlineConfig: false,
});

const results = await eslint.lintFiles(['packages/**/*.{ts,tsx,mts,cts}']);

const current = {};
for (const result of results) {
const hits = result.messages.filter((m) => m.message === SLOT_LOOKUP_ANY_MESSAGE).length;
if (hits > 0) current[relative(repoRoot, result.filePath).replace(/\\/g, '/')] = hits;
}

const sorted = Object.fromEntries(Object.entries(current).sort(([a], [b]) => a.localeCompare(b)));

if (update) {
writeFileSync(resolve(repoRoot, BASELINE_PATH), JSON.stringify(sorted, null, 2) + '\n');
const files = Object.keys(sorted).length;
const sites = Object.values(sorted).reduce((a, b) => a + b, 0);
console.log(`slot-lookup baseline updated: ${sites} site(s) in ${files} file(s).`);
process.exit(0);
}

const errors = [];
for (const [file, count] of Object.entries(sorted)) {
const allowed = baseline[file];
if (allowed === undefined) {
errors.push(
`${file}: NEW service-lookup erasure (${count} site(s)). Pass the slot's ` +
`contract type instead of \`any\` — see eslint.config.mjs and issue #4251. ` +
`This file is not grandfathered, and the baseline never grows.`,
);
} else if (count > allowed) {
errors.push(
`${file}: erasure count grew ${allowed} → ${count}. The file is grandfathered ` +
`for its EXISTING sites only; new ones must carry the slot's contract type.`,
);
}
}
for (const [file, allowed] of Object.entries(baseline)) {
const now = sorted[file];
if (now === undefined) {
errors.push(
`${file}: baselined file is clean/gone (was ${allowed}) — ratchet DOWN: run ` +
`\`pnpm check:slot-lookup --update\` and commit the baseline.`,
);
} else if (now < allowed) {
errors.push(
`${file}: erasure count fell ${allowed} → ${now} — ratchet DOWN: run ` +
`\`pnpm check:slot-lookup --update\` and commit the baseline.`,
);
}
}

// The key set must only ever SHRINK. Counts alone cannot see the last move:
// a genuinely-erasing NEW file added to the baseline matches its own count and
// sails through, which would turn the grandfather list into a general-purpose
// mute button. The reference is the baseline as it stands on the merge base
// with origin/main — on a sweep branch keys only disappear, and on main the
// merge base is HEAD, so the comparison is a no-op there.
let monotonicity = null;
try {
const git = (...args) =>
execFileSync('git', args, { cwd: repoRoot, encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] }).trim();
let base;
for (const ref of ['origin/main', 'main']) {
try { base = git('merge-base', 'HEAD', ref); break; } catch { /* try the next ref */ }
}
if (base) {
const previous = JSON.parse(git('show', `${base}:${BASELINE_PATH}`));
const added = Object.keys(baseline).filter((f) => !(f in previous));
monotonicity = { base: base.slice(0, 7), added };
}
} catch {
// No git, a shallow clone without the base, or the baseline is new on this
// branch (`git show` fails). Reported below rather than passed over — a
// check that cannot run must not read as a check that passed.
}

if (monotonicity?.added.length) {
for (const file of monotonicity.added) {
errors.push(
`${file}: ADDED to the baseline (not present at ${monotonicity.base}). The ` +
`grandfather list is not a mute button — it only ever shrinks. Type this ` +
`file's lookups instead; see issue #4251.`,
);
}
}

const totalSites = Object.values(sorted).reduce((a, b) => a + b, 0);
const totalFiles = Object.keys(sorted).length;

if (errors.length > 0) {
console.error(`✗ slot-lookup ratchet (${errors.length} problem(s)):\n`);
for (const e of errors) console.error(` • ${e}`);
console.error(
`\nUnswept: ${totalSites} site(s) in ${totalFiles} file(s). ` +
`Sweeping is #4251's batch work — see SLOT_LOOKUP_UNSWEPT in eslint.config.mjs.`,
);
process.exit(1);
}

console.log(
`✓ slot-lookup ratchet holds: ${totalSites} unswept site(s) in ${totalFiles} file(s), ` +
`none new. Every other file under packages/ is covered by \`pnpm lint\`.`,
);
console.log(
monotonicity
? ` baseline key set verified against ${monotonicity.base}: no files added.`
: ` NOT verified: could not read the baseline at the merge base with main ` +
`(no git, shallow clone, or the baseline is new here), so "no files added" ` +
`is unchecked this run.`,
);
Loading
Loading