refactor(test): no test may reach the send path; delivery gets its own probe

The middleware fail-open assertion needed fake credentials and a stubbed global
fetch just to reach a catch block. Before 1e6c94c the stub was missing and the suite
POSTed junk to google-analytics.com on every `npm test`, pre-commit and CI run; with
the stub it was harmless but still a test that had to be defused to be safe.

Replaced with a request the middleware cannot parse — `new URL('://not-a-url')`
throws inside the analytics block — which reaches the same catch with no credentials,
no stub and no send path. Nothing in this suite now goes near fetch: verified by
running it under a global fetch spy, which records nothing.

Delivery is the other half, and it cannot be asserted offline. `npm run
probe:agent-analytics` sends three events (one of them a 404) through
lib/agent-analytics.js itself, so a pass means the module, the credential and the
property agree rather than that a hand-written payload happens to be valid. Opt-in,
never in `npm test`: it needs a credential and a network, neither of which belongs in
a gate that runs at pre-commit and on pull requests.

Three things the probe refuses to do: run without both variables exported (with a
hint about the shell-variable-vs-export trap that costs an hour the first time), send
to the property eleventy.config.js reports to unless --force, and print the secret —
including in error messages, where a fetch failure would otherwise echo a URL that
carries api_secret. The site's property is READ from the config, not copied here.

Author: Mikhus

README.md

@@ -114,10 +114,21 @@ Setup, all free:
    `GA4_MP_MEASUREMENT_ID` and `GA4_MP_API_SECRET` (encrypt the secret). With either
    missing the module does nothing at all, which is also what keeps forks and preview
    deploys silent.
-4. **Verify once** by also setting `GA4_MP_DEBUG=1`: hits go to GA4's validation
-   endpoint and the result is logged to the project's function logs, where an empty
-   `validationMessages` means the payload is good. **Then unset it** — the validation
-   endpoint reports but records nothing.
+4. **Verify delivery** — `npm test` proves the logic offline and can prove nothing about
+   a real property, so this is a separate, opt-in step:
+
+   ```bash
+   export GA4_MP_MEASUREMENT_ID='G-…' GA4_MP_API_SECRET='…'
+   npm run probe:agent-analytics     # GA4_MP_DEBUG=1 to validate instead of send
+   ```
+
+   It sends three events through `lib/agent-analytics.js` itself — including a 404 — so
+   a pass means the module, the credential and the property agree. GA4 answers 204 to
+   valid and invalid hits alike, so the proof is **Realtime**, not the exit code. The
+   probe refuses to run against the property `eleventy.config.js` reports to, and never
+   prints the secret. On Pages, `GA4_MP_DEBUG=1` does the same validation server-side
+   and logs the verdict to the project's function logs; **unset it afterwards**, since
+   the validation endpoint reports but records nothing.
 5. Optional, for slicing: Admin → Custom definitions → register `crawler`,
    `operator`, `surface`, `status` and `edition` as **event-scoped custom
    dimensions**. Events are sent as `page_view` with `page_location`, so the built-in

package.json

@@ -30,6 +30,7 @@
     "test": "npm run check:redirects && npm run check:agent-analytics && npm run check:dates && npm run check:links && npm run check:sitemap",
     "check:redirects": "node scripts/check-redirects.js",
     "check:agent-analytics": "node scripts/check-agent-analytics.js",
+    "probe:agent-analytics": "node scripts/probe-agent-analytics.js",
     "check:dates": "node scripts/gen-page-dates.js --check",
     "check:sitemap": "node scripts/check-sitemap.js",
     "check:api-versions": "node scripts/check-api-versions.js",

scripts/check-agent-analytics.js

@@ -136,46 +136,22 @@ async function main() {
   ok('imqueue.net still 301s onto imqueue.org');
 
   // Constraint 1 in functions/_middleware.js: this file runs in front of every
-  // request to BOTH sites, so a throw here is an outage. A context missing
-  // waitUntil is the cheapest way to simulate the runtime not behaving as expected.
+  // request to BOTH sites, so a throw there is an outage, not a lost metric.
   //
-  // This is the ONLY assertion that reaches the send path, and global fetch is
-  // stubbed for it. Not tidiness: trackRequest starts the fetch BEFORE the caller
-  // touches waitUntil, so without the stub this suite POSTs to Google on every
-  // `npm test`, every pre-commit and every CI run — which contradicts the "offline
-  // pure logic" claim at the top of this file and litters a stranger's property with
-  // junk hits. Stubbing also makes the assertion stronger: the endpoint gets checked
-  // without anyone being contacted.
-  const realFetch = globalThis.fetch;
-  const sends = [];
-  globalThis.fetch = (target) => {
-    sends.push(String(target));
-
-    return Promise.resolve(new Response('{}'));
-  };
-
-  let broken;
-  try {
-    broken = await onRequest({
-      request: new Request('https://imqueue.org/llms.txt'),
-      env: { GA4_MP_MEASUREMENT_ID: 'G-X', GA4_MP_API_SECRET: 's' },
-      next: async () => page,
-      // no waitUntil at all
-    });
-  } finally {
-    globalThis.fetch = realFetch;
-  }
-
-  assert.strictEqual(broken, page, 'a broken analytics path must still serve the page');
+  // Provoked with a request the middleware cannot even parse — `new URL()` throws
+  // inside the analytics block — rather than by configuring credentials and stubbing
+  // fetch to reach the same catch. Same invariant, no fake secrets, and nothing in
+  // this suite goes anywhere near the send path: `npm test` runs at pre-commit and on
+  // every pull request, and a gate has no business making network calls, valid or not.
+  const unparseable = await onRequest({
+    request: { url: '://not-a-url', headers: { get: () => null } },
+    env: { GA4_MP_MEASUREMENT_ID: 'G-X', GA4_MP_API_SECRET: 's' },
+    next: async () => page,
+    waitUntil: () => {},
+  });
+  assert.strictEqual(unparseable, page, 'a broken analytics path must still serve the page');
   ok('analytics failure degrades to "no measurement", never to "no page"');
 
-  assert.strictEqual(sends.length, 1, 'exactly one send per tracked request');
-  assert.ok(
-    sends[0].startsWith('https://www.google-analytics.com/mp/collect?'),
-    `the send must go to GA4's collect endpoint, got: ${sends[0].split('?')[0]}`,
-  );
-  ok('one send, to GA4\'s collect endpoint (verified without a network call)');
-
   console.log(`\nAll ${checks} agent-analytics checks passed.`);
 }
 

scripts/probe-agent-analytics.js

@@ -0,0 +1,119 @@
+#!/usr/bin/env node
+// probe-agent-analytics.js — does the agent-analytics pipeline actually deliver?
+//
+// `npm test` proves the module's LOGIC offline: what gets sent, what gets skipped,
+// that the crawler's user-agent never reaches Google, that a failure cannot cost a
+// page view. What it cannot prove is delivery — that needs a real property, a real
+// Measurement Protocol secret and a network, none of which belong in a gate that
+// runs at pre-commit and on every pull request.
+//
+// This is that other half, and it is deliberately opt-in:
+//
+//   export GA4_MP_MEASUREMENT_ID='G-…'   # the AGENT property, not the site's
+//   export GA4_MP_API_SECRET='…'         # from that property's own data stream
+//   npm run probe:agent-analytics
+//
+// It calls the same lib/agent-analytics.js the Cloudflare middleware calls, so a pass
+// here means the module, the credential and the property agree — not that a
+// hand-written payload happens to be valid.
+//
+// GA4_MP_DEBUG=1 routes to GA4's validation endpoint instead, which reports what is
+// wrong with a payload but RECORDS NOTHING. Useful once; useless as a habit.
+//
+// Prints no secret. The value never appears in output, and never in an error message.
+const fs = require('node:fs');
+const path = require('node:path');
+
+const ROOT = path.resolve(__dirname, '..');
+
+// The site's own GA4 property, read from where it actually lives rather than copied
+// here. Sending crawler events to it is the one mistake with no undo — GA4 has no
+// selective delete — so the probe refuses by default. Deriving it means this cannot
+// go stale when the site's id changes; if the read fails, the check is skipped rather
+// than guessed at.
+function sitePropertyIds() {
+  try {
+    const cfg = fs.readFileSync(path.join(ROOT, 'eleventy.config.js'), 'utf8');
+
+    return new Set([...cfg.matchAll(/ga4:\s*"(G-[A-Z0-9]+)"/g)].map((m) => m[1]));
+  } catch {
+    return new Set();
+  }
+}
+
+const CASES = [
+  ['GPTBot/1.2 (+https://openai.com/gptbot)',      '/llms.txt',                  200],
+  ['ClaudeBot/1.0 (+claudebot@anthropic.com)',     '/tutorial/index.md',         200],
+  ['PerplexityBot/1.0',                            '/probe-missing/index.md',    404],
+];
+
+async function main() {
+  const force = process.argv.includes('--force');
+  const env = {
+    GA4_MP_MEASUREMENT_ID: process.env.GA4_MP_MEASUREMENT_ID,
+    GA4_MP_API_SECRET: process.env.GA4_MP_API_SECRET,
+    GA4_MP_DEBUG: process.env.GA4_MP_DEBUG,
+  };
+
+  if (!env.GA4_MP_MEASUREMENT_ID || !env.GA4_MP_API_SECRET) {
+    console.error(
+      'Both GA4_MP_MEASUREMENT_ID and GA4_MP_API_SECRET must be EXPORTED in this shell.\n'
+      + '  ID:     ' + (env.GA4_MP_MEASUREMENT_ID || '<not visible to child processes>') + '\n'
+      + '  SECRET: ' + (env.GA4_MP_API_SECRET ? 'visible' : '<not visible to child processes>') + '\n\n'
+      + 'Note `echo` finds a plain `FOO=bar` assignment but no child process does — only\n'
+      + 'exported variables are handed to one. If echo shows them but this does not:\n'
+      + '  export GA4_MP_MEASUREMENT_ID GA4_MP_API_SECRET',
+    );
+    process.exit(1);
+  }
+
+  const siteIds = sitePropertyIds();
+
+  if (siteIds.has(env.GA4_MP_MEASUREMENT_ID) && !force) {
+    console.error(
+      `Refusing: ${env.GA4_MP_MEASUREMENT_ID} is the property the SITE reports to `
+      + '(eleventy.config.js).\nCrawler events there mix into the numbers that describe '
+      + 'humans, and GA4 has no\nselective delete. Point this at the agent property, '
+      + 'or pass --force if you mean it.',
+    );
+    process.exit(1);
+  }
+
+  const { trackRequest, buildEvent } = await import('../lib/agent-analytics.js');
+  const debug = Boolean(env.GA4_MP_DEBUG);
+
+  console.log(`property: ${env.GA4_MP_MEASUREMENT_ID}${debug ? '  (GA4_MP_DEBUG — validating, NOT recording)' : ''}\n`);
+
+  for (const [userAgent, pathname, status] of CASES) {
+    const url = new URL(`https://imqueue.org${pathname}`);
+    const { params } = buildEvent({ url, userAgent, status, edition: 'org' }).events[0];
+
+    await trackRequest({
+      request: { headers: { get: (h) => (h.toLowerCase() === 'user-agent' ? userAgent : null) } },
+      env,
+      url,
+      status,
+      edition: 'org',
+    });
+
+    console.log(`  sent  ${params.crawler.padEnd(15)} ${params.surface.padEnd(16)} ${params.status}  ${pathname}`);
+  }
+
+  console.log(
+    debug
+      ? '\nValidation output is above: an empty validationMessages array means the payload'
+        + '\nis well-formed. Nothing was recorded — unset GA4_MP_DEBUG and re-run to send.'
+      : '\nDone. GA4 answers 204 to valid and invalid hits alike, so no error here proves'
+        + '\nnothing — the proof is the data. Open GA4 → Reports → Realtime on that property;'
+        + '\nthree page_view events should appear within a minute or two, including the 404.'
+        + '\nIf Realtime stays empty: wrong property, a secret from a different stream, or'
+        + '\nGA4 filtering. Re-run with GA4_MP_DEBUG=1 to see what it says about the payload.',
+  );
+}
+
+main().catch((err) => {
+  // Never interpolate the error blindly — a fetch failure can echo the request URL,
+  // and that URL carries api_secret as a query parameter.
+  console.error(`\nFAIL  ${String(err.message).replace(/api_secret=[^&\s]+/g, 'api_secret=<redacted>')}`);
+  process.exit(1);
+});