Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- **Rate-limit counters can be shared.** `RateLimitRule` hard-constructed an in-memory `Map` with no seam to replace it, so on any deployment with more than one process the limit was effectively `max × instances` — and on Vercel or Lambda it reset on every cold start. `rateLimit({ store })` now takes a `RateLimitStore`.
- `upstashRateLimitStore({ url, token })` ships in the core package. Upstash speaks Redis over HTTP, which is the only shape that works on Vercel Edge, Workers and Deno, where an ordinary client cannot open a socket. It calls the REST API with `fetch` rather than depending on `@upstash/redis`.
- Fails open by default when Redis is unreachable; `onError: 'closed'` denies instead. Either way the outcome is visible in `decision.results`.
- `Rule` gained an optional `prepare(context)` that `protect()` awaits before evaluation — the same pre-fetch already used for IP enrichment and Web Bot Auth. `evaluate()` stays synchronous, so the default in-memory path is unchanged and allocation-free.
- The synchronous `evaluateRules()` cannot consume a networked store and now reports `NOT_RUN` for such a rule, rather than allowing silently. A rate limiter that has quietly stopped limiting looks identical to one that is working.

- **`protect()` returns a typed decision.** It used to return `{ allowed, detection }`, and the adapters typed the value handed to `onBlocked` as `any`.
- `conclusion: 'ALLOW' | 'DENY' | 'CHALLENGE' | 'ERROR'`, with `isAllowed()` / `isDenied()` / `isChallenged()` / `isErrored()` and `deniedBy(rule)`. `ERROR` is a distinct conclusion, so a caller can tell "allowed" from "never decided" — both still serve the request.
- `results` — every configured rule in evaluation order with a `state` of `RUN`, `DRY_RUN`, `NOT_RUN` or `CACHED`. `NOT_RUN` is new information: a `filter()` rule with no IP enrichment, or a `webBotAuth()` rule on a request with no host, used to report ALLOW, which reads as "checked and fine" rather than "never checked". A dry-run rule that matched now reports `conclusion: 'DENY'` with `state: 'DRY_RUN'`, rather than the ALLOW its action said.
Expand Down
36 changes: 35 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ const wd = new WebDecoy({
});
```

- **`rateLimit({ max, window, algorithm?, action?, keyBy? })`** — fixed or sliding window, keyed by IP (or a custom function). No key.
- **`rateLimit({ max, window, algorithm?, action?, keyBy?, store? })`** — fixed or sliding window, keyed by IP (or a custom function). No key. See [shared rate limits](#rate-limits-across-more-than-one-process) before you run two replicas.
- **`tripwire({ paths?, prefixes?, patterns?, includeDefaults? })`** — deterministic honeypot-path detection. No key.
- **`webBotAuth({ onImpersonation?, onClaimed?, allowCategories? })`** — verify AI-agent signatures (Web Bot Auth / RFC 9421) locally; deny impersonators of known agents. No key.
- **`filter({ expression, action? })`** — an expression language over IP reputation/geo (e.g. `ip.tor`, `ip.country in ["CN", "RU"]`). Requires an API key for enrichment.
Expand Down Expand Up @@ -204,6 +204,40 @@ if (!result.allowed) {
| [@webdecoy/nextjs](https://www.npmjs.com/package/@webdecoy/nextjs) | [![npm](https://img.shields.io/npm/v/@webdecoy/nextjs.svg)](https://www.npmjs.com/package/@webdecoy/nextjs) | Next.js middleware |
| [@webdecoy/client](https://www.npmjs.com/package/@webdecoy/client) | [![npm](https://img.shields.io/npm/v/@webdecoy/client.svg)](https://www.npmjs.com/package/@webdecoy/client) | Browser-side signal collector |

## Rate limits across more than one process

`rateLimit()` counts in this process's memory by default. That is correct for a
single process and wrong the moment you run two: the effective limit becomes
`max × instances`, and on Vercel or Lambda it also resets on every cold start.

Point the rule at a shared store to make one limit one limit:

```typescript
import { rateLimit, upstashRateLimitStore } from '@webdecoy/node';

const store = upstashRateLimitStore({
url: process.env.UPSTASH_REDIS_REST_URL!,
token: process.env.UPSTASH_REDIS_REST_TOKEN!,
});

const wd = new WebDecoy({
rules: [rateLimit({ max: 100, window: 60, store })],
});
```

Upstash speaks Redis over HTTP, so this works on Vercel Edge, Cloudflare Workers
and Deno, where an ordinary Redis client cannot open a socket. It uses `fetch`
directly — no `@upstash/redis` dependency.

If Redis is unreachable the store **fails open** and the request is allowed; pass
`onError: 'closed'` to deny instead. Either way the outcome appears in
`decision.results`, so it does not look like a normal evaluation.

Any object implementing `RateLimitStore` works. A store that returns promises is
consumed during `protect()`'s async pre-fetch; a `sync` store is consumed inline.
The synchronous `evaluateRules()` cannot consume a networked store, and reports
`NOT_RUN` rather than allowing silently.

## Client IP behind a proxy

Rate limits, IP enrichment and every detection we record are keyed on the caller's address, so it matters that the address is real. `X-Forwarded-For` is written by the client for the first hop — the leftmost value in it is whatever the caller decided to send — so the middleware believes it only as far as you say it should, counted from the **right** of the chain, which is the end your own infrastructure wrote.
Expand Down
8 changes: 8 additions & 0 deletions packages/webdecoy/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,9 @@ export {
injectHoneytokenLink,
isInjectableHtml,
HONEYTOKEN_BASE_PATH,
MemoryRateLimitStore,
upstashRateLimitStore,
UpstashRateLimitStore,
} from './rules';

export type {
Expand All @@ -118,6 +121,11 @@ export type {
HoneytokenLinkProps,
ViolationEvent,
IPEnrichmentData,
RateLimitStore,
SyncRateLimitStore,
RateLimitOutcome,
RateLimitConsume,
UpstashStoreOptions,
} from './rules';

// Local Web Bot Auth verification (RFC 9421, tag "web-bot-auth")
Expand Down
9 changes: 9 additions & 0 deletions packages/webdecoy/src/rules/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ export { BotRule } from './bot-rule';
export { WebBotAuthRule, webBotAuth } from './web-bot-auth-rule';
export { honeytoken } from './honeytoken';
export { InMemoryRateLimiter } from './rate-limiter';
export { MemoryRateLimitStore } from './rate-limit-store';
export { upstashRateLimitStore, UpstashRateLimitStore } from './upstash-store';

export type {
Rule,
Expand All @@ -25,6 +27,13 @@ export type {
} from './types';
export type { WebBotAuthConfig } from './web-bot-auth-rule';
export type { HoneytokenOptions, Honeytoken } from './honeytoken';
export type {
RateLimitStore,
SyncRateLimitStore,
RateLimitOutcome,
RateLimitConsume,
} from './rate-limit-store';
export type { UpstashStoreOptions } from './upstash-store';

import { RateLimitRule } from './rate-limit-rule';
import { FilterRule } from './filter-rule';
Expand Down
65 changes: 52 additions & 13 deletions packages/webdecoy/src/rules/rate-limit-rule.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,21 @@
* Implements the Rule interface using InMemoryRateLimiter
*/

import { InMemoryRateLimiter } from './rate-limiter';
import { Rule, RuleContext, RuleResult, RateLimitConfig } from './types';
import { MemoryRateLimitStore } from './rate-limit-store';
import type { RateLimitStore, RateLimitOutcome, RateLimitConsume } from './rate-limit-store';

export class RateLimitRule implements Rule {
readonly name: string;
private limiter: InMemoryRateLimiter;
private store: RateLimitStore;
private config: Required<
Pick<RateLimitConfig, 'max' | 'window' | 'algorithm' | 'action' | 'dryRun'>
> &
Pick<RateLimitConfig, 'keyBy'>;

constructor(config: RateLimitConfig) {
this.name = `rate-limit:${config.max}/${config.window}s`;
this.limiter = new InMemoryRateLimiter();
this.store = config.store ?? new MemoryRateLimitStore();
this.config = {
max: config.max,
window: config.window,
Expand All @@ -27,17 +28,55 @@ export class RateLimitRule implements Rule {
};
}

/**
* Which bucket this request counts against.
*
* Precedence: this rule's own keyBy, then the SDK-wide characteristics, then
* the IP. `context.key` is always populated, so the last fallback only matters
* for a context built by hand.
*/
private keyFor(context: RuleContext): string {
return this.config.keyBy ? this.config.keyBy(context) : (context.key ?? context.ip);
}

private consumption(context: RuleContext): RateLimitConsume {
return {
key: this.keyFor(context),
max: this.config.max,
windowMs: this.config.window * 1000,
algorithm: this.config.algorithm,
};
}

/** Consume from a networked store before evaluation. No-op for a sync store. */
async prepare(context: RuleContext): Promise<void> {
if (this.store.sync) return;
const outcome = await this.store.consume(this.consumption(context));
context.prepared ??= {};
context.prepared[this.name] = outcome;
}

evaluate(context: RuleContext): RuleResult {
// Precedence: this rule's own keyBy, then the SDK-wide characteristics,
// then the IP. `context.key` is always populated, so the last fallback only
// matters for a context built by hand.
const key = this.config.keyBy ? this.config.keyBy(context) : (context.key ?? context.ip);
const windowMs = this.config.window * 1000;
let result: RateLimitOutcome;

const result =
this.config.algorithm === 'sliding'
? this.limiter.checkSlidingWindow(key, this.config.max, windowMs)
: this.limiter.checkFixedWindow(key, this.config.max, windowMs);
if (this.store.sync) {
result = this.store.consume(this.consumption(context)) as RateLimitOutcome;
} else {
const prepared = context.prepared?.[this.name] as RateLimitOutcome | undefined;
if (!prepared) {
// A networked store that was never consumed. Saying so beats allowing
// silently: a rate limiter that has quietly stopped limiting looks
// identical to one that is working.
return {
action: 'ALLOW',
rule: this.name,
state: 'NOT_RUN',
reason:
'Rate limit uses an async store and was not prepared — call protect() or evaluateRulesAsync()',
};
}
result = prepared;
}

if (!result.allowed) {
const retryAfter = Math.ceil((result.resetAt - Date.now()) / 1000);
Expand Down Expand Up @@ -68,6 +107,6 @@ export class RateLimitRule implements Rule {
}

destroy(): void {
this.limiter.destroy();
void this.store.destroy?.();
}
}
Loading
Loading