Skip to content

feat(sdk): rate-limit counters can live somewhere shared - #29

Merged
cport1 merged 1 commit into
mainfrom
feat/rate-limit-store
Aug 22, 2026
Merged

feat(sdk): rate-limit counters can live somewhere shared#29
cport1 merged 1 commit into
mainfrom
feat/rate-limit-store

Conversation

@cport1

@cport1 cport1 commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Closes #726. Stacked on #28 (typed decision) — the NOT_RUN state it reports comes from there.

The bug

rate-limit-rule.ts:19 hard-constructed new InMemoryRateLimiter() — a Map, with no seam to replace it. On any deployment with more than one process the limit is effectively max × instances, and on Vercel or Lambda it also resets on every cold start. rateLimit({ max: 100, window: 60 }) across eight replicas is an 800/min limit that an autoscaler can raise for you.

This was inconsistent with the rest of the SDK: detection/stores.ts and the captcha's challenge and token stores were already behind swappable interfaces with in-memory defaults. The one piece of state that actually has to be shared was the only one that could not be.

The sync/async problem

Rule.evaluate() is synchronous. Making it async would turn every rule evaluation into a promise for the sake of the one rule that might need it, and the in-memory path — which is most installs — would pay for a network store nobody configured.

So a store declares which it is:

  • Sync store → consumed inline during evaluate(). The default in-memory path, unchanged and allocation-free.
  • Async store → consumed in a new optional Rule.prepare(context), which protect() awaits before evaluation. This is the pre-fetch pattern the SDK already uses for IP enrichment and Web Bot Auth verdicts, so it fits the existing architecture rather than inventing a second one.

A rule whose async store was never prepared reports NOT_RUN, not ALLOW. The synchronous evaluateRules() cannot consume a networked store, and a rate limiter that has quietly stopped limiting is indistinguishable from one that is working — that is the failure mode worth being loud about.

The Upstash store

Ships in the core package, not a sixth workspace, and calls the REST API with fetch rather than depending on @upstash/redis. Two commands in a pipeline is not worth a dependency, a version to track, or a transitive node: import finding its way into an edge bundle. check:edge covers it automatically.

Upstash is the right first target because it speaks Redis over HTTP: Vercel Edge, Workers and Deno have no node:net, so an ordinary Redis client cannot open a socket in exactly the serverless deployments that need shared counters most.

Two details worth reviewing:

  • Fixed window puts the window id in the key (wd:rl:f:<key>:<windowStart>), so expiry is the only cleanup needed and two processes cannot disagree about which window they are in. EXPIRE … NX, or a long window gets extended into a sliding one by later requests.
  • Sliding window gives each request a random member suffix (<now>-<rand>). Two requests in the same millisecond would otherwise be one ZADD that overwrites rather than two that count — undercounting exactly when the limit matters.

Fails open by default when Redis is unreachable, because a rate limiter that takes the site down when its datastore has a bad minute has done more damage than the traffic it was shaping. onError: 'closed' for limits protecting something more expensive than availability. Either way the outcome lands in decision.results rather than looking like a normal evaluation.

Verification

14 new tests, 341 total. The one that matters: two WebDecoy instances pointed at one store enforce max: 2 as two, where before each had its own Map and it was four. Also asserts exactly one consume() per request — a second consume in evaluate() after prepare() would double-count and halve every limit. Build, lint and check:edge green.

@cport1
cport1 changed the base branch from feat/typed-decision to main August 22, 2026 02:14
RateLimitRule hard-constructed an InMemoryRateLimiter -- a Map, with no
seam to replace it. On any deployment with more than one process the limit
was effectively max x instances, and on Vercel or Lambda it also reset on
every cold start. rateLimit({ max: 100, window: 60 }) across eight
replicas is an 800/min limit that an autoscaler can raise for you.

That was inconsistent with the rest of the SDK: the detection stores and
the captcha's challenge and token stores were already behind swappable
interfaces. The one piece of state that actually has to be shared was the
only one that could not be.

evaluate() is synchronous and making it async would turn every rule
evaluation into a promise for the sake of the one rule that needs it, so a
store declares which it is. A sync store is consumed inline -- the default
in-memory path, unchanged. An async store is consumed in a new optional
Rule.prepare(), which protect() awaits, the same pre-fetch already used
for IP enrichment and Web Bot Auth verdicts.

A rule whose async store was never prepared reports NOT_RUN rather than
allowing. A rate limiter that has quietly stopped limiting is
indistinguishable from one that works, which is the worst of the options.

The Upstash store lives in core and uses fetch, not @upstash/redis: two
commands in a pipeline is not worth a dependency, a version to track, or a
transitive node: import finding its way into an edge bundle. Fixed windows
put the window id in the key so expiry is the only cleanup and two
processes cannot disagree about which window they are in; sliding windows
give each request a random member suffix, because two requests in the same
millisecond would otherwise be one ZADD that overwrites rather than two
that count.

Closes WebDecoy/app#726
@cport1
cport1 force-pushed the feat/rate-limit-store branch 2 times, most recently from 014e46a to fba9c9a Compare August 22, 2026 02:14
@cport1
cport1 merged commit fd75118 into main Aug 22, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant