Context
JS Crawlee Router supports .use() middleware — functions that run sequentially before every matched route handler. Python Router has no equivalent. Users cannot add cross-cutting concerns (logging, auth, rate limiting) at the router level.
JS implementation
packages/core/src/router.ts
router.use(middleware: (ctx: Context) => Awaitable<void>): void
- Middlewares stored in a private array, executed in registration order
- All run before the matched handler, sharing the same context object
- No early-exit /
next() pattern — just sequential execution, throwing stops the chain
Dispatch flow (router.ts L193–L202):
for (const middleware of router.middlewares) {
await middleware(context);
}
return router.getHandler(label)(context);
Tests: test/core/router.test.ts
Current Python state
src/crawlee/router.py — simple label-based dispatcher with default_handler and handler(label) decorators, no middleware support. The ContextPipeline provides crawler-level middleware, but not at the router level.
What to implement
Add a use() method to Router that registers pre-handler middleware, matching JS semantics.
Context
JS Crawlee Router supports
.use()middleware — functions that run sequentially before every matched route handler. Python Router has no equivalent. Users cannot add cross-cutting concerns (logging, auth, rate limiting) at the router level.JS implementation
packages/core/src/router.tsnext()pattern — just sequential execution, throwing stops the chainDispatch flow (
router.tsL193–L202):Tests:
test/core/router.test.tsCurrent Python state
src/crawlee/router.py— simple label-based dispatcher withdefault_handlerandhandler(label)decorators, no middleware support. TheContextPipelineprovides crawler-level middleware, but not at the router level.What to implement
Add a
use()method toRouterthat registers pre-handler middleware, matching JS semantics.