feat: experimental traffic analysis#2848
Conversation
🦋 Changeset detectedLatest commit: 59a5bff The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
sobanieca-redocly
changed the title
Coverage Report
File Coverage
|
||||||||||||||||||||||||||||||||||||||||||||
Performance Benchmark (Lower is Faster)
|
|
@cursor review |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
|
@cursor review |
|
@cursor review |
|
@cursor review |
|
@cursor review |
|
@cursor review |
|
@cursor review |
|
|
||
| - Starts a reverse proxy that forwards every request to an upstream `--target` | ||
| and records each request/response exchange into a HAR file. | ||
| - The HAR file is written incrementally (after each exchange) and flushed on |
There was a problem hiding this comment.
As mentioned during demo - do we want to keep this behavior for the first release? (this means proxy will be usable only for short lived processes like e2e tests etc.). In future we can introduce chunking (and give drift command option to read whole folder of traffic files). Eventually we can switch to ndjson format and append line by line.
| - `--api <path>`: OpenAPI file or folder to validate against. Omit to generate. | ||
| - `--traffic-format <auto|har|kong|nginx-json|apache-json|ndjson>` (default: `auto`) | ||
| - `--format <pretty|json|csv|sarif>` (default: `pretty`) | ||
| - `--match-mode <strict-host|basepath>` (default: `strict-host`): how requests are located |
There was a problem hiding this comment.
Since we add server option which seems more flexible maybe we should get rid of match-mode param at all? That way we will always use strict-host so we validate against OAD, but user may also override server.
| Mutually exclusive with `--match-mode`: use `--match-mode` when the traffic URLs align | ||
| with the description `servers`, use `--server` to declare the actual server when they | ||
| do not. | ||
|
|
There was a problem hiding this comment.
As presented during demo - shall we include ignore option for the first release? To avoid messages about unknown headers like x-caddy-authtoken etc.
| 'packages/**/__tests__/**/*', | ||
| 'packages/cli/src/index.ts', | ||
| 'packages/cli/src/utils/assert-node-version.ts', | ||
| 'packages/cli/src/commands/drift/**', |
There was a problem hiding this comment.
Is it fine if we ignore coverage for experimental commands? I would expect that source code there may change significantly over time after getting feedback on usage....
| - missing required parameters/body, | ||
| - request/response schema mismatches, | ||
| - baseline security issues (opt-in OWASP API risk heuristics). | ||
| - When no spec is provided, infers an OpenAPI 3.1 description from the traffic. |
There was a problem hiding this comment.
Do we need to improve OAD generation option for the first release? Maybe we should explicitly require path to folder where OAD will be saved and implement more sophisticated schemas generation?
On the other hand - we may wait for feedback and do this in next iteration.
There was a problem hiding this comment.
As discussed in Slack - we will remove OAD generation in this PR.
| maxFindings?: number; | ||
| } | ||
|
|
||
| const ANSI = { |
There was a problem hiding this comment.
we have colorette as a dep in this repo, let's use it
| import { handleDrift, type DriftArgv } from './commands/drift/index.js'; | ||
| import type { FindingSeverity, MatchMode, TrafficFormat } from './commands/drift/types/index.js'; |
| yargs | ||
| .env('REDOCLY_CLI_DRIFT') | ||
| .positional('traffic', { | ||
| describe: 'Path to a traffic log file or folder (HAR, Kong, Nginx/Apache JSON, NDJSON).', |
There was a problem hiding this comment.
does it support those other formats?
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 0e8c763. Configure here.
Co-authored-by: Jacek Łękawa <164185257+JLekawa@users.noreply.github.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 59a5bff. Configure here.
|
|
||
| export function normalizeServerPrefix(server: string | undefined): string | undefined { | ||
| const trimmed = server?.replace(/\/+$/, ''); | ||
| return trimmed || undefined; |
There was a problem hiding this comment.
Root path server override dropped
Medium Severity
Passing --server / as a path-only prefix is treated as no server override because trailing slashes are stripped and an empty remainder becomes undefined. Drift then falls back to description servers matching instead of path-only filtering, which breaks the documented path-prefix mode for the site root.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 59a5bff. Configure here.
3 participants
What/Why/How?
Added two new (experimental) commands -
driftandproxyfor collecting traffic data and validating it against provided OpenAPI definition.Reference
Testing
Screenshots (optional)
Check yourself
Security
Note
Medium Risk
Large new surface area including security/CORS/cookie heuristics and a live forwarding proxy; behavior is experimental but misconfiguration could leak or mishandle captured credentials in HAR files.
Overview
Adds experimental
driftandproxyCLI commands (minor@redocly/clichangesets) to compare real HTTP traffic against OpenAPI descriptions.driftingests traffic from files or folders (HAR, Kong, Nginx/Apache JSON, NDJSON, auto-detect, plus pluggable traffic parsers), bundles and indexes OpenAPI via existing@redocly/openapi-core, matches each exchange to operations (--match-modeor--server), and runs builtin/plugin rules. Findings roll up in memory and export as pretty, json, csv, or sarif; exit code 1 on error-level drift. Builtin rules cover undocumented endpoints, schema/parameter consistency (AJV with readOnly/writeOnly handling), security-baseline (documentedsecurityvs traffic), and opt-in OWASP API heuristics.proxyruns a local reverse proxy (undiciupstream), appends exchanges to a HAR on each request, and optionally reuses the sameValidationSessionfor live findings plus a shutdown report. Captured HARs are intended for replay withdrift.CLI wiring lazy-loads both commands;
lintconfig output skips mixing in when format is csv or sarif. E2E tests and READMEs document flags and PoC limits (in-memory JSON arrays, full HAR rewrite on proxy).Reviewed by Cursor Bugbot for commit fe177c4. Bugbot is set up for automated code reviews on this repo. Configure here.