Fix AWS CLI man-page help parsing

[thomhurst]

Summary

• normalize groff/man-page backspace overstrikes before CLI help parsing
• add an AWS CLI 2.36.29 root-help fixture reproducing the current +\bo service bullets
• verify the fixture through the full AWS ScrapeAsync traversal

Test plan

• AWS scraper tests (4 passed)
• OptionsGenerator tests (817 passed)
• OptionsGenerator Release build (0 warnings, 0 errors)
• Changed-file formatting

Full solution dotnet format --verify-no-changes --severity info also reports pre-existing findings outside this change, including IHelpTextCache.cs whitespace and AzureCliHelpTypeDetector.cs style errors.

Closes #3986

Summary by CodeRabbit

• Bug Fixes

  • Improved command-line help parsing by removing formatting artifacts commonly found in man-page output.
  • Increased reliability when extracting available AWS CLI services from help text.

• Tests

  • Added coverage for AWS CLI version 2.36.29 help output.
  • Added test fixture handling to ensure CLI help data is available during test execution.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ca26777e-a637-4b8d-bd66-c9fbaa90743e

📥 Commits

Reviewing files that changed from the base of the PR and between ee067d0 and dc61771.

📒 Files selected for processing (4)

• tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Fixtures/AwsCli/aws-2.36.29-root-help.json
• tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/ModularPipelines.OptionsGenerator.Tests.csproj
• tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Scrapers/AwsCliScraperTests.cs
• tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator/Scrapers/Cli/CliScraperBase.cs

Included review availability: Your plan provides up to 4 included reviews per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The AWS CLI scraper now removes man-page overstrike sequences from help text. A fixture-backed test validates extraction of three services from AWS CLI 2.36.29 output.

Changes

AWS CLI scraping

Layer / File(s)	Summary
Normalize AWS CLI help text tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator/Scrapers/Cli/CliScraperBase.cs	NormalizeHelpText now removes ANSI escape sequences and man-page overstrike sequences.
Validate AWS service extraction tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Fixtures/AwsCli/*, tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/ModularPipelines.OptionsGenerator.Tests.csproj, tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Scrapers/AwsCliScraperTests.cs	The test project copies fixture files to its output directory. A fixture-backed test uses AWS CLI 2.36.29 help output and verifies discovery of three services through AwsFixtureExecutor.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to dc617

This localized parsing fix is merge-ready after normal checks and review; no actionable merge-blocking risk remains.

Poem

I’m a rabbit with a fixture to share,
Three AWS services now appear there.
Overstrikes vanish from help in a flash,
Tests catch format changes before they clash.
Hop, hop—the scraper parses with care.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main change: fixing AWS CLI man-page help parsing.
Linked Issues check	✅ Passed	The changes normalize overstrikes and add a pinned AWS CLI fixture test that verifies service extraction, meeting issue #3986 objectives.
Out of Scope Changes check	✅ Passed	All changes support AWS help parsing, fixture loading, or regression testing for issue #3986.
Docstring Coverage	✅ Passed	Docstring check was indeterminate for this PR — some files could not be analyzed in time. Not blocking.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch issue-3986-aws-help-parser

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Greptile Summary

This PR normalizes groff backspace overstrikes before shared CLI help parsing and adds an AWS CLI 2.36.29 regression fixture.

• Removes ANSI sequences and man-page overstrike prefixes before scraper parsers consume help text.
• Exercises AWS root-help discovery through the complete ScrapeAsync traversal.
• Copies test fixtures into the test output directory while preserving their relative paths.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator/Scrapers/Cli/CliScraperBase.cs	Extends shared help-text normalization to remove groff backspace overstrike prefixes before parsing.
tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Scrapers/AwsCliScraperTests.cs	Adds an end-to-end AWS scraper regression test using recorded root-help output.
tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/ModularPipelines.OptionsGenerator.Tests.csproj	Copies fixture files to the test output directory at the relative path expected by tests.
tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Fixtures/AwsCli/aws-2.36.29-root-help.json	Records AWS CLI 2.36.29 help containing the groff overstrike service bullets addressed by the fix.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[AWS CLI help output] --> B[Strip ANSI sequences]
  B --> C[Remove groff overstrike prefixes]
  C --> D[Extract AWS services]
  D --> E[Traverse discovered commands]
  E --> F[CliCommandDefinition results]

Loading

_{Reviews (6): Last reviewed commit: "fix(options): normalize man-page overstr..." | Re-trigger Greptile}

[claude]

Code review

Reviewed the diff (CliScraperBase.cs, AwsCliScraper.cs fixture/test, csproj).

Summary: This fixes AWS CLI service-list parsing when help output comes through man/groff and contains backspace overstrike sequences (e.g. +\bo accessanalyzer for the bullet glyph). The new ManPageOverstrikePattern (.\x08 → empty) strips the "character-then-backspace" pair, which correctly collapses both bold/underline overstrikes (_\bC → C) and the bullet overstrike in the fixture, leaving o accessanalyzer, which the existing AwsCommandPattern (^\s*(?:o|\*)\s+...) already knows how to parse.

Architecture note (positive): Landing the fix in NormalizeHelpText on the shared CliScraperBase rather than only in AwsCliScraper is the right call — it fixes the root cause once for every CLI scraper that might hit the same man-page rendering quirk, instead of a scraper-specific patch that would need to be re-applied elsewhere later.

Test coverage: The new fixture-driven test (Extracts_Services_From_Aws_2_36_29_Help_Fixture) follows the existing AwsHelpExecutor/ICliCommandExecutor test pattern already used in this file, exercises the real ScrapeAsync traversal rather than the regex in isolation, and pins the fixture to the exact CLI version that reproduced the bug — good for regression tracking.

No bugs or CLAUDE.md violations found in the diff. This is a well-scoped, well-tested fix.

[claude]

Code Review — PR #4005 (Fix AWS CLI man-page help parsing)

Verification performed: Built tools/ModularPipelines.OptionsGenerator/ModularPipelines.OptionsGenerator.slnx and ran the full ModularPipelines.OptionsGenerator.Tests project (817 tests, all passing, including the new Extracts_Services_From_Aws_2_36_29_Help_Fixture test). Hand-traced the new ManPageOverstrikePattern (.\x08) against the +\bo overstrike fixture to confirm it correctly collapses char+backspace+char sequences without corrupting adjacent bullet text.

Overall this is a small, well-tested, well-scoped fix — the regex change is minimal and the new fixture reproduces the real-world AWS CLI 2.36.29 output that motivated it. No correctness bugs found. Two low-severity, non-blocking notes:

1. Shared blast radius, single-source test coverage (CliScraperBase.cs:664) — NormalizeHelpText is shared by all ~48 CLI scrapers (Docker, Terraform, kubectl, gcloud, etc.), but the new overstrike-stripping regex is only exercised by a synthetic AWS fixture. If any other scraped CLI ever emits a literal backspace byte for a reason other than bold/underline overstrike (e.g. a stray control character from a spinner leaking into captured help text), the preceding character would be silently deleted with no error surfaced. Not a reason to block — just worth keeping in mind if another scraper's generated output ever looks subtly truncated.
2. Doc comment is now slightly stale (CliScraperBase.cs:657) — the XML doc on NormalizeHelpText describes ANSI-sequence removal but doesn't mention the new man-page overstrike handling, which could lead a future maintainer to duplicate the logic elsewhere instead of extending this method.

Nice fix — approving from a review standpoint modulo those two minor notes.

[claude]

Code review

Re-reviewed the current diff — it's unchanged from the version already reviewed in the earlier comment on this PR (same three files: CliScraperBase.cs, AwsCliScraperTests.cs fixture/test, and the test csproj; same commit content).

Confirms:

• ManPageOverstrikePattern (.\x08 → "") correctly implements the standard groff overstrike-collapse: for a single overstrike (_\bC → C) and for repeated-character bold (A\bA\bA), .NET's non-overlapping left-to-right Regex.Replace reduces the pair-at-a-time match down to the final character in each run, which is the expected outcome. No RegexOptions.Singleline/multiline needed since overstrikes never span a newline.
• Root-caused in the shared CliScraperBase.NormalizeHelpText rather than patched only in AwsCliScraper, so any other scraper hitting man-page rendering gets the same fix for free — good call over a scraper-local patch.
• The new fixture test exercises the real ScrapeAsync traversal (not just the regex in isolation) and is pinned to the CLI version that reproduced the bug, which is good for regression tracking.

No bugs or CLAUDE.md violations found. Nothing outstanding from the prior review needs addressing — this is a well-scoped, well-tested fix.

[thomhurst]

Rebased onto current main (13fd094). Validation: AWS scraper tests 2/2; OptionsGenerator Release build succeeded with 0 warnings/errors.

[claude]

Code review

Re-checked after the rebase onto current main (13fd094b) — the diff content is unchanged from the version already reviewed twice above (same three files: CliScraperBase.cs, AwsCliScraperTests.cs fixture/test, and the test .csproj; same commit 1d7b64bc). Confirmed there's no conflicting concurrent work on main's CliScraperBase.cs that the rebase would need to reconcile with.

No bugs or CLAUDE.md violations found. Nothing outstanding from the prior reviews needs addressing — this remains a well-scoped, well-tested fix.

[thomhurst]

Rebased onto current main after #4023/#3985 and force-pushed with an exact lease. Validation: AwsCliScraperTests passed 2/2; OptionsGenerator Release build succeeded with 0 warnings/errors; diff checks clean. Verified remote head: 5c5f318.