Skip to content

run the gnu testsuite in the ci#8

Merged
sylvestre merged 3 commits into
mainfrom
gnu-test
May 29, 2026
Merged

run the gnu testsuite in the ci#8
sylvestre merged 3 commits into
mainfrom
gnu-test

Conversation

@sylvestre
Copy link
Copy Markdown
Contributor

No description provided.

sylvestre added 3 commits May 29, 2026 18:43
Add util/fetch-gnu.sh (downloads the GNU grep 3.12 release tarball from
ftp.gnu.org) and util/run-gnu-testsuite.sh, which reuses the gnulib test
framework shipped in the tarball (tests/init.sh + init.cfg) and injects the
Rust grep binary via PATH, replicating tests/Makefile.am's TESTS_ENVIRONMENT.
Each test is classified by its gnulib exit code (0=PASS, 77=SKIP, else FAIL)
and results are emitted as JSON.

Modelled on ../sed (lightweight PATH-injection runner) and ../coreutils
(release-tarball fetch). Current baseline: 61 pass / 39 fail / 28 skip of 128
tests -- the failures quantify the remaining GNU-compatibility gap.
Add a GnuTests workflow that, on push and PR, fetches the GNU grep release
tarball, builds the Rust grep binary, runs util/run-gnu-testsuite.sh, and
uploads the JSON results. An aggregate job compares the run against the
reference summary from the default branch (util/compare_test_results.py,
borrowed from ../sed) and fails only on new, non-intermittent regressions;
known-flaky tests are listed in .github/workflows/ignore-intermittent.txt.
Add a GnuComment workflow that runs after GnuTests completes on a pull
request, downloads the 'comment' artifact (PR number + comparison text), and
posts it as a PR comment. Mirrors ../sed's GnuComment workflow.
@codecov
Copy link
Copy Markdown

codecov Bot commented May 29, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 0.00%. Comparing base (79db36e) to head (e885f66).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@          Coverage Diff          @@
##            main      #8   +/-   ##
=====================================
  Coverage   0.00%   0.00%           
=====================================
  Files          6       6           
  Lines       1404    1404           
  Branches     139     139           
=====================================
  Misses      1404    1404           
Flag Coverage Δ
macOS_latest 0.00% <ø> (ø)
ubuntu_latest 0.00% <ø> (ø)
windows_latest 0.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sylvestre sylvestre merged commit f781350 into main May 29, 2026
16 checks passed
@sylvestre sylvestre deleted the gnu-test branch May 30, 2026 17:16
@sylvestre
Copy link
Copy Markdown
Contributor Author

and it works:
#11 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant