Skip to content

[codex] add parity matrix trend gate#4286

Draft
andrewtdiz wants to merge 1 commit into
PerryTS:mainfrom
andrewtdiz:codex/node-compat-parity-matrix-trend
Draft

[codex] add parity matrix trend gate#4286
andrewtdiz wants to merge 1 commit into
PerryTS:mainfrom
andrewtdiz:codex/node-compat-parity-matrix-trend

Conversation

@andrewtdiz
Copy link
Copy Markdown
Contributor

Linked issues

Behavior cluster summary

This adds the CI/reporting slice for the Node parity matrix:

  • run_parity_tests.sh now emits compact per-test results records in latest.json.
  • scripts/parity_matrix_trend.py builds a per-module matrix for top-level test_parity_<module> runs, including status, known-failure state, Node/Perry line counts, and unified-diff changed-line counts.
  • test-parity/parity_matrix_baseline.json seeds the current module-inventory gaps while treating any absent module as must-pass with zero diff lines.
  • The existing parity CI job now checks the matrix, writes JSON/Markdown artifacts, and appends the table to the Actions step summary.

Why these changes are batched

The report gate needs all three pieces to be useful: the runner has to expose per-test status, the checker has to convert that into a module trend, and CI has to publish/fail on the artifact. Keeping them together avoids a dead script or an unused report field.

Tests and checks

  • python3 -m unittest tests.test_parity_matrix_trend
  • bash -n run_parity_tests.sh
  • python3 -m py_compile scripts/parity_matrix_trend.py tests/test_parity_matrix_trend.py
  • jq empty test-parity/parity_matrix_baseline.json
  • local YAML parse of .github/workflows/test.yml
  • ./scripts/check_file_size.sh
  • cargo fmt --all -- --check
  • git diff --check

Known limitations

  • I did not run the full parity suite locally because an unrelated cargo build --release -p perry -p perry-runtime -p perry-stdlib job was already active in the shared workspace; starting another full parity run would have kicked off a competing release build.
  • Existing module-inventory entries use max_diff_lines: null until a full CI/release parity run records exact diff baselines. New untriaged module failures still fail immediately.

Non-goals

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant