test: check flow metrics under bazel

[maliberty]

Summary

The bazel flow tests ran the full RTL-to-GDS flow and then checked only the
exit code. BIG_TESTS get check_log = False, and PASSFAIL_TESTS is just
commands_without_load, so check_passfail was False as well. The
<design>.metrics_limits files were read only by test/flow_metrics.tcl, which
only test/regression.tcl sources, so QoR was checked by ./regression and by
nothing else. A green bazel test //test:flow_tests said nothing about QoR.

This adds a check_metrics attribute to regression_rule_test. When set,
regression_test.sh passes -metrics so openroad writes the result json, then
compares it against <design>.metrics_limits and fails the test on a violation.
It is enabled for every BIG_TEST, since those produce no golden log and their
QoR is all there is to check. A new design must save a <design>.metrics_limits
file; without one the check fails with "missing metrics limits file" rather than
running the flow and asserting nothing.

The comparison reuses the existing metric table and margins through a new
compare_metrics_files, so bazel and ./regression apply identical limits and
print identical messages.

Reading the json without tcllib

compare_metrics_files cannot use tcllib's json package: the bazel-built
openroad links core tcl_lang and has no tcllib, so the first attempt failed
with can't find package json. Vendoring tcllib for this is far more than the
check is worth, and depending on the host's tcllib would make the test
non-hermetic.

So it reads the json with a new read_flat_json. The metrics and limits files
are machine written by utl::metric and save_metric_limits and are always one
flat object of string or number values, which is also all compare_metrics_files
can consume. Nested objects and arrays are rejected rather than guessed at, so a
format change fails loudly instead of comparing the wrong numbers.

read_flat_json returns dicts identical to json::json2dict for all 31 metrics
and limits files in the tree (keys and values, zero mismatches). It rejects
nested objects, arrays, a non-object top level, unterminated objects and strings,
unquoted keys, and a missing colon; it handles string escapes, exponent and
negative numbers, true/false/null, and empty objects. The other five
json::json2dict sites in flow_metrics.tcl are report and save paths that run
only under ./regression, where tcllib is present, so they are left alone.

Impact

Two commits: the first adds the mechanism, the second regenerates the limits it
found to be out of date.

Turning the check on surfaced four designs whose QoR was outside their limits.
Their limits had last been derived from cmake ./regression runs, but CI runs
the flow tests under bazel (jenkins-ci vars/pipelineOR.groovy gives each design
its own Bazel Flow Test <design> stage, naming //test:<design>-tcl_test
explicitly to bypass the manual tag), whose -O3 -flto binary produces
different QoR. So these had been drifting in CI behind a green tick.

design	before	after regeneration
aes_asap7	FAIL DRT::max_slew_slack -32% < -11%	pass
aes_nangate45	FAIL RSZ::hold_buffer_count 78 > 37	pass
aes_sky130hs	FAIL worst_slack_max, max_slew_slack, max_capacitance_slack	pass
jpeg_sky130hs	FAIL hold_buffer_count, clock_skew, max_capacitance_slack	pass

The limits are regenerated from the metrics json the check now writes as an
undeclared test output, so bazel runs are self-sufficient for this. All 22 gated
metrics remain in every limits file, so the gate is not weakened. The limits move
both ways: tighter where bazel is better (aes_nangate45
DRT::max_capacitance_slack -33% to -20%), looser where it is worse
(aes_asap7 DRT::max_slew_slack -11% to -38%).

gcd_sky130hd_fast_slow

This design was registered only in test/BUILD, not in regression_tests.tcl or
CMakeLists.txt. CI gave it its own bazel flow stage, but with check_log and
check_passfail both false and no limits file it asserted nothing beyond openroad
exiting 0 — the one flow design with no output checked at all.

It had no limits file because the tooling could not produce one:
save_metric_limits indexes test_langs($test), so save_flow_metrics_limits
answered "test not found". Adding it to record_flow_tests lets the tooling
generate its limits and lets ./regression run it like the other designs. Its
metrics and limits come from a bazel run, and it agrees between the two binaries,
passing under both bazel and ./regression. All 14 flow designs are now checked.

./regression consequence

Deriving limits from bazel means ./regression now fails these designs wherever
cmake QoR is worse than bazel's:

• aes_nangate45 — RSZ::max_capacitance_slack -14% < -11%, DRT::max_capacitance_slack -23% < -20%
• aes_sky130hs — DRT::clock_skew 0.29 > 0.13, DRT::ANT::errors 1 > 0
• jpeg_sky130hs — DRT::ANT::errors 1 > 0

(aes_asap7 now passes both.) One set of limits cannot satisfy both binaries for
these designs, and bazel is what CI gates on, so bazel wins. The alternative, if
keeping ./regression green on these matters, is to take the worse of the two
engines per metric, which weakens the CI gate to the cmake number.

Verification

• bazel test //test:flow_tests — 10 of 14 passed before regeneration; the 4 above failed on metrics
  only, each with Exitcode: 0, so the flows themselves succeed. After regeneration all 4 pass under bazel.
• Negative test both ways: tightening gcd_nangate45's DPL::design_area limit
  to 1.0 makes bazel report
  Metrics do not satisfy limits: DPL::design_area 615 >= 1 and fail, and
  ./regression report *FAIL* DPL::design_area 615 >= 1. Restored after.
• ctest — 8338/8338 pass, confirming the shared regression_test.sh change
  does not disturb non-flow tests.
• bazel test //test:upf_aes-tcl_test //test:write_db-tcl_test //src/psm/test:report_writers_require_solution-tcl_test — pass.
• buildifier, tclfmt and tclint clean.

[gemini-code-assist]

Code Review

This pull request integrates flow metrics validation into the Bazel-based regression test suite. It introduces a custom, lightweight JSON parser in Tcl (test/flow_metrics.tcl) to avoid external dependencies like tcllib, adds a standalone script (test/check_metrics.tcl) to compare metrics against limits, and updates the Bazel rules and shell scripts to support the new check_metrics option. Additionally, a new test case (gcd_sky130hd_fast_slow) is added, and several existing test metrics are updated. The reviewer suggested a portability improvement in test/regression_test.sh to use "$0" instead of "${BASH_SOURCE[0]}" to ensure compatibility with non-Bash shells.

[maliberty]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 3ee6969173

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".