PyTest Migration

[Victor-Jung]

We used to write the tests directly in the GitHub Action YAML files; this is not a good practice, as it is hard to run several tests locally. PyTest is a well-known test framework that allows decoupling the testing from the GitHub CI, so users can easily run all tests for any platform locally if needed.

Currently, I use PyTest only for the Generic platform. I'd like a first round of review from @Xeratec or @diaconuccalin before moving forward. Moving the CI for other platforms will be done in a similar way.

All platforms and internal deeploy tests are now moved to the PyTest suite and the CI is only using the PyTest suite.

Additionally, the PyTest suite can run tests in parallel and generate one build folder per worker to prevent conflicts.

Tagging @haugoug as he's interested in this feature.

To run all the tests of the generic platform with 8 parallel threads, you can do

pytest test_generic.py -v -n 8

You can select a specific marker with -m, like only the kernels, you can do:

pytest test_generic.py -v -n 8 -m kernels

To list all the captured tests for a given expression, you can do:

pytest test_generic.py -v -n 8 -m kernels --collect-only

To execute a specific test, you can use:

pytest 'test_generic.py::test_generic_kernels[Adder]' -v

You can also use an expression to filter tests, for instance the following command execute all test whose name contain "Adder":

pytest test_generic.py -v -k "Adder"

I summarized the important points to use the PyTest suite in a README in the DeeployTest folder.

Added

• pytest and pytest-xdist as dependencies of Deeploy.
• A pytest.ini for the global configuration of PyTest for the project.
• conftest.py to define CLI args for PyTest for the whole project, it also defines a set of global fixtures and markers.
• pytestRunner.py contains helper functions and fixtures for the whole project.
• test_platforms.py lists the E2E tests and sorts them into marked categories (per platform and per kernel/model).
• Each platform has a test config file where a list or a dict describes the tests.

Changed

• Each CI workflow has been simplified to call the pytest suite with certain markers.

Fixed

• nvidia-pyindex was broken as it now tries to build the wheel to respect the new policy on packages using pyproject. Instead of installing this package, we just add the https://pypi.ngc.nvidia.com channel to the pip config file.
• Pin versions of broken dependencies of Banshee.

PR Merge Checklist

1. The PR is rebased on the latest devel commit and pointing to devel.
2. Your PR reviewed and approved.
3. All checks are passing.
4. The CHANGELOG.md file has been updated.
5. If the docker was modified, change back its link after review.

[diaconuccalin]

Great feature, fantastically useful! :)

One more comment other than the ones I've already left, maybe add the instructions you included in the PR description to a dedicated .md file in the DeeployTest directory, with a mention of it in the big README, for better documentation for future users (including myself :) ).

[coderabbitai]

Actionable comments posted: 2

🤖 Fix all issues with AI agents

In `@DeeployTest/testUtils/core/execution.py`:
- Line 197: The log.warning call uses an unnecessary f-string: replace the call
to log.warning(f"Could not parse error count from output") with a plain string
literal (log.warning("Could not parse error count from output")) to remove the
extraneous f-prefix in the code path where the parser logs an unparsable error
count.

In `@DeeployTest/testUtils/deeployRunner.py`:
- Around line 178-185: The build_dir is currently hardcoded to "build" which can
collide across pytest-xdist workers; modify the logic around build_dir in
DeeployTest/testUtils/deeployRunner.py to append the pytest worker id (from
os.environ.get("PYTEST_XDIST_WORKER","master")) so each worker uses a unique
directory (e.g., "build_master" or "build_{worker_id}"); update the variable
build_dir (and any subsequent uses) to construct Path(base_dir) /
f"TEST_{platform.upper()}" / f"build_{worker_id}" when worker_id != "master"
(and default to build_master for consistency) to match the approach used in
pytestRunner.py/get_pytest_worker_id.

♻️ Duplicate comments (2)

DeeployTest/testUtils/core/execution.py (2)

68-69: Replace the assert with an explicit runtime check.

This concern was raised previously: asserts are stripped when running Python with -O, which would cause config.toolchain_install_dir to remain None and be passed to CMake as "DTOOLCHAIN_INSTALL_DIR=None". The function already uses RuntimeError for similar validation (line 73).

Proposed fix

-    assert config.toolchain_install_dir is not None, \
-        "LLVM_INSTALL_DIR environment variable not set"
+    if config.toolchain_install_dir is None:
+        raise RuntimeError("LLVM_INSTALL_DIR environment variable not set")

---

186-199: Check subprocess return code to avoid false positives.

This concern was raised previously: a non-zero exit status from the simulation subprocess could be reported as success if the output parsing happens to find zero errors. The return code should be checked before or after parsing.

Proposed fix

     result = subprocess.run(cmd, capture_output = True, text = True, env = env)
 
     if result.stdout:
         print(result.stdout, end = '')
     if result.stderr:
         print(result.stderr, end = '', file = sys.stderr)
 
     # Parse output for error count and cycles
     test_result = parse_test_output(result.stdout, result.stderr)
 
+    # Check subprocess return code - non-zero indicates failure regardless of parsed output
+    if result.returncode != 0 and test_result.success:
+        log.warning(f"Simulation returned non-zero exit code {result.returncode} but output parsed as success")
+        test_result = TestResult(
+            success = False,
+            error_count = test_result.error_count,
+            total_count = test_result.total_count,
+            stdout = test_result.stdout,
+            stderr = test_result.stderr,
+            runtime_cycles = test_result.runtime_cycles,
+        )
+
     if not test_result.success and test_result.error_count == -1:
-        log.warning(f"Could not parse error count from output")
+        log.warning("Could not parse error count from output")
 
     return test_result

🧹 Nitpick comments (2)

DeeployTest/deeployRunner_snitch.py (1)

1-7: Consider adding a module docstring for consistency.

Other platform runners (e.g., deeployRunner_softhier.py, deeployRunner_chimera.py) include a module docstring like """Deeploy runner for <Platform> platform.""". Adding one here would maintain consistency across the codebase.

Suggested fix

 # SPDX-License-Identifier: Apache-2.0
+"""Deeploy runner for Snitch platform."""
 
 from testUtils.deeployRunner import main

DeeployTest/testUtils/deeployRunner.py (1)

395-406: Preserve stack traces on failures.

Catching Exception without logging the traceback makes failures harder to debug. Consider logging the full exception (or re-raising when verbose/debug).

🪵 Minimal logging improvement

-    except Exception as e:
+    except Exception as e:
         RED = '\033[91m'
         RESET = '\033[0m'
-        print(f"\n{RED}✗ Test {config.test_name} FAILED with exception: {e}{RESET}")
+        log.exception("Test %s failed with exception", config.test_name)
+        print(f"\n{RED}✗ Test {config.test_name} FAILED with exception: {e}{RESET}")
         return 1

📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 0eb56b9 and 5b7dd63.

📒 Files selected for processing (13)

• DeeployTest/deeployRunner_chimera.py
• DeeployTest/deeployRunner_cortexm.py
• DeeployTest/deeployRunner_generic.py
• DeeployTest/deeployRunner_mempool.py
• DeeployTest/deeployRunner_siracusa.py
• DeeployTest/deeployRunner_snitch.py
• DeeployTest/deeployRunner_softhier.py
• DeeployTest/deeployRunner_tiled_siracusa.py
• DeeployTest/deeployRunner_tiled_siracusa_w_neureka.py
• DeeployTest/deeployRunner_tiled_snitch.py
• DeeployTest/testUtils/core/execution.py
• DeeployTest/testUtils/core/output_parser.py
• DeeployTest/testUtils/deeployRunner.py

🚧 Files skipped from review as they are similar to previous changes (7)

• DeeployTest/deeployRunner_tiled_siracusa.py
• DeeployTest/deeployRunner_tiled_snitch.py
• DeeployTest/deeployRunner_cortexm.py
• DeeployTest/deeployRunner_generic.py
• DeeployTest/deeployRunner_tiled_siracusa_w_neureka.py
• DeeployTest/deeployRunner_siracusa.py
• DeeployTest/deeployRunner_mempool.py

🧰 Additional context used

🧠 Learnings (1)

📚 Learning: 2025-09-24T11:43:47.236Z

Learnt from: diaconuccalin
Repo: pulp-platform/Deeploy PR: 117
File: .github/workflows/ci-platform-siracusa.yml:57-60
Timestamp: 2025-09-24T11:43:47.236Z
Learning: In the Deeploy test system, test names in CI workflows correspond to directory names under DeeployTest/Tests/, not Python function names. The TestRunner class executes tests by passing directory paths via the `-t` argument, where each directory contains test configurations and definitions.

Applied to files:

• DeeployTest/testUtils/core/execution.py
• DeeployTest/testUtils/deeployRunner.py
• DeeployTest/deeployRunner_softhier.py
• DeeployTest/deeployRunner_chimera.py
• DeeployTest/deeployRunner_snitch.py

🧬 Code graph analysis (5)

DeeployTest/testUtils/core/execution.py (5)

DeeployTest/testUtils/core/config.py (1)

• DeeployTestConfig (11-33)

DeeployTest/testUtils/core/output_parser.py (2)

• TestResult (11-17)
• parse_test_output (20-49)

DeeployTest/testUtils/testRunner.py (1)

• run (325-336)

DeeployTest/conftest.py (3)

• toolchain (135-137)
• skipgen (123-125)
• skipsim (129-131)

Deeploy/DeeployTypes.py (1)

• copy (1016-1020)

DeeployTest/testUtils/deeployRunner.py (4)

DeeployTest/testUtils/core/config.py (1)

• DeeployTestConfig (11-33)

DeeployTest/testUtils/core/execution.py (1)

• run_complete_test (202-220)

DeeployTest/testUtils/core/paths.py (1)

• get_test_paths (11-48)

DeeployTest/conftest.py (3)

• toolchain (135-137)
• skipgen (123-125)
• skipsim (129-131)

DeeployTest/deeployRunner_softhier.py (8)

DeeployTest/testUtils/deeployRunner.py (1)

• main (294-406)

DeeployTest/deeployRunner_chimera.py (1)

• setup_parser (13-14)

DeeployTest/deeployRunner_mempool.py (1)

• setup_parser (13-18)

DeeployTest/deeployRunner_siracusa.py (1)

• setup_parser (12-13)

DeeployTest/deeployRunner_snitch.py (1)

• setup_parser (12-17)

DeeployTest/deeployRunner_tiled_siracusa.py (1)

• setup_parser (12-13)

DeeployTest/deeployRunner_tiled_siracusa_w_neureka.py (1)

• setup_parser (12-15)

DeeployTest/deeployRunner_tiled_snitch.py (1)

• setup_parser (12-17)

DeeployTest/deeployRunner_chimera.py (7)

DeeployTest/deeployRunner_mempool.py (1)

• setup_parser (13-18)

DeeployTest/deeployRunner_siracusa.py (1)

• setup_parser (12-13)

DeeployTest/deeployRunner_snitch.py (1)

• setup_parser (12-17)

DeeployTest/deeployRunner_softhier.py (1)

• setup_parser (13-19)

DeeployTest/deeployRunner_tiled_siracusa.py (1)

• setup_parser (12-13)

DeeployTest/deeployRunner_tiled_siracusa_w_neureka.py (1)

• setup_parser (12-15)

DeeployTest/deeployRunner_tiled_snitch.py (1)

• setup_parser (12-17)

DeeployTest/deeployRunner_snitch.py (8)

DeeployTest/testUtils/deeployRunner.py (1)

• main (294-406)

DeeployTest/deeployRunner_chimera.py (1)

• setup_parser (13-14)

DeeployTest/deeployRunner_mempool.py (1)

• setup_parser (13-18)

DeeployTest/deeployRunner_siracusa.py (1)

• setup_parser (12-13)

DeeployTest/deeployRunner_softhier.py (1)

• setup_parser (13-19)

DeeployTest/deeployRunner_tiled_siracusa.py (1)

• setup_parser (12-13)

DeeployTest/deeployRunner_tiled_siracusa_w_neureka.py (1)

• setup_parser (12-15)

DeeployTest/deeployRunner_tiled_snitch.py (1)

• setup_parser (12-17)

🪛 Ruff (0.14.11)

DeeployTest/testUtils/core/execution.py

59-59: subprocess call: check for execution of untrusted input

(S603)

---

63-63: Avoid specifying long messages outside the exception class

(TRY003)

---

73-73: Avoid specifying long messages outside the exception class

(TRY003)

---

111-111: subprocess call: check for execution of untrusted input

(S603)

---

115-115: Avoid specifying long messages outside the exception class

(TRY003)

---

136-136: subprocess call: check for execution of untrusted input

(S603)

---

140-140: Avoid specifying long messages outside the exception class

(TRY003)

---

155-155: Avoid specifying long messages outside the exception class

(TRY003)

---

186-186: subprocess call: check for execution of untrusted input

(S603)

---

197-197: f-string without any placeholders

Remove extraneous f prefix

(F541)

DeeployTest/testUtils/deeployRunner.py

352-352: f-string without any placeholders

Remove extraneous f prefix

(F541)

---

400-400: Consider moving this statement to an else block

(TRY300)

---

402-402: Do not catch blind exception: Exception

(BLE001)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (16)

• GitHub Check: chimera-kernels / test-runner-chimera
• GitHub Check: linting
• GitHub Check: softhier-kernels / test-runner-softhier
• GitHub Check: deeploy-internal-tests
• GitHub Check: build-deeploy
• GitHub Check: snitch-kernels-tiled-singlebuffer-L2 / test-runner-snitch-tiled
• GitHub Check: siracusa-kernels / test-runner-siracusa
• GitHub Check: siracusa-models / test-runner-siracusa
• GitHub Check: siracusa-models-tiled-l2-singlebuffer / test-runner-siracusa-tiled
• GitHub Check: siracusa-kernels-tiled-l2-doublebuffer / test-runner-siracusa-tiled
• GitHub Check: siracusa-models-tiled-l2-doublebuffer / test-runner-siracusa-tiled
• GitHub Check: siracusa-models-tiled-l3-singlebuffer / test-runner-siracusa-tiled
• GitHub Check: siracusa-kernels-tiled-l2-singlebuffer / test-runner-siracusa-tiled
• GitHub Check: siracusa-models-tiled-l3-doublebuffer / test-runner-siracusa-tiled
• GitHub Check: cortexm-kernels / test-runner-cortexm
• GitHub Check: cortexm-models / test-runner-cortexm

🔇 Additional comments (7)

DeeployTest/deeployRunner_softhier.py (1)

1-25: LGTM!

The SoftHier runner correctly follows the established pattern for platform-specific runners. The --num-clusters and --cores arguments are properly configured and will be handled by the main function's CMake argument extraction logic.

DeeployTest/deeployRunner_chimera.py (1)

1-20: LGTM!

The Chimera runner follows the established pattern and is consistent with similar runners like Siracusa.

DeeployTest/testUtils/core/output_parser.py (1)

10-49: LGTM!

The TestResult dataclass and parse_test_output function are well-designed. The fallback behavior of treating unparseable output as a failure (with sentinel values -1) is a safe approach.

DeeployTest/testUtils/core/execution.py (3)

17-63: LGTM!

The generate_network function properly handles the skip case, constructs the command correctly, and raises RuntimeError on failure.

---

118-140: LGTM!

The build_binary function properly handles the build process and raises RuntimeError on failure.

---

202-220: LGTM!

The run_complete_test function properly orchestrates the test flow and propagates the result.

DeeployTest/testUtils/deeployRunner.py (1)

382-389: The concern about None values is not applicable—all platform-specific arguments have explicit non-None defaults.

All platform-specific runners define these arguments through argparse with explicit defaults: num_cores defaults to 8 or 16, num_clusters defaults to 1, and cores defaults to 8. Since argparse enforces type=int on all these arguments and provides non-None defaults, the attributes will never be None at runtime. The hasattr() checks are correctly used here to determine whether optional platform-specific attributes exist on the args object; they don't need additional value guards.

Likely an incorrect or invalid review comment.

_{✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.}

[Victor-Jung]

@runwangdl I implemented your feedback, and we now have a CLI interface for developers. The logic behind is centralized and used also by the pytest suite.

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@DeeployTest/testUtils/deeployRunner.py`:
- Line 359: Remove the unnecessary f-string prefix on the print call that
currently reads print(f"Please use one of the following options:", file =
sys.stderr); change it to a normal string literal (print("Please use one of the
following options:", file=sys.stderr)) and tidy the keyword argument spacing
(use file=sys.stderr) so no f-string is used for a string with no placeholders.

🧹 Nitpick comments (1)

DeeployTest/testUtils/deeployRunner.py (1)

409-413: Consider catching more specific exceptions.

Catching bare Exception can mask unexpected bugs. While this is common in CLI entry points for graceful error handling, consider catching more specific exception types (e.g., RuntimeError, OSError, subprocess.CalledProcessError) that the underlying run_complete_test might raise.

Proposed refinement

-    except Exception as e:
+    except (RuntimeError, OSError, KeyboardInterrupt) as e:
         RED = '\033[91m'
         RESET = '\033[0m'
         print(f"\n{RED}✗ Test {config.test_name} FAILED with exception: {e}{RESET}")
         return 1

📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9dc8b58 and f81c378.

📒 Files selected for processing (2)

• DeeployTest/testUtils/deeployRunner.py
• DeeployTest/testUtils/pytestRunner.py

🧰 Additional context used

🧠 Learnings (2)

📓 Common learnings

Learnt from: diaconuccalin
Repo: pulp-platform/Deeploy PR: 117
File: .github/workflows/ci-platform-siracusa.yml:57-60
Timestamp: 2025-09-24T11:43:47.236Z
Learning: In the Deeploy test system, test names in CI workflows correspond to directory names under DeeployTest/Tests/, not Python function names. The TestRunner class executes tests by passing directory paths via the `-t` argument, where each directory contains test configurations and definitions.

📚 Learning: 2025-09-24T11:43:47.236Z

Learnt from: diaconuccalin
Repo: pulp-platform/Deeploy PR: 117
File: .github/workflows/ci-platform-siracusa.yml:57-60
Timestamp: 2025-09-24T11:43:47.236Z
Learning: In the Deeploy test system, test names in CI workflows correspond to directory names under DeeployTest/Tests/, not Python function names. The TestRunner class executes tests by passing directory paths via the `-t` argument, where each directory contains test configurations and definitions.

Applied to files:

• DeeployTest/testUtils/deeployRunner.py
• DeeployTest/testUtils/pytestRunner.py

🧬 Code graph analysis (2)

DeeployTest/testUtils/deeployRunner.py (4)

DeeployTest/testUtils/core/config.py (1)

• DeeployTestConfig (11-33)

DeeployTest/testUtils/core/execution.py (1)

• run_complete_test (202-220)

DeeployTest/testUtils/core/paths.py (1)

• get_test_paths (11-48)

DeeployTest/conftest.py (3)

• toolchain (135-137)
• skipgen (123-125)
• skipsim (129-131)

DeeployTest/testUtils/pytestRunner.py (4)

DeeployTest/testUtils/core/config.py (1)

• DeeployTestConfig (11-33)

DeeployTest/testUtils/core/execution.py (4)

• build_binary (118-140)
• configure_cmake (66-115)
• run_complete_test (202-220)
• run_simulation (143-199)

DeeployTest/testUtils/core/paths.py (1)

• get_test_paths (11-48)

DeeployTest/conftest.py (6)

• deeploy_test_dir (86-88)
• toolchain (135-137)
• toolchain_dir (98-103)
• cmake_args (141-143)
• skipgen (123-125)
• skipsim (129-131)

🪛 Ruff (0.14.11)

DeeployTest/testUtils/deeployRunner.py

359-359: f-string without any placeholders

Remove extraneous f prefix

(F541)

---

407-407: Consider moving this statement to an else block

(TRY300)

---

409-409: Do not catch blind exception: Exception

(BLE001)

DeeployTest/testUtils/pytestRunner.py

11-18: __all__ is not sorted

Apply an isort-style sorting to __all__

(RUF022)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (17)

• GitHub Check: snitch-kernels / test-runner-snitch
• GitHub Check: siracusa-models / test-runner-siracusa
• GitHub Check: siracusa-kernels / test-runner-siracusa
• GitHub Check: deeploy-internal-tests
• GitHub Check: mempool-models / test-runner-mempool
• GitHub Check: mempool-kernels / test-runner-mempool
• GitHub Check: linting
• GitHub Check: softhier-kernels / test-runner-softhier
• GitHub Check: siracusa-kernels-tiled-l2-doublebuffer / test-runner-siracusa-tiled
• GitHub Check: siracusa-kernels-tiled-l2-singlebuffer / test-runner-siracusa-tiled
• GitHub Check: siracusa-models-tiled-l3-singlebuffer / test-runner-siracusa-tiled
• GitHub Check: siracusa-models-tiled-l3-doublebuffer / test-runner-siracusa-tiled
• GitHub Check: siracusa-models-tiled-l2-singlebuffer / test-runner-siracusa-tiled
• GitHub Check: siracusa-models-tiled-l2-doublebuffer / test-runner-siracusa-tiled
• GitHub Check: cortexm-models / test-runner-cortexm
• GitHub Check: cortexm-kernels / test-runner-cortexm
• GitHub Check: docs

🔇 Additional comments (9)

DeeployTest/testUtils/pytestRunner.py (4)

1-19: LGTM on imports and module structure.

The imports are correct and the module exposes a clean public API. Note: Ruff flags that __all__ is unsorted, but this is a minor style consideration.

---

21-28: LGTM!

Clean utility for detecting pytest-xdist worker ID with appropriate default for non-parallel execution.

---

31-107: LGTM!

The configuration builder correctly:

• Derives paths using get_test_paths
• Isolates build directories per pytest-xdist worker
• Conditionally appends tiling and platform-specific arguments only when they differ from defaults

This provides clean separation between test orchestration and the core execution layer.

---

110-123: LGTM!

The dual assertion strategy is appropriate:

1. First assertion ensures the overall test succeeded
2. Second assertion (when error_count >= 0) provides additional verification that no errors were logged

This correctly handles cases where error_count = -1 indicates output parsing failure (as documented in run_simulation).

DeeployTest/testUtils/deeployRunner.py (5)

1-23: LGTM on imports and helper function.

The cmake_str helper correctly handles unicode escaping for CMake arguments.

---

26-171: LGTM!

The argument parser is well-structured with:

• Conditional tiling arguments based on the tiling_arguments flag
• Appropriate verbosity-to-log-level mapping
• Clear help text for all options

---

174-245: LGTM! Worker isolation fix is correctly implemented.

Lines 186-191 now properly handle pytest-xdist worker isolation with worker-specific build directories, consistent with the pattern in pytestRunner.py. This resolves the previous concern about parallel test collisions.

---

248-298: LGTM!

The colored output functions provide clear, informative feedback for CLI usage. The configuration display comprehensively shows all relevant test parameters.

---

416-417: LGTM!

Standard CLI entry point pattern.

_{✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.}

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@DeeployTest/conftest.py`:
- Around line 91-94: The deeploy_test_dir fixture returns a pathlib.Path but
create_test_config currently types its deeploy_test_dir parameter as str; update
the type hints to accept both Path and str (e.g., use Union[str, Path] or str |
Path) on the create_test_config parameter (and any related functions) so type
checkers align with the fixture, and ensure get_test_paths (already converting
to Path) remains unchanged.

🧹 Nitpick comments (4)

DeeployTest/testUtils/deeployRunner.py (1)

411-422: Exception handling in main entry point.

The broad Exception catch on line 418 is acceptable for a CLI entry point where you want to ensure a clean error message and proper exit code rather than a stack trace. However, consider logging the full traceback at debug level for troubleshooting.

💡 Optional: Log traceback for debugging

     except Exception as e:
         RED = '\033[91m'
         RESET = '\033[0m'
         print(f"\n{RED}✗ Test {config.test_name} FAILED with exception: {e}{RESET}")
+        log.debug(f"Exception traceback:", exc_info=True)
         return 1

DeeployTest/testUtils/pytestRunner.py (1)

11-18: Consider sorting __all__ for consistency.

The __all__ list is not alphabetically sorted. This is a minor style preference flagged by Ruff (RUF022).

💡 Optional: Sort __all__ alphabetically

 __all__ = [
+    'build_binary',
+    'configure_cmake',
+    'create_test_config',
     'get_worker_id',
-    'create_test_config',
     'run_and_assert_test',
-    'build_binary',
-    'configure_cmake',
     'run_simulation',
 ]

DeeployTest/test_platforms.py (2)

63-103: PLATFORM_CONFIGS provides good centralization.

The configuration dictionary centralizes platform settings. Note that Siracusa and Neureka tests hardcode their platform/simulator values directly rather than using this dict - consider extending PLATFORM_CONFIGS to include them for consistency.

---

295-295: Prefix unused config_name with underscore.

The config_name variable is unpacked but never used in the function body. This pattern appears in multiple tiled test functions. Prefix with _ to indicate intentional non-use.

💡 Proposed fix for all occurrences

-    test_name, l1, config_name = test_params
+    test_name, l1, _config_name = test_params

Apply this change to all tiled test functions where config_name is unpacked but unused:

• Lines 295, 324, 353, 382, 411, 440, 531, 564, 593, 622, 651, 681, 712

[runwangdl]

Looks good to me! Thank you for your contribution.

[diaconuccalin]

Great new features, looks like lots of work so thank you. LGTM