NXP backend - added support for `aten.conv2d` using the new Neutron flow by novak-vaclav · Pull Request #19717 · pytorch/executorch

novak-vaclav · 2026-05-21T08:04:13Z

Summary

Added support for aten.conv2d using new Neutron flow.

Test plan

tests can be manually run using pytest -c /dev/null backends/nxp/tests/

cc @robert-kalmar @JakeStevens @digantdesai @rascani @MartinPavella

pytorch-bot · 2026-05-21T08:04:18Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19717

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Cancelled Job, 2 Unclassified Failures

As of commit 735d329 with merge base 10bc51e ():

NEW FAILURE - The following job has failed:

pull / unittest-editable / windows / windows-job (gh)
examples/models/test/test_export.py::ExportTest::test_efficient_sam_export_to_executorch

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

Cadence Build & Test / hifi-build / hifi4 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
Input required and not supplied: aws-region
Cadence Build & Test / vision-build / vision (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
Input required and not supplied: aws-region

CANCELLED JOB - The following job was cancelled. Please retry:

pull / unittest-nxp-neutron / linux-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

novak-vaclav · 2026-05-21T08:04:33Z

@pytorchbot label "module: nxp"

novak-vaclav · 2026-05-21T08:04:35Z

@pytorchbot label "release notes: nxp"

novak-vaclav · 2026-05-21T08:05:27Z

Please also review @MartinPavella @roman-janik-nxp @StrycekSimon. Thank you 😄

Copilot

Pull request overview

Adds aten.conv2d delegation coverage for the NXP Neutron “new flow” and expands the NXP backend test suite to validate delegation behavior and (where possible) numerical correctness.

Changes:

Add a new-flow support-check path in the convolution converter and tighten failure behavior when group conv decomposition is missing.
Add extensive new pytest coverage for conv2d delegation/non-delegation scenarios under the new Neutron flow (including depthwise and edge/bounds cases, plus targeted xfails for known issues).
Improve NPU-vs-CPU mismatch reporting in the output comparator by including the max diff in the assertion message.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
backends/nxp/tests/model_output_comparator.py	Improves assertion failure message for output mismatches.
backends/nxp/tests/ir/converter/node_converter/test_conv_converter.py	Adds a large new test suite for conv2d under the new Neutron flow, including delegation checks and known-issue `xfail`s.
backends/nxp/backend/ir/converter/node_converters/ops_converters/convolution_converter.py	Introduces new-flow target support checks and adjusts convolution argument handling / group-conv behavior.
backends/nxp/aten_passes/split_group_convolution.py	Updates group-conv splitting pass to handle optional bias and propagate FakeTensor meta safely.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

novak-vaclav · 2026-05-21T10:30:16Z

Fixed the issues found by Copilot.

MartinPavella

Will review the rest of test_conv_converter tomorrow.

MartinPavella · 2026-05-25T12:21:22Z

@novak-vaclav please take a look at this issue: https://jira.sw.nxp.com/browse/EIEX-809
Is it relevant/applicable?

linux-foundation-easycla · 2026-06-03T15:50:08Z

❌ - login: @novak-vaclav / name: Vaclav Novak. The commit (6521277, 735d329) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please visit our EasyCLA portal and chat with our support bot.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.

novak-vaclav · 2026-06-03T15:52:45Z

Fixed issues mentioned by @MartinPavella and updated the tests so they are based on Neutron SW 3.1.2.
@MartinPavella Do you need me to rebase it onto your Neutron SW 3.1.2 update branch?

The support for padding mode same / valid will be solved in a separate issue.

novak-vaclav · 2026-06-03T16:33:48Z

Fixed issues mentioned by Copilot.

Note: I am not using the use_qat pytest fixture in test_conv_converter deliberately, because sometimes I need to mark a test configuration with xfail when qat=False and sometimes when qat=True

novak-vaclav · 2026-06-04T08:27:01Z

Now I see I will have to rebase onto the Neutron SW 3.1.2 branch, will do

MartinPavella

LGTM 👍🏻
I will approve after you rebase to the 3.1.2 PR

Copilot

Pull request overview

Copilot reviewed 50 out of 50 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

backends/nxp/tests/ir/converter/node_converter/test_conv_converter.py:1

AllCloseOutputComparator is imported from executorch.backends.nxp.tests.nsys_testing, but in this PR other test modules import output comparators from executorch.backends.nxp.tests.model_output_comparator. If nsys_testing doesn’t define/re-export AllCloseOutputComparator, this will fail at import time. Import the comparator from its actual module (or re-export it explicitly in nsys_testing/a shared test utilities module) to keep imports consistent and avoid test collection errors.
examples/nxp/setup.sh:1
The --index-url value is unquoted; if EIQ_PYPI_URL ever contains special characters (or is empty/oddly formatted), word-splitting can break the command. Quote the variable (\"${EIQ_PYPI_URL}\") to make the script more robust.

novak-vaclav · 2026-06-08T12:05:29Z

Fixed the issues with not taking into account the transposed variant of conv2d. Now it should be fixed.
I also rebased onto the current main, updating the code to use Neutron 3.1.2

MartinPavella

LGTM. Once tests pass, we can merge.

MartinPavella · 2026-06-09T21:12:49Z

@novak-vaclav please take a look at the failing linting and neutron job.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

            w_data = self._get_tensor_constant_from_node(w)
            b_data = self._get_tensor_constant_from_node(b)
-            if w_data is None or b_data is None:
-                continue  # Only the standard case with static weights and bias is supported.
+
+            with_bias = b is not None
+            # Only the standard case with static weights and static bias (or bias=False) is supported.
+            if w_data is None or (b_data is None and with_bias):
+                continue


+    pytest.mark.xfail(
+        reason="AIR-14679: should start working after `conv2d` giving incorrect results is fixed by Neutron team.",
+        strict=True,
+    )
+
    def test_nsys_inference__with_conv(self, mocker):


            elif conv_utils.group_conv_convertible_into_multiple_convolutions(
                t_op, conv_params.groups
-            ):  # Convert to separated `Conv2D`.
-                t_op.builtin_options = conv_2d_options.Conv2D()
-
-                return conv_utils.create_separated_convolutions_based_on_group(
-                    t_op,
-                    conv_params,
-                    self.builder,
-                    self._convert_unpadded_2D,
-                    conv_utils.conv_op_factory,
-                )
+            ):
+                raise RuntimeError("NXP backend: Group convolution was not decomposed.")


    ).exported_program()

-    assert len(edge_program.graph.nodes) == 21
+    assert len(edge_program.graph.nodes) == 7


novak-vaclav requested a review from robert-kalmar as a code owner May 21, 2026 08:04

Copilot AI review requested due to automatic review settings May 21, 2026 08:04

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 21, 2026

pytorch-bot Bot added module: nxp Issues related to NXP Neutron NPU delegation and code under backends/nxp/ release notes: nxp Changes to the NXP Neutron backend delegate labels May 21, 2026

Copilot started reviewing on behalf of novak-vaclav May 21, 2026 08:04 View session

Copilot AI reviewed May 21, 2026

View reviewed changes

Comment thread backends/nxp/backend/ir/converter/node_converters/ops_converters/convolution_converter.py Outdated

Comment thread backends/nxp/backend/ir/converter/node_converters/ops_converters/convolution_converter.py Outdated

novak-vaclav force-pushed the feature/EIEX-861-add-conv2d-support-new-neutron-flow branch from 9cedbc4 to bc6ecdf Compare May 21, 2026 10:29

MartinPavella reviewed May 21, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings June 3, 2026 15:50

novak-vaclav force-pushed the feature/EIEX-861-add-conv2d-support-new-neutron-flow branch from bc6ecdf to e6db65f Compare June 3, 2026 15:50

Copilot AI reviewed Jun 3, 2026

View reviewed changes

novak-vaclav force-pushed the feature/EIEX-861-add-conv2d-support-new-neutron-flow branch from e6db65f to 1f46333 Compare June 3, 2026 16:31

novak-vaclav requested a review from MartinPavella June 3, 2026 16:33

MartinPavella reviewed Jun 4, 2026

View reviewed changes

Comment thread backends/nxp/backend/ir/converter/node_converters/ops_converters/convolution_converter.py Outdated

Comment thread backends/nxp/tests/ir/converter/node_converter/test_conv_converter.py

Copilot AI review requested due to automatic review settings June 8, 2026 11:45

novak-vaclav force-pushed the feature/EIEX-861-add-conv2d-support-new-neutron-flow branch from 1f46333 to ba6aee9 Compare June 8, 2026 11:45

novak-vaclav requested a review from mergennachin as a code owner June 8, 2026 11:45

Copilot AI reviewed Jun 8, 2026

View reviewed changes

Comment thread backends/nxp/backend/ir/converter/node_converters/ops_converters/convolution_converter.py

Comment thread backends/nxp/tests/generic_tests/test_context_sensitive_delegation.py

novak-vaclav force-pushed the feature/EIEX-861-add-conv2d-support-new-neutron-flow branch from ba6aee9 to 469260c Compare June 8, 2026 11:57

novak-vaclav requested a review from MartinPavella June 8, 2026 11:58

MartinPavella approved these changes Jun 9, 2026

View reviewed changes

Comment thread backends/nxp/backend/ir/converter/node_converters/ops_converters/convolution_converter.py

Copilot AI review requested due to automatic review settings June 10, 2026 13:37

novak-vaclav force-pushed the feature/EIEX-861-add-conv2d-support-new-neutron-flow branch from 469260c to c6029b3 Compare June 10, 2026 13:37

Copilot AI reviewed Jun 10, 2026

View reviewed changes

novak-vaclav force-pushed the feature/EIEX-861-add-conv2d-support-new-neutron-flow branch from c6029b3 to 6e3f568 Compare June 10, 2026 13:48

feat: enabled splitting of group convolution for bias=False

6521277

Copilot AI review requested due to automatic review settings June 10, 2026 14:34

novak-vaclav force-pushed the feature/EIEX-861-add-conv2d-support-new-neutron-flow branch from 6e3f568 to 270b744 Compare June 10, 2026 14:34

Copilot AI reviewed Jun 10, 2026

View reviewed changes

novak-vaclav force-pushed the feature/EIEX-861-add-conv2d-support-new-neutron-flow branch from 270b744 to 5a99b69 Compare June 10, 2026 15:12

feat: added support for conv2d using new NeutronC flow

735d329

Copilot AI review requested due to automatic review settings June 12, 2026 11:18

novak-vaclav force-pushed the feature/EIEX-861-add-conv2d-support-new-neutron-flow branch from 5a99b69 to 735d329 Compare June 12, 2026 11:18

Copilot AI reviewed Jun 12, 2026

View reviewed changes

novak-vaclav had a problem deploying to cadence June 12, 2026 13:34 — with GitHub Actions Failure

Conversation

novak-vaclav commented May 21, 2026 • edited by pytorch-bot Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

pytorch-bot Bot commented May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19717

❌ 1 New Failure, 1 Cancelled Job, 2 Unclassified Failures

Uh oh!

novak-vaclav commented May 21, 2026

Uh oh!

novak-vaclav commented May 21, 2026

Uh oh!

novak-vaclav commented May 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

novak-vaclav commented May 21, 2026

Uh oh!

MartinPavella left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MartinPavella commented May 25, 2026

Uh oh!

linux-foundation-easycla Bot commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

novak-vaclav commented Jun 3, 2026

Uh oh!

novak-vaclav commented Jun 3, 2026

Uh oh!

novak-vaclav commented Jun 4, 2026

Uh oh!

MartinPavella left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

novak-vaclav commented Jun 8, 2026

Uh oh!

MartinPavella left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MartinPavella commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

novak-vaclav commented May 21, 2026 •

edited by pytorch-bot Bot

Loading

pytorch-bot Bot commented May 21, 2026 •

edited

Loading

linux-foundation-easycla Bot commented Jun 3, 2026 •

edited

Loading

MartinPavella commented Jun 9, 2026 •

edited

Loading