add torch-tensorrt-executorch-delegate wheel build

[lanluo-nvidia]

Description

Adds a separately packaged torch-tensorrt-executorch-delegate wheel that bundles the ExecuTorch Python portable runtime with TensorRTBackend registered. This allows an exported .pte model to be loaded and executed with torch_tensorrt.load(..., format="executorch") without requiring an ExecuTorch source checkout or a native build on the deployment system.

The main torch-tensorrt package continues to own compilation and export. ExecuTorch runtime loading and its native libraries are isolated in the optional delegate package.

Changes

ExecuTorch delegate package and build

• Add the torch-tensorrt-executorch-delegate Python package.
• Build the delegate's native runtime with Bazel and CMake.
• Bundle _portable_lib.so and data_loader.so in the wheel.
• Force every pip wheel invocation to rebuild and recopy the native libraries, preventing stale .so files from being packaged.
• Add Bazel targets and root source mappings required by the delegate build.

Package and API separation

• Keep ExecuTorch compile/export functionality in the main torch-tensorrt package.
• Move Program, .pte loading, runtime activation, and native module handling into the delegate package.
• Add format="executorch" support to torch_tensorrt.load.
• Lazily check for the delegate package when ExecuTorch loading is requested and provide an installation error when it is unavailable.
• Add the delegate package to the Linux executorch and all extras.

Runtime activation

• Load the bundled backend-enabled portable runtime in place of ExecuTorch's stock _portable_lib.
• Register the bundled data-loader module under ExecuTorch's canonical module path before portable-runtime initialization.
• Avoid duplicate pybind registration of PyDataLoader.
• Detect incompatible import ordering and report a targeted compatibility error.
• Validate that TensorRTBackend is present in the runtime backend registry.

ExecuTorch version and dependencies

• Pin the Bazel ExecuTorch source to commit e2f18eb23c45bd22ca332b0b8b49a81de304b472, matching executorch==1.3.1.
• Declare exact compatible versions of torch, executorch, and torch-tensorrt for the delegate wheel.
• Update the CI workspace module configuration to use the same ExecuTorch source.

CI, examples, and documentation

• Build and install the delegate wheel in Linux wheel CI.
• Add backend-registration and Python load/run smoke tests.
• Update the static ExecuTorch workflow to export once and verify the same .pte model with both the C++ reference runner and the Python delegate runtime.
• Update load_model.py to accept --model_path and --num_runs.
• Document how to build, install, save, and load an ExecuTorch model with the delegate wheel.

Tests

• Add coverage for ExecuTorch load API dispatch and optional-package behavior.
• Add runtime tests for model loading and execution, unknown methods, missing files, idempotent activation, stock-runtime import collisions, backend registration, and data-loader rollback.
• Update serialization and packaging tests for the separated delegate distribution.

Validation

• Built the native Bazel delegate target successfully.
• Built and installed a fresh delegate wheel.
• Passed the focused ExecuTorch API, runtime, serialization, and packaging tests.
• Loaded model.pte through torch_tensorrt.load(..., format="executorch") and verified the TensorRT delegate result against the expected x + 1 output.

Type of change

• New feature
• Bug fix
• Documentation update
• Test and CI coverage

[narendasan]

@lanluo-nvidia the new package should be in

//py
-> torch-tensorrt
-> torch-tensorrt-executorch-delegate

See: https://packaging.python.org/en/latest/guides/packaging-namespace-packages/

[shoumikhin]

Nice work on this. The core idea is right. You build _portable_lib from source and pull TensorRTBackend into the same module that holds the backend registry, so the two always share one copy of the ExecuTorch core. That is the correct way to fix the undefined symbol crash people hit today. I checked the linking part and it holds up: one registry, no duplicate core, and the whole-archive flag is actually needed.

[shoumikhin]

One more note, on a file this PR does not change so I cannot leave it inline. In cpp/src/torch_tensorrt/executorch/TensorRTBackend.cpp around line 653, the result of register_backend is stored in kRegistered but never checked, so if registration fails you find out much later with a confusing "Backend TensorRTBackend is not registered" message. It would help to notice the failure at registration time. One thing to watch out for: this runs very early, before logging is set up, so do not call ET_LOG here. A safer option is to save the result and check it the first time the backend is used.

[github-actions]

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch-tensorrt-executorch-delegate/setup.py	2026-07-17 19:30:44.124103+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch-tensorrt-executorch-delegate/setup.py	2026-07-17 19:31:08.985550+00:00
@@ -14,10 +14,11 @@
from setuptools import Extension, find_packages, setup
from setuptools.command.build_ext import build_ext

HERE = pathlib.Path(__file__).resolve().parent
REPO_ROOT = HERE.parents[1]
+

def torchtrt_version() -> str:
    if value := os.getenv("TORCH_TENSORRT_EXECUTORCH_DELEGATE_VERSION"):
        return value
    version_py = REPO_ROOT / "py/torch_tensorrt/_version.py"

[github-actions]

There are some changes that do not conform to C++ style guidelines:

diff --git a/home/runner/work/TensorRT/TensorRT/cpp/src/torch_tensorrt/executorch/TensorRTBackend.cpp b/tmp/changes.txt
index 648fb88..e335748 100644
--- a/home/runner/work/TensorRT/TensorRT/cpp/src/torch_tensorrt/executorch/TensorRTBackend.cpp
+++ b/tmp/changes.txt
@@ -56,8 +56,7 @@ extern const Error kRegistrationResult;

Error check_registration() {
  if (kRegistrationResult != Error::Ok) {
-    ET_LOG(
-        Error, "TensorRTBackend registration failed: %s", ::executorch::runtime::to_string(kRegistrationResult));
+    ET_LOG(Error, "TensorRTBackend registration failed: %s", ::executorch::runtime::to_string(kRegistrationResult));
  }
  return kRegistrationResult;
}
ERROR: Some files do not conform to style guidelines

[github-actions]

There are some changes that do not conform to C++ style guidelines:

diff --git a/home/runner/work/TensorRT/TensorRT/cpp/src/torch_tensorrt/executorch/TensorRTBackend.cpp b/tmp/changes.txt
index 648fb88..e335748 100644
--- a/home/runner/work/TensorRT/TensorRT/cpp/src/torch_tensorrt/executorch/TensorRTBackend.cpp
+++ b/tmp/changes.txt
@@ -56,8 +56,7 @@ extern const Error kRegistrationResult;

Error check_registration() {
  if (kRegistrationResult != Error::Ok) {
-    ET_LOG(
-        Error, "TensorRTBackend registration failed: %s", ::executorch::runtime::to_string(kRegistrationResult));
+    ET_LOG(Error, "TensorRTBackend registration failed: %s", ::executorch::runtime::to_string(kRegistrationResult));
  }
  return kRegistrationResult;
}
ERROR: Some files do not conform to style guidelines

[github-actions]

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/setup.py	2026-07-27 20:11:18.445428+00:00
+++ /home/runner/work/TensorRT/TensorRT/setup.py	2026-07-27 20:11:43.549449+00:00
@@ -183,12 +183,11 @@
else:
    __version__ = f"{get_base_version()}.dev0+{get_git_revision_short_hash()}"

EXECUTORCH_REQUIREMENT = "executorch>=1.3.1"
EXECUTORCH_DELEGATE_REQUIREMENT = (
-    f"torch-tensorrt-executorch-delegate=={__version__}; "
-    "platform_system == 'Linux'"
+    f"torch-tensorrt-executorch-delegate=={__version__}; " "platform_system == 'Linux'"
)
EXTRAS_REQUIRE = {
    "executorch": [EXECUTORCH_REQUIREMENT, EXECUTORCH_DELEGATE_REQUIREMENT],
    "all": [EXECUTORCH_REQUIREMENT, EXECUTORCH_DELEGATE_REQUIREMENT],
}

[github-actions]

There are some changes that do not conform to C++ style guidelines:

diff --git a/home/runner/work/TensorRT/TensorRT/cpp/src/torch_tensorrt/executorch/TensorRTBackend.cpp b/tmp/changes.txt
index 648fb88..e335748 100644
--- a/home/runner/work/TensorRT/TensorRT/cpp/src/torch_tensorrt/executorch/TensorRTBackend.cpp
+++ b/tmp/changes.txt
@@ -56,8 +56,7 @@ extern const Error kRegistrationResult;

Error check_registration() {
  if (kRegistrationResult != Error::Ok) {
-    ET_LOG(
-        Error, "TensorRTBackend registration failed: %s", ::executorch::runtime::to_string(kRegistrationResult));
+    ET_LOG(Error, "TensorRTBackend registration failed: %s", ::executorch::runtime::to_string(kRegistrationResult));
  }
  return kRegistrationResult;
}
ERROR: Some files do not conform to style guidelines

[shoumikhin]

Net: let's land this as the TensorRT-only Python runtime and treat the coalesced TRT+CUDA case as a separate follow-up that we'll drive, so it doesn't block you here. Quick context on the two ExecuTorch-side features in play, since both postdate this PR:

• Coalesced delegates: ExecuTorch can split one exported model across more than one backend, so a single .pte can have some partitions running on our TensorRTBackend and the rest on ExecuTorch's own CUDA (AOTInductor) backend. Running such a model needs BOTH backends registered in the same runtime, in one process. This path registers only TensorRTBackend (native/CMakeLists.txt builds no CUDA backend and whole-archives only ours), and activate() swaps the stock runtime out via sys.modules and refuses if the stock one was imported first, so our backend and ExecuTorch's CudaBackend can't be alive together. So a coalesced .pte can't run through here yet.

• External .ptd weights: this is a new, still-WIP ExecuTorch feature where a delegate stores its weight constants outside the .pte, in a separate .ptd file next to it (for example the CUDA/AOTInductor backend with package_constants_in_so=False emits an aoti_cuda_blob.ptd). Coalesced models rely on it for their CUDA weights. Since it's WIP, the only impact here is that load() reads just the .pte bytes (load_program(data)) with no channel for a .ptd, so a model that keeps weights outside the .pte would load with no weights. Nothing to build for it now; just don't let this path silently mishandle such a model.

Both are out of scope for a TensorRT-only runner, so to land:

1. Pin one executorch commit: keep it in MODULE.bazel.tmpl (root is generated from it) and make the README and pip install executorch==... match, so the .so is built and shipped against the same ExecuTorch.
2. Detect a .pte that needs a backend or external weights this runtime doesn't provide (a coalesced or external-.ptd program) and refuse it with a clear error, and document this as the TensorRT-only path, not the general .pte runner.
3. Make pip install self-sufficient: pin torch without the local +cuXXX suffix, and either bake an $ORIGIN rpath or document that TensorRT and CUDA must be on the library path (today it only works with a hand-set LD_LIBRARY_PATH).
4. While we're pre-publish and renames are cheap: can we name the package torch-tensorrt-executorch-runtime instead of -delegate? It ships a whole backend-enabled runtime, not just a delegate shim, so -runtime reads more accurately, and it's far easier to change now than once it's on PyPI.

Everything else I raised (external .ptd support, auditwheel, the build nonce, the CPU-copy doc note) is fine as fast-follows, and let's just confirm the aarch64/Jetson wheel actually builds and is tested. We'll drive the coalesced + .ptd follow-up separately so it stays off your critical path.

[shoumikhin]

Clarifying point 2: coalesced or external-.ptd models already fail with a hard ExecuTorch error (like "Backend CudaBackend is not registered"), just deferred to the first forward() and worded cryptically, so no silent-wrong-answer risk, so skip any .pte pre-scanner.