feat(executorch): forward ExecuTorch lowering kwargs through save(output_format="executorch") - #4433
Conversation
7dd5c3f to
8f73ee8
Compare
|
Thanks for adding these save-level controls. This overlaps with the composable ExecuTorch export work in #4440, so I think we should merge #4440 first and then rebase this PR on top of it. #4440 centralizes engine validation and rewriting, TensorRT-first partitioning, compile-config defaults, multi-method handling, and non-destructive Edge lowering in
That avoids maintaining two TensorRT-to-Edge lowering implementations. A few additional items should be addressed during the rebase:
The feature itself is useful; this sequencing should keep |
…put_format="executorch")
torch_tensorrt.save already forwards partitioners= and backend_config= to the
ExecuTorch lowering. Forward the remaining to_edge_transform_and_lower kwargs the
same way -- pop them in save(), thread them through the three _save_as_executorch
call sites, and pass them to to_edge_transform_and_lower(...):
- constant_methods: bake constant methods (e.g. LLM-runner metadata) into the .pte
- transform_passes: run caller-supplied edge transform passes
- compile_config: override the EdgeCompileConfig. Defaults to
_check_ir_validity=False (the TRT execute_engine placeholder
graph fails edge IR validation); a caller-supplied config is
forwarded verbatim so an explicit _check_ir_validity is honored.
- generate_etrecord: generate + persist an ETRecord as "<base>_etrecord.bin" next
to the .pte, following ExecuTorch's example convention
(e.g. examples/cuda/scripts/export.py)
These kwargs only apply to output_format="executorch"; passing any of them with a
different output_format now logs an ignored-kwarg warning, matching partitioners=,
compile_specs= and backend_config=.
Tests:
- forward each kwarg through _save_as_executorch, covering an omitted compile_config
(defaults to _check_ir_validity=False) vs a caller-supplied one (forwarded
verbatim), and the ETRecord sidecar write.
- forward the kwargs through the public torch_tensorrt.save() across all three
dispatch branches: an ExportedProgram, a GraphModule with retrace=True, and a
GraphModule with retrace=False.
- a real lowering with generate_etrecord=True whose sidecar parses back into an
Inspector-consumable ETRecord via executorch.devtools.
Docs: document the lowering options, the compile_config default, the
<base>_etrecord.bin sidecar, and an ETRecord parse/Inspector example in the
ExecuTorch saving guide.
8f73ee8 to
f5c6d80
Compare
Description
torch_tensorrt.save already forwards partitioners= and backend_config= to the
ExecuTorch lowering. Forward the remaining to_edge_transform_and_lower kwargs the
same way -- pop them in save(), thread them through the three _save_as_executorch
call sites, and pass them to to_edge_transform_and_lower(...):
_check_ir_validity=False (the TRT execute_engine placeholder
graph fails edge IR validation); a caller-supplied config is
forwarded verbatim so an explicit _check_ir_validity is honored.
to the .pte, following ExecuTorch's example convention
(e.g. examples/cuda/scripts/export.py)
These kwargs only apply to output_format="executorch"; passing any of them with a
different output_format now logs an ignored-kwarg warning, matching partitioners=,
compile_specs= and backend_config=.
Docs: document the lowering options, the compile_config default, the
_etrecord.bin sidecar, and an ETRecord parse/Inspector example in the ExecuTorch saving guide.Tests
(defaults to _check_ir_validity=False) vs a caller-supplied one (forwarded
verbatim), and the ETRecord sidecar write.
dispatch branches: an ExportedProgram, a GraphModule with retrace=True, and a
GraphModule with retrace=False.
Inspector-consumable ETRecord via executorch.devtools.