Optimize Newton replicated camera startup - #6928
Draft
kellyguo11 wants to merge 2 commits into
Draft
Conversation
Construct reorientation assets inside ReplicateSession so backend replication observes the complete asset queue without authoring every environment first. Preserve the existing PhysX collision filtering behavior after replication.
Avoid materializing unused USD clones for pure headless Newton runs and resolve cameras from clone prototypes. Scope global USD import and collision processing to relevant source roots, cache repeated articulation and sensor work, and map segmentation labels through the clone plan. Keep concrete USD replication for PhysX, RTX, visualizers, and heterogeneous prototype mappings that require it.
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Optimize the full replicated-scene startup path used by the 4K Shadow Hand
camera task while preserving heterogeneous-environment support.
The change is intentionally kept in one draft PR so the cloning, camera,
Newton import, mesh/collision, articulation, sensor, and semantic-mapping
optimizations can be tested together.
What changed
ReplicateSession, so backend replication sees the complete asset queue.runs; retain concrete USD for PhysX, RTX, GUI/offscreen output, visualizers,
and explicit USD clone consumers.
intrinsics, and skip Newton-irrelevant physics validation in
FrameView.replicated environment tree from the pseudo-root.
and register only the selected solver's builder attributes.
site-frame view work.
once per prototype while preserving per-environment instance identities and
heterogeneous source selection.
Dependencies: none.
Performance
Measured against
upstream/developate3c7dd87b6on an RTX PRO 6000Blackwell with warm asset and kernel caches.
Shadow Hand camera, 4096 environments
A second post-fix run measured 15.91 s through the first step, showing the
expected small run-to-run variance. This does not yet meet the 10 s
end-to-end goal. The largest remaining fixed costs include builder finalization
(~2.46 s), solver initialization plus graph capture (~3.22 s), and the remaining
model/callback/reset work around them.
Kuka-Allegro lift camera, 4096 heterogeneous environments
Type of change
Validation
articulation caching, scoped global/collision imports, clone-plan semantic
mapping, and Kuka heterogeneous rendering.
upstream/develop.semantic segmentation.
Isaac-Lift-KukaAllegro-Camera: RSL-RL, 16 environments, Newton MJWarp +Newton renderer, two learning iterations completed.
Isaac-Reorient-Cube-Shadow-Camera-Direct: RSL-RL, 16 environments, NewtonMJWarp + Newton renderer, two learning iterations completed.
uv run --frozen python tools/changelog/cli.py check develop: passed.uv run --frozen isaaclab -f: passed.worktree because the full Isaac Sim/Kit runtime is not installed. This remains
an explicit CI/reviewer validation request.
Screenshots
Not applicable.
Checklist
source/<pkg>/changelog.d/for every touched packageCONTRIBUTORS.mdor my name already exists there