Skip to content

Optimize Newton replicated startup paths - #6931

Closed
kellyguo11 wants to merge 2 commits into
isaac-sim:developfrom
kellyguo11:kellyg/optimize-newton-replicated-startup
Closed

Optimize Newton replicated startup paths#6931
kellyguo11 wants to merge 2 commits into
isaac-sim:developfrom
kellyguo11:kellyg/optimize-newton-replicated-startup

Conversation

@kellyguo11

Copy link
Copy Markdown
Contributor

Description

Optimize replicated-scene startup for Newton camera and non-camera tasks while preserving heterogeneous-environment support.

What changed

  • Construct Shadow Hand reorientation assets and cameras inside ReplicateSession, so backend replication sees the complete asset queue.
  • Avoid concrete USD clones for pure headless Newton and Newton Warp runs while retaining them for PhysX, RTX, GUI/offscreen output, visualizers, and explicit USD clone consumers.
  • Resolve camera prototypes through the clone plan, cache shared camera intrinsics, and avoid redundant stage traversal.
  • Import clone-external Newton USD roots directly instead of traversing the replicated environment tree from the pseudo-root.
  • Scope collision-approximation, mesh, transform, color, and heightfield work to imported roots and selected solver shapes.
  • Cache articulation target-mode resolution and reuse joint-wrench and site-frame view work.
  • Resolve semantic labels through the clone plan, reading labels once per prototype while preserving per-environment instance identities.
  • Preserve heterogeneous tasks by retaining per-source builders, source selection, collision compatibility checks, and safe fallbacks.

Dependencies: none.

Performance

Measured against upstream/develop at e3c7dd87b6 on an RTX PRO 6000 Blackwell with 4096 environments and warm asset/kernel caches. These are standardized isaaclab benchmark startup cProfile totals from launch phases through the completed first step.

Task Presets Before After Change
Shadow Hand vision newton_mjwarp,newton_renderer 38.524 s 16.152 s -22.372 s (-58.1%)
Kuka-Allegro lift camera newton_mjwarp,newton_renderer,single_camera,rgb64 29.385 s 27.831 s -1.554 s (-5.3%)
G1 rough terrain, no camera newton_mjwarp 20.118 s 12.912 s -7.206 s (-35.8%)

Environment-creation-only timing improved from 35.832 s to 13.574 s for Shadow (-62.1%), 25.097 s to 23.503 s for Kuka (-6.4%), and 18.294 s to 11.087 s for G1 (-39.4%). First-step time remained effectively unchanged for all three tasks.

Kuka improves less because its heterogeneous multi-object scene still requires per-source builders, compatibility checks, collision fallback handling, and four contact-sensor constructions. The homogeneous fast path cannot collapse those task-specific costs.

The standardized profiled totals should not be compared directly with earlier unprofiled prototype measurements such as 9.898 s because cProfile instruments Python-heavy scene traversal and the measurement boundaries differ.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Performance improvement

Validation

  • 33 focused tests passed previously on this exact commit, including Kuka heterogeneous RGB and semantic-segmentation golden tests.
  • Repeated the focused cloner, collision, articulation, and segmentation suite: 31 passed.
  • Verified the new regression tests fail against unmodified upstream/develop.
  • Isaac-Lift-KukaAllegro-Camera: RSL-RL, 16 environments, Newton MJWarp plus Newton renderer, two learning iterations completed.
  • Isaac-Reorient-Cube-Shadow-Camera-Direct: RSL-RL, 16 environments, Newton MJWarp plus Newton renderer, two learning iterations completed.
  • All three 4096-environment benchmarks completed scene creation and the first step.
  • uv run --frozen python tools/changelog/cli.py check develop: passed.
  • uv run --frozen isaaclab -f: passed.
  • The legacy source/isaaclab/test/sim/test_cloner.py requires the full Isaac Sim runtime and cannot collect in this kitless worktree. PhysX plus RTX benchmarking remains an explicit CI or reviewer validation request.

Screenshots

Not applicable.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks
  • Documentation changes are not required for this internal startup path
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package
  • My name already exists in CONTRIBUTORS.md

Construct reorientation assets inside ReplicateSession so backend
replication observes the complete asset queue without authoring every
environment first. Preserve the existing PhysX collision filtering
behavior after replication.
Avoid materializing unused USD clones for pure headless Newton runs and
resolve cameras from clone prototypes. Scope global USD import and collision
processing to relevant source roots, cache repeated articulation and sensor
work, and map segmentation labels through the clone plan.

Keep concrete USD replication for PhysX, RTX, visualizers, and heterogeneous
prototype mappings that require it.
@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Aug 6, 2026
@kellyguo11

Copy link
Copy Markdown
Contributor Author

Closing this duplicate PR; development and review will continue in #6928.

@kellyguo11 kellyguo11 closed this Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant