Skip to content

Enable synchronous texture streaming for IsaacSim RTX - #6932

Draft
huidongc wants to merge 5 commits into
isaac-sim:developfrom
huidongc:disable-async-texture-streaming
Draft

Enable synchronous texture streaming for IsaacSim RTX#6932
huidongc wants to merge 5 commits into
isaac-sim:developfrom
huidongc:disable-async-texture-streaming

Conversation

@huidongc

@huidongc huidongc commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Description

Disable async texture streaming in the headless Kit app while keeping texture streaming itself enabled:

'rtx-transient'.resourcemanager.enableTextureStreaming = true
'rtx-transient'.resourcemanager.texturestreaming.async = false

Motivation: With async texture streaming enabled, Isaac RTX camera renders could vary across runs, which breaks golden-image / pixel-diff rendering correctness tests. Synchronous texture streaming restores stable pixel output.

Scope: apps/isaaclab.python.headless.kit only. Texture streaming remains on; only the async path is turned off.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Validation

Pixel determinism

Re-ran Isaac RTX rendering correctness tests (lift_kuka_hetero, lift_kuka_homo, shadow_hand; AOVs: rgb / rgba / albedo / simple_shading_diffuse_mdl / simple_shading_full_mdl) 20 times after this change.

  • All cases: 0.00% PixelDiff vs golden on every run (N=20, std=0)
  • Actual images were byte-identical across runs (same SHA256)

This confirms the setting restores deterministic Isaac RTX render output for these tests.

Performance / memory impact

Dexsuite runtime benchmark on Isaac-Lift-KukaAllegro-Camera (newton_mjwarp + isaacsim_rtx, seed 42, 1000 measured iterations) before (develop / 62e1433e) vs after (this change).

Variant Envs FPS Δ% Env create Δ% First step Δ% GPU mem peak Δ% RAM peak Δ%
homogeneous 1024 −2.98% +2.99% +0.03% +2.54% −0.12%
homogeneous 2048 +1.04% −0.52% +1.24% −1.11% +0.43%
homogeneous 4096 +0.33% +1.54% +1.25% −2.31% −0.09%
homogeneous 8192 −0.95% −1.65% −6.06% −1.51% +0.06%
heterogeneous 1024 −0.77% +1.74% +3.77% −4.58% +0.12%
heterogeneous 2048 −0.59% +1.35% +3.51% −3.38% +0.64%
heterogeneous 4096 +0.31% −0.12% +0.61% −1.97% +0.09%
heterogeneous 8192 +1.48% +0.70% +2.70% −1.01% +0.13%

Δ% = (after − before) / before × 100. Sign meaning by metric:

Metric + means means Prefer
FPS after is faster after is slower higher (+)
Env create / First step after takes longer after is quicker lower ()
GPU / RAM mem peak after uses more memory after uses less memory lower ()

Takeaways:

  • FPS impact is within run-to-run noise (~±3%; confirmed by repeating noisy develop outliers).
  • GPU memory peak is slightly lower in most configs (~1–5%).
  • Host RAM and env-creation time are essentially unchanged.

Machine:

CPU: AMD Ryzen Threadripper PRO 5975WX (32 cores / 64 threads)
GPU: NVIDIA RTX A6000.
RAM: 251 GB (~251.5 GiB)
OS: Ubuntu 22.04

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package (do not edit CHANGELOG.rst or bump extension.toml — CI handles that)
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@huidongc
huidongc requested a review from a team August 6, 2026 05:32
@huidongc
huidongc marked this pull request as draft August 6, 2026 05:33
@github-actions github-actions Bot added the isaac-sim Related to Isaac Sim team label Aug 6, 2026

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The PR adds one headless Kit default that keeps texture streaming enabled while selecting its synchronous path. The setting is colocated with the existing texture-streaming and asynchronous-rendering controls, and no actionable defect is evidenced by the patch.

  • Design and architecture: The change is scoped to apps/isaaclab.python.headless.kit and aligns the texture resource manager with the headless application's existing synchronous-rendering defaults. Other Kit application configurations remain unchanged, which is a deliberate scope boundary rather than a demonstrated defect.
  • API: No public Python API, CLI argument, symbol, or configuration dataclass changes. The behavioral compatibility surface is limited to the headless Kit default; texture streaming remains enabled, while its scheduling changes from asynchronous to synchronous.
  • Implementation: The new rtx-transient.resourcemanager.texturestreaming.async setting is placed directly beside the existing texture-streaming enablement and follows the surrounding Kit/TOML syntax. The main non-blocking tradeoff is potentially different texture load-time behavior in the headless application.

No blocking issues. No inline issue met the actionable-evidence threshold; the assessment above records the review feedback.

Automated review; human maintainers own approval decisions.

@greptile-apps

greptile-apps Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR configures the default headless IsaacLab experience to retain texture streaming while disabling its asynchronous mode.

  • Adds the synchronous texture-streaming setting to the base headless Kit configuration.
  • The setting is inherited by the headless rendering experience.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
apps/isaaclab.python.headless.kit Adds a single Kit setting that disables asynchronous texture streaming for standard headless launch paths.

Reviews (2): Last reviewed commit: "Disable async texture streaming" | Re-trigger Greptile

@huidongc
huidongc force-pushed the disable-async-texture-streaming branch from 67bc705 to cc5ed71 Compare August 6, 2026 11:22
@huidongc
huidongc marked this pull request as ready for review August 6, 2026 11:43
@huidongc huidongc changed the title Disable async texture streaming Disable async texture streaming to improve IsaacSim RTX pixel determinism across runs Aug 6, 2026
@huidongc
huidongc requested a review from ooctipus as a code owner August 6, 2026 12:34
@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Aug 6, 2026
Comment thread source/isaaclab_tasks/test/core/test_rendering_registered_tasks.py Outdated
@huidongc huidongc changed the title Disable async texture streaming to improve IsaacSim RTX pixel determinism across runs Enable synchronous texture streaming to improve IsaacSim RTX pixel determinism across runs Aug 6, 2026
@nvsekkin

nvsekkin commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

question: there are also golden image comparison tests in source/isaaclab_visualizers/test/golden_images/* those test passed without update https://github.com/isaac-sim/IsaacLab/actions/runs/31102324567/job/92630269455

strangely, the updated goldens failed validation: https://github.com/isaac-sim/IsaacLab/actions/runs/31102324567/job/92630269451#step:4:528

@huidongc

huidongc commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

question: there are also golden image comparison tests in source/isaaclab_visualizers/test/golden_images/* those test passed without update https://github.com/isaac-sim/IsaacLab/actions/runs/31102324567/job/92630269455

The scope of the PR will only affect sensor renders using Kit RTX. The visualizer part is not touched.

@huidongc

huidongc commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator Author

Synchronous texture streaming determinism fix landed on Kit 110.3.0-356733 , but we pin Isaac Sim 6.1.0-alpha.38  which is using Kit  110.3.0-351613 .

My MR is now blocked until the Isaac Sim issue is fixed and we pin newer Isaac Sim and thus newer Kit.

@huidongc
huidongc marked this pull request as draft August 7, 2026 02:54
Texture streaming is still enabled, but we will use synchronous texture streaming.
- Refresh isaacsim_rtx / default_renderer goldens for cartpole, Franka
cloth, Lift-Kuka, Shadow Hand, and registered camera tasks.
- Add Reorient Cube Shadow goldens.
- Drop the unused scene-partition fixture from the registered-tasks rendering test due to NVBUG#6566677.
@huidongc
huidongc force-pushed the disable-async-texture-streaming branch from c6cc350 to bf9b44b Compare August 7, 2026 03:40
@huidongc huidongc changed the title Enable synchronous texture streaming to improve IsaacSim RTX pixel determinism across runs Enable synchronous texture streaming for IsaacSim RTX Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

isaac-lab Related to Isaac Lab team isaac-sim Related to Isaac Sim team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants