[IsaacLab 3.0.0 Multi-Backend] Visual Logging for Headless Runs

[mataylor-nvidia]

Description

REQ-12: Support capturing images from the visualizer when running headless such that we can capture the full or subset of environments during training to observe progress through frameworks such as tensorboard or wandb

This feature exports per step frames from simulation from full or subset of the environments.

Fixes # (OMPE-78691)

https://jirasw.nvidia.com/browse/OMPE-78691

IsaacLab/source/isaaclab_tasks/test/rendering_test_utils.py

Line 553 in 0316aea

def validate_camera_outputs(

I save the images in the same way as the tests compare goldens moving this shared logic into utils/images.py

Type of change

• New feature (non-breaking change which adds functionality)

Screenshots

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with ./isaaclab.sh --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have updated the changelog and the corresponding version in the extension's config/extension.toml file
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[greptile-apps]

Greptile Summary

This PR adds a CaptureEnvSensors gymnasium wrapper to scripts/reinforcement_learning/common.py that captures image-like sensor outputs during headless RL training and writes them to TensorBoard (per-run tagged series) or disk. All four RL framework training scripts (rl_games, rsl_rl, sb3, skrl) are updated to apply the wrapper after wrap_record_video, and a CLI argument group (--capture_env_sensors, --capture_env_sensors_length, --capture_env_sensors_interval, --capture_env_sensors_format) is added to the common argument parser.

• CaptureEnvSensors wraps any gym.Env, accesses self.unwrapped.scene.sensors, converts NHWC buffers via _to_image_tensor with per-type normalization (depth, normals, or generic min-max), and writes frames according to a burst-capture schedule (_step_count % interval < frame_count). TensorBoard output uses dataformats="NHWC" and tags each run as sensor/type/run_NNNNN to avoid overwriting across episodes.
• Test coverage in test_reinforcement_learning_common.py validates file-output scheduling, float-buffer normalization, ProxyArray access, invalid-buffer rejection, unknown-format rejection, and the wrap_sensor_capture helper, with expected filenames (run_00001_step_XXXXXXXX.png) matching the production format.

Confidence Score: 5/5

Safe to merge — the wrapper is additive, opt-in via a new CLI flag that defaults to disabled, and does not alter any existing training path.

The burst-capture scheduling logic, NHWC normalization, explicit dataformats=NHWC for TensorBoard, per-run tag scoping, and filename format are all correct. Issues flagged in earlier review rounds have been addressed.

No files require special attention.

Important Files Changed

Filename	Overview
scripts/reinforcement_learning/common.py	Adds CaptureEnvSensors wrapper and wrap_sensor_capture helper; burst-capture scheduling, NHWC normalization, TensorBoard/file output, and CLI args are all implemented correctly.
source/isaaclab/test/test_reinforcement_learning_common.py	New unit-test file; parameter names, filename format strings, and shape assertions all align with production code after fixes from prior review rounds.
scripts/reinforcement_learning/rl_games/train_rl_games.py	Adds wrap_sensor_capture after wrap_record_video; wrapper ordering is correct.
scripts/reinforcement_learning/rsl_rl/train_rsl_rl.py	Trivial one-line addition of wrap_sensor_capture in correct position.
scripts/reinforcement_learning/sb3/train_sb3.py	Trivial one-line addition of wrap_sensor_capture in correct position.
scripts/reinforcement_learning/skrl/train_skrl.py	Trivial one-line addition of wrap_sensor_capture in correct position.
source/isaaclab/changelog.d/capture-env-sensors-tests.skip	Changelog skip marker for tests-only fragment; no issues.

_{Reviews (9): Last reviewed commit: "Update scripts/reinforcement_learning/co..." | Re-trigger Greptile}

[mataylor-nvidia]

@greptileai

[mataylor-nvidia]

@greptileai

[mataylor-nvidia]

@greptileai

[mataylor-nvidia]

IsaacLab/source/isaaclab_visualizers/test/visualizer_integration_utils.py

Line 1101 in 0316aea

camera_sensor.update(dt=0.0, force_recompute=True)

without this line I get inconsistent renders

[mataylor-nvidia]

[mataylor-nvidia]

This does not seem to consistently fix this problem