Skip to content

Fix kit-less container bugs and ship OVPhysX plus all Newton viewers - #6953

Open
hujc7 wants to merge 7 commits into
isaac-sim:developfrom
hujc7:jichuanh/kitless-docker-fixes
Open

Fix kit-less container bugs and ship OVPhysX plus all Newton viewers#6953
hujc7 wants to merge 7 commits into
isaac-sim:developfrom
hujc7:jichuanh/kitless-docker-fixes

Conversation

@hujc7

@hujc7 hujc7 commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

1. Summary

  • Fixes 4 QA-reported kit-less container bugs (nvbug 6569001 / 6569031 / 6569217 / 6569864), each verified against QA's verbatim command.
  • container.py no longer reports success on a failed Docker command — it exited 0 after printing Finished building, which is why 6569864 arrived as a screenshot with no error text.
  • Kit-less image gains OVPhysX and the Viser/Rerun viewers: physics=ovphysx and --viz newton,viser,rerun now work out of the box. Image 14.3 GB → 15.4 GB (+7.7%).
  • libxrender1 added: --viz newton crashed with AttributeError: 'NoneType' object has no attribute 'XRenderFindVisualFormat'.

2. Why OVPhysX was excluded, and why it no longer needs to be

ovphysx caps packaging<24; ray (via rl[all]) needs >=24.2. Installing OVPhysX downgraded packaging to 23.2 and broke ray — a real conflict, so the exclusion was correct.

The cap is over-strict: ovphysx's entire use of the library is one from packaging.version import Version in ovphysx/api.py. A packaging>=26.0 override reconciles it, as the neighbouring overrides already do for other over-strict pins. The in-file comment claimed an OmniClient pin instead; omniclient appears nowhere in uv.lock.

3. Validation

Run against the rebuilt image with QA's verbatim commands:

nvbug Command Result
6569001 train … physics=ovphysx renderer=ovrtx rc 0, 0 tracebacks, Training time: 155.44 s
6569031 play …, empty log dir ValueError: No runs present in the directory (was a bare FileNotFoundError)
6569217 play … --viz newton,viser,rerun rc 124 (full 300 s window), 0 tracebacks; viser :8080, rerun :9090 serving
6569864 forced Docker failure exits 1, no Finished building (was exit 0)

Newton regression: presets=newton_mjwarpTraining time: 2.06 s. Build gated by in-image assertions on ovphysx, ovrtx, viser, rerun-sdk.

4. Notes

  • nvbug 6569864's reported symptom is a host docker.sock permission error, not an Isaac Lab defect — but it surfaced the silent-success bug fixed here.
  • uv pip check now reports ovphysx's declared packaging<24 as violated. That is inherent to any override; no CI job runs uv pip check.
  • Not addressed here: Dockerfile.* hard-codes useradd --uid 1000, which breaks bind-mounted source reads on hosts with a different uid and leaves the X cookie unreadable. Separate change.

5. Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Documentation update

6. Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added a changelog fragment for the touched Isaac Lab packages

hujc7 added 4 commits August 6, 2026 17:06
Every docker compose invocation ran with check=False and its return code
was never inspected, so a failed build printed
"Finished building the docker image" and exited 0. A permission or daemon
error was therefore indistinguishable from success, leaving no non-zero
exit and no error text to report.

Route build, start, stop and config through a helper that inspects the
return code and raises. copy() warns instead, since an absent artifact
directory is normal. The two probe helpers and interactive enter() keep
check=False deliberately: for them a non-zero code is the answer, or the
exit belongs to the user's shell.
The image advertised Newton physics and OVRTX rendering but shipped only
one of Newton's three viewers, and excluded OVPhysX entirely.

OVPhysX was held out over a packaging pin: it caps packaging at <24 while
ray, pulled in by rl[all], requires >=24.2. That cap is over-strict --
ovphysx's whole use of the library is a single
'from packaging.version import Version' -- so an override reconciles it,
as it already does for other over-strict pins.

viser and rerun are Newton viewers too (isaaclab_visualizers wraps
newton.viewer.ViewerViser and ViewerRerun), so an image built around
Newton now hosts all three. libxrender1 is added because pyglet's xlib
backend resolves libXrender.so.1 at import time and nothing else in the
runtime set pulls it in.

The in-build assertions are inverted and extended so a future selector
change cannot silently drop any of these again.
get_checkpoint_path documents 'Raises: ValueError: When no runs are found
in the input directory', but os.scandir raises FileNotFoundError when the
directory does not exist at all, and only IndexError was caught. Callers
saw a bare traceback naming scandir rather than the intended message.

Both cases mean the same thing to the caller: no runs.
The description listed Newton physics and OVRTX rendering and said the
OVPhysX runtime was not included, which no longer matches the image. It
also said no visualizer was selected without saying which ones are
available, so the only way to learn that --viz viser or --viz rerun would
work was to try them.

State the shipped set, and name kit as the exception since it comes from
Omniverse Kit, which this image does not contain.
@github-actions github-actions Bot added bug Something isn't working documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team infrastructure labels Aug 7, 2026
hujc7 added 3 commits August 6, 2026 17:14
Two independent breakages from the preceding commits.

test_container_profiles patches subprocess.run with a bare MagicMock, so
the return code container_interface now inspects was a mock attribute,
never equal to 0, and the new guard raised. The mocks have to model a
command that succeeded.

test_dockerfile_nonroot pinned the old install selector and asserted
'ovphysx' not in names, which is the contract the image just stopped
having. Renamed to match, and extended to cover the viewers and
libxrender1 so a future selector change fails here rather than in a
user's container.
The Dockerfile carried an eleven-line block restating the packaging/ray
argument that pyproject.toml already makes beside the override itself,
the same rationale appeared twice in adjacent pyproject blocks, and one
comment was repeated verbatim at three mock sites.

Keep each rationale in one place and let the assertions speak.
@hujc7
hujc7 marked this pull request as ready for review August 7, 2026 08:01
@hujc7
hujc7 requested a review from a team August 7, 2026 08:01

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The Docker command wrapper correctly centralizes failure propagation, but its error-reporting path breaks for commands containing Path arguments, preventing the intended warning or RuntimeError.

  • Design and architecture: Centralizing Docker invocation in _run_docker_command is appropriate, and the check=False option preserves non-fatal artifact-copy behavior. The helper must support the argument types accepted by its call sites.
  • API: The intended behavior changes are coherent: Docker build/start/stop/config failures become non-zero exits, artifact-copy failures remain warnings, and get_checkpoint_path now raises its documented ValueError for a missing log directory. However, the Docker failure contract is not reliably met when command arguments include paths.
  • Implementation: At line 140, ' '.join(cmd) assumes every command element is a string. Both copy() and config() can include Path objects, so a failed command raises TypeError while constructing the message. This particularly defeats copy(check=False), where missing artifacts are expected to produce a warning. Convert each element when formatting, such as ' '.join(str(part) for part in cmd).

Minor fixes needed. Posted 1 actionable finding inline.

The full PR diff was reviewed; some supplemental surrounding file context was omitted.

Automated review; human maintainers own approval decisions.

"""
returncode = subprocess.run(cmd, check=False, cwd=self.context_dir, env=self.environ).returncode
if returncode != 0:
message = f"Failed to {action} (exit code {returncode}). Command: {' '.join(cmd)}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Warning · Implementation — Failure message joins non-string command parts

' '.join(cmd) requires every element to be str, but copy() appends host_path (a Path) and config() appends output_yaml (a Path). On a non-zero exit these paths raise TypeError instead of emitting the intended warning or RuntimeError. This is most damaging in copy(), where a missing artifact directory returns non-zero and is explicitly meant to be tolerated via check=False. Format with ' '.join(str(part) for part in cmd).

@greptile-apps

greptile-apps Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR improves the kit-less container by shipping OVPhysX and all Newton visualizers, adds the required XRender runtime library, surfaces failed Docker commands, and normalizes missing checkpoint-log errors.

  • Expands the kit-less install to ov[all] and visualizer[all], with in-image package assertions.
  • Adds libxrender1 for the Newton GL viewer.
  • Centralizes Docker subprocess handling so required command failures abort.
  • Converts absent checkpoint log directories into the documented ValueError.
  • Regenerates the dependency lock with a packaging>=26.0 override.

Confidence Score: 5/5

The PR appears safe to merge based on the reviewed changed paths, with no concrete actionable defect established.

The container command wrapper consistently propagates required Docker failures, optional artifact copying remains explicitly non-fatal, the checkpoint error translation matches the documented contract, and the expanded kit-less dependencies are validated during image construction.

Important Files Changed

Filename Overview
docker/Dockerfile.kitless Expands the kit-less environment with OVPhysX and all visualizers, validates their presence, and adds the XRender runtime dependency.
docker/utils/container_interface.py Introduces centralized Docker command execution that raises on failures while retaining best-effort artifact copying.
source/isaaclab_tasks/isaaclab_tasks/utils/parse_cfg.py Maps an absent checkpoint log directory to the same documented no-runs ValueError used for an empty directory.
pyproject.toml Overrides the OVPhysX packaging cap with packaging 26 or newer while preserving unvalidated extra conflicts.
uv.lock Regenerates the multi-platform dependency resolution for the new override and expanded kit-less extras.
docker/test/test_container_profiles.py Updates Docker subprocess mocks to represent successful commands under the new return-code checks.
docker/test/test_dockerfile_nonroot.py Updates static Dockerfile assertions for OVPhysX, visualizers, and libxrender1.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    CLI[container.py operation] --> Helper[_run_docker_command]
    Helper --> Docker[Docker Compose or docker cp]
    Docker -->|exit 0| Success[Continue and report success]
    Docker -->|nonzero, required| Error[Raise RuntimeError]
    Docker -->|nonzero, optional artifact| Warning[Warn and continue]
    Build[Kit-less image build] --> Install[Install Newton, RL, OV, and visualizer extras]
    Install --> Assert[Assert OVPhysX, OVRTX, Viser, and Rerun packages]
    Assert --> Runtime[Kit-less runtime image]
Loading

Reviews (1): Last reviewed commit: "Merge branch 'develop' into jichuanh/kit..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working documentation Improvements or additions to documentation infrastructure isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant