Skip to content

Add force-driven Newton conveyor task - #6952

Open
maxkra15 wants to merge 6 commits into
isaac-sim:developfrom
maxkra15:maximiliank/conveyor-franka-env
Open

Add force-driven Newton conveyor task#6952
maxkra15 wants to merge 6 commits into
isaac-sim:developfrom
maxkra15:maximiliank/conveyor-franka-env

Conversation

@maxkra15

@maxkra15 maxkra15 commented Aug 6, 2026

Copy link
Copy Markdown

Description

This PR adds a manager-based Newton demonstration task with two counter-rotating racetrack conveyors and a table-mounted MuJoCo Menagerie Franka. Parcels on each lane are transported by a force-based moving-surface model while the environment remains playable with the zero-action agent and ready for future MDP terms.

The conveyor driver reads Newton's reported normal contact forces, computes Coulomb-limited traction toward an analytic racetrack velocity field, and applies the resulting body wrench on the next physics step. Each belt and guardrail is one procedurally generated watertight mesh, avoiding the collision seams created by overlapping straight and curved primitives.

The task also needs to spawn authored triangle data without maintaining a task-local USD spawner. To keep that part reusable, this PR adds MeshCustomCfg and spawn_mesh_custom alongside Isaac Lab's existing generated mesh spawners. The helper preserves authored winding and subdivision, supports configurable mesh collision approximation, and authors displayColor so PreviewSurface colors remain available to kitless renderers such as Newton GL.

isaaclab_tasks now declares its in-repository isaaclab_newton runtime dependency. The preferred Newton visualizer remains optional and is configured only when isaaclab_visualizers is installed. No external dependency or lockfile changes are included.

Run the task with:

uv run isaaclab zero_agent \
  --task IsaacContrib-Conveyor-Franka-Newton-v0 \
  --num_envs 1 \
  --visualizer newton

Type of change

  • New feature (non-breaking change which adds functionality)

Screenshots

The scene was verified through a 960 x 720 Newton framebuffer render. The generated image is intentionally not committed to keep the PR focused on source and tests.

Validation

  • uv run isaaclab -f
  • uv run --extra test python -m pytest source/isaaclab_tasks/test/contrib/test_conveyor_franka_geometry.py -q (8 passed, including the no-visualizer configuration path)
  • uv run isaaclab zero_agent --task IsaacContrib-Conveyor-Franka-Newton-v0 --max_steps 360 --num_envs 2 --visualizer newton
  • 720-step single-environment parcel traversal through straight/curve transitions
  • uv run --isolated --extra test -- make -C docs current-docs

The existing Kit integration suite now includes coverage for MeshCustomCfg. It was not runnable locally because this checkout is kitless; the Newton end-to-end task run exercised the same spawner path.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with uv run isaaclab -f
  • I have made corresponding changes to the documentation/API exports
  • My changes generate no new warnings
  • I have added tests that prove the feature works
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package
  • My name already exists in CONTRIBUTORS.md

@maxkra15
maxkra15 marked this pull request as ready for review August 6, 2026 23:52
@maxkra15
maxkra15 requested a review from a team August 6, 2026 23:52
@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Aug 6, 2026
@greptile-apps

greptile-apps Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds a Newton-based demonstration environment with force-driven racetrack conveyors and reusable authored-triangle mesh spawning.

  • Registers the conveyor/Franka task and implements its scene, geometry, lifecycle, and contact-driven traction model.
  • Adds MeshCustomCfg and spawn_mesh_custom, including collision, subdivision, material, export, and test coverage.
  • Declares the Newton runtime dependency while keeping the preferred visualizer optional.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
source/isaaclab_tasks/pyproject.toml Declares the in-repository Newton runtime package and its uv source mapping, completing the previously missing required dependency.
source/isaaclab_tasks/isaaclab_tasks/contrib/conveyor_franka/conveyor_franka_env_cfg.py Defines the Newton simulation, scene assets, conveyor parameters, and an optional visualizer configuration that safely handles an absent visualizer package.
source/isaaclab_tasks/isaaclab_tasks/contrib/conveyor_franka/conveyor_franka_env.py Integrates the conveyor force driver into the manager-based environment step and reset lifecycle.
source/isaaclab_tasks/isaaclab_tasks/contrib/conveyor_franka/conveyor_force_driver.py Implements contact classification and Coulomb-limited conveyor traction using Newton and Warp data.
source/isaaclab_tasks/isaaclab_tasks/contrib/conveyor_franka/conveyor_geometry.py Generates watertight racetrack belt and guard geometry consumed by the custom mesh spawner.
source/isaaclab/isaaclab/sim/spawners/meshes/meshes.py Adds authored triangle-mesh spawning with validation, collision approximation, subdivision preservation, and kitless display colors.
source/isaaclab/isaaclab/sim/spawners/meshes/meshes_cfg.py Introduces the reusable MeshCustomCfg configuration contract for vertices, faces, collision approximation, and subdivision.
source/isaaclab_tasks/test/contrib/test_conveyor_franka_geometry.py Covers generated conveyor geometry and verifies environment configuration without the optional visualizer package.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  CLI["zero_agent / gym.make"] --> Registry["Conveyor task registration"]
  Registry --> Env["ConveyorFrankaEnv"]
  Env --> Scene["Franka, parcels, belts, and guards"]
  Geometry["Procedural racetrack geometry"] --> MeshCfg["MeshCustomCfg"]
  MeshCfg --> USD["Authored USD triangle meshes"]
  USD --> Scene
  Scene --> Newton["Newton physics step"]
  Newton --> Contacts["Reported parcel contact forces"]
  Contacts --> Driver["ConveyorForceDriver"]
  Driver --> Wrench["Coulomb-limited body wrench"]
  Wrench --> Newton
Loading

Reviews (2): Last reviewed commit: "Address conveyor review feedback" | Re-trigger Greptile

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The public custom-mesh spawner and Newton conveyor task are coherently structured, but three scoped fixes are needed before merge: limit the new displayColor behavior so existing mesh spawners do not change rendering semantics, preserve supported environment constructor keywords such as render_mode, and apply the repository-mandated copyright header to the new task files.

  • Design and architecture: The separation between procedural geometry, the Warp force driver, environment configuration, and the reusable MeshCustomCfg spawner is sound. However, renderer-independent color authoring was added to the shared mesh helper, unintentionally extending the behavior beyond custom meshes and creating inconsistent color/material handling for existing spawners.
  • API: MeshCustomCfg and spawn_mesh_custom are consistently exported and documented through the public stubs and changelog. The task environment constructor currently discards all Gym-provided keyword arguments, narrowing the base ManagerBasedRLEnv constructor contract and breaking paths that rely on render_mode; supported arguments should be forwarded.
  • Implementation: The custom-mesh validation, collision configuration, racetrack generation, and force-driver organization are internally coherent. Required corrections are to scope displayColor authoring appropriately, retain the kitless warning when material properties remain unapplied, and update the new files to the required 2022-2026 copyright header.

Minor fixes needed. Posted 3 actionable findings inline.

Automated review; human maintainers own approval decisions.

Comment thread source/isaaclab/isaaclab/sim/spawners/meshes/meshes.py Outdated
Comment thread source/isaaclab_tasks/isaaclab_tasks/contrib/conveyor_franka/__init__.py Outdated
@maxkra15

maxkra15 commented Aug 7, 2026

Copy link
Copy Markdown
Author

@greptileai check again

Add reset-state curriculum, observations, rewards, and relative joint actions for four-cube cross-conveyor transfer. Calibrate Franka contact dynamics and belt friction for stable moving-belt manipulation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant