Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
1c2c37a
Read metadata from onnx files instead of ros parameters
qiayuanl Jun 15, 2025
cdf0fa7
Merge branch 'master' of https://github.com/qiayuanl/motion_tracking_…
Jun 15, 2025
d739a00
Merge remote-tracking branch 'origin/master'
qiayuanl Jun 17, 2025
aeb7d38
Add rosbag2 into launch
qiayuanl Jun 19, 2025
1652140
Add rosbag2 into launch
qiayuanl Jun 20, 2025
faca70e
Exclude all Unitree topics in ros2 bag
qiayuanl Jun 29, 2025
1a30d2b
Fix the bug of motion first frame alignment
qiayuanl Jul 12, 2025
8adb2da
Add start_step
qiayuanl Jul 23, 2025
b600740
Update default position
qiayuanl Jul 28, 2025
e00018f
Active walking controller by default
qiayuanl Jul 31, 2025
bea8438
Change "reference" to "anchor" for consistency with the paper
qiayuanl Aug 15, 2025
5f64fa4
Provide interface for policy_path, wandb_path, and start_step for the…
qiayuanl Aug 18, 2025
abdfd4e
Get "anchor_body_name" and "body_names" from ONNX metadata.
qiayuanl Aug 22, 2025
ff31bec
Remove on_board.yaml, add ext_pos_corr user interface
qiayuanl Aug 23, 2025
f7599bc
Update README.md
qiayuanl Aug 24, 2025
10f8432
Add LICENCE
qiayuanl Aug 24, 2025
2cb1c25
Refactor MotionTrackingController to inherit from RlController and up…
qiayuanl Sep 15, 2025
d441508
Add walking_controller parameters and enable use_sim_time in launch file
qiayuanl Sep 28, 2025
491efc4
Update README to include navigation to parent directory of `src`
qiayuanl Sep 28, 2025
cfe4ec7
Update README and controllers.yaml for ROS 2 Jazzy
qiayuanl Nov 20, 2025
57b316c
Fix regex option in rosbag2 command for topic exclusion
qiayuanl Nov 21, 2025
f7841ad
Add Dockerfile and helper scripts to simplify installation and build
Renkunzhao Feb 22, 2026
6645ce8
Update README to correct directory path for container setup
Renkunzhao Feb 22, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__pycache__
19 changes: 19 additions & 0 deletions LICENCE
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
Copyright (c) 2025, Hybrid Robotics.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
167 changes: 159 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,161 @@
### HOW TO USE THIS TEMPLATE
# BeyondMimic Motion Tracking Inference

> **DO NOT FORK** this is meant to be used from **[Use this template](https://github.com/qiayuanl/legged_template_controller/generate)** feature.
[[Website]](https://beyondmimic.github.io/)
[[Arxiv]](https://arxiv.org/abs/2508.08241)
[[Video]](https://youtu.be/RS_MtKVIAzY)

1. Click on **[Use this template](https://github.com/qiayuanl/legged_template_controller/generate)**
3. Give a name to your repo, this name will be your ROS package name
(e.g. `rss25_controllers`, `koushil_controller`, **all lowercase and underscores separation for the name should be used!**)
3. Wait until the first run of CI finishes
(Github Actions will process the template and commit to your new repo)
4. Then clone your new project to the colcon workspace (e.g. `colcon_ws/src`) and happy coding!
This repository provides the inference pipeline for motion tracking policies in BeyondMimic. The pipeline is implemented
in C++ using the ONNX CPU inference engine. Model parameters (joint order, impedance, etc.)
are stored in ONNX metadata, and the reference motion is returned via the `forward()` function.
See [this script](https://github.com/HybridRobotics/whole_body_tracking/blob/main/source/whole_body_tracking/whole_body_tracking/utils/exporter.py)
for details on exporting models.

This repo also serves as an example of how to implement a custom controller using the
[legged_control2](https://qiayuanl.github.io/legged_control2_doc/) framework.

## Install and Build (docker)
The Dockerfile and helper scripts are provided to simplify installation and build.
Make sure [Docker](https://docs.docker.com/engine/install/ubuntu/) and the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) are installed first.
```bash
# Host
git clone https://github.com/HybridRobotics/motion_tracking_controller.git
cd motion_tracking_controller/docker
docker compose up -d --build
docker exec -it wbt_ws bash
```

```bash
# container
cd /wbt_ws/src/motion_tracking_controller
./scripts/colcon-config.sh Release
```
This script automatically clones and builds the required dependencies.
See [Basic Usage](#basic-usage) for usage instructions.

## Installation

### Dependencies

This software is built on
the [ROS 2 Jazzy](https://docs.ros.org/en/jazzy/Installation/Ubuntu-Install-Debs.html#ubuntu-deb-packages), which
needs to be installed first. Additionally, this code base depends on `legged_control2`.

### Install `legged_control2`

Pre-built binaries for `legged_control2` are available on ROS 2 Jazzy. We recommend first reading
the [full documentation](https://qiayuanl.github.io/legged_control2_doc/overview.html).

Specifically, For this repo, follow
the [Debian Source installation](https://qiayuanl.github.io/legged_control2_doc/installation.html#debian-source-recommended).
Additionally, install Unitree-specific packages:

```bash
# Add debian source
echo "deb [trusted=yes] https://github.com/qiayuanl/unitree_buildfarm/raw/noble-jazzy-amd64/ ./" | sudo tee /etc/apt/sources.list.d/qiayuanl_unitree_buildfarm.list
echo "yaml https://github.com/qiayuanl/unitree_buildfarm/raw/noble-jazzy-amd64/local.yaml jazzy" | sudo tee /etc/ros/rosdep/sources.list.d/1-qiayuanl_unitree_buildfarm.list
sudo apt-get update
```

```bash
# Install packages
sudo apt-get install ros-jazzy-unitree-description
sudo apt-get install ros-jazzy-unitree-systems
```

### Build Package

After installing `legged_control2`, you can build this package. You’ll also need the
`unitree_bringup` repo, which contains utilities not included in the pre-built binaries.

Create a ROS 2 workspace if you don't have one. Below we use `~/colcon_ws` as an example.

```bash
mkdir -p ~/colcon_ws/src
```

Clone two repo into the `src` of workspace.

```bash
cd ~/colcon_ws/src
git clone https://github.com/qiayuanl/unitree_bringup.git
git clone https://github.com/HybridRobotics/motion_tracking_controller.git
cd ../
```

Install dependencies automatically:

```bash
rosdep install --from-paths src --ignore-src -r -y
```

Build the packages:

```bash
colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=RelwithDebInfo --packages-up-to unitree_bringup
colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=RelwithDebInfo --packages-up-to motion_tracking_controller
source install/setup.bash
```

## Basic Usage

### Sim-to-Sim

We provide a launch file for running the policy in MuJoCo simulation.

```bash
# Load policy from WandB
ros2 launch motion_tracking_controller mujoco.launch.py wandb_path:=<your_wandb_run_path>
```

```bash
# OR load policy from local ONNX file (should be absolute or start with `~`)
ros2 launch motion_tracking_controller mujoco.launch.py policy_path:=<your_onnx_file_path>
```

### Real Experiments

> ⚠️ **Disclaimer**
> Running these models on real robots is **dangerous** and entirely at your own risk.
> They are provided **for research only**, and we accept **no responsibility** for any harm, damage, or malfunction.

1. Connect to the robot via ethernet cable.
2. Set the ethernet adapter to static IP: `192.168.123.11`.
3. Use `ifconfig` to find the `<network_interface>`, (e.g.,`eth0` or `enp3s0`).

```bash
# Load policy from WandB
ros2 launch motion_tracking_controller real.launch.py network_interface:=<network_interface> wandb_path:=<your_wandb_run_path>
```

```bash
# OR load policy from local ONNX file (should be absolute or start with `~`)
ros2 launch motion_tracking_controller real.launch.py network_interface:=<network_interface> policy_path:=<your_onnx_file_name>.onnx
```

The robot should enter standby controller in the beginning.
Use the Unitree remote (joystick) to start and stop the policy:

- Standby controller (joint position control): `L1 + A`
- Motion tracking controller (the policy): `R1 + A`
- E-stop (damping): `B`

## Code Structure

This section will be especially helpful if you decide to write your own legged_control2 controller.
For a minimal starting point, check
the [legged_template_controller](https://github.com/qiayuanl/legged_template_controller).

Below is an overview of the code structure for this repository:

- **`include`** or **`src`**
- **`MotionTrackingController`** Manages observations (like an RL environment) and passes them to the policy.

- **`MotionOnnxPolicy`** Wraps the neural network, runs inference, and extracts reference motion from the ONNX file.

- **`MotionCommand`** Defines observation terms aligned with the training code.


- **`launch`**
- Includes launch files like `mujoco.launch.py` and `real.launch.py` for simulation and real robot execution.
- **`config`**
- Stores configuration files for standby controller and state estimation params.
24 changes: 5 additions & 19 deletions config/g1/controllers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,15 @@ controller_manager:
type: legged_controllers/StandbyController
walking_controller:
type: motion_tracking_controller/MotionTrackingController
walking_controller1:
type: legged_rl_controllers/OnnxController

state_estimator:
ros__parameters:
model:
base_name: "pelvis"
six_dof_contact_names: [ "LL_FOOT", "LR_FOOT" ]
estimation:
contact:
height_sensor_noise: 1e10
position:
noise: 1e-2
topic: "/mid360"
frame_id: "mid360_link"

standby_controller:
Expand All @@ -36,11 +31,11 @@ standby_controller:
left_elbow_joint, left_wrist_roll_joint, left_wrist_pitch_joint, left_wrist_yaw_joint,
right_shoulder_pitch_joint, right_shoulder_roll_joint, right_shoulder_yaw_joint,
right_elbow_joint, right_wrist_roll_joint, right_wrist_pitch_joint, right_wrist_yaw_joint ]
default_position: [ -0.312, 0.0, 0.0, 0.669, -0.363, 0.0,
-0.312, 0.0, 0.0, 0.669, -0.363, 0.0,
default_position: [ -0.312, 0.0, 0.0, 0.669, -0.33, 0.0,
-0.312, 0.0, 0.0, 0.669, -0.33, 0.0,
0.0, 0.0, 0.0,
0.2, 1.57, 0.0, 1.57, 0.0, 0.0, 0.0,
0.2, -1.57, 0.0, 1.57, 0.0, 0.0, 0.0 ]
0.2, 0.2, 0.0, 0.6, 0.0, 0.0, 0.0,
0.2, -0.2, 0.0, 0.6, 0.0, 0.0, 0.0 ]
kp: [ 350.0, 200.0, 200.0, 300.0, 300.0, 150.0,
350.0, 200.0, 200.0, 300.0, 300.0, 150.0,
200.0, 200.0, 200.0,
Expand All @@ -54,13 +49,4 @@ standby_controller:

walking_controller:
ros__parameters:
policy:
path: "config/g1/policy.onnx"
motion:
reference_body: "torso_link"
body_names: [ 'pelvis',
'left_hip_roll_link', 'left_knee_link', 'left_ankle_roll_link',
'right_hip_roll_link', 'right_knee_link', 'right_ankle_roll_link',
'torso_link',
'left_shoulder_roll_link', 'left_elbow_link', 'left_wrist_yaw_link',
'right_shoulder_roll_link', 'right_elbow_link', 'right_wrist_yaw_link' ]
update_rate: 50 # Remove this line if using Humble!!!
63 changes: 0 additions & 63 deletions config/g1/on_board.yaml

This file was deleted.

78 changes: 78 additions & 0 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Ubuntu 24.04 image based on ROS Jazzy (supports both x86_64 and arm64)
FROM ros:jazzy-perception-noble

ENV ROS_DISTRO=jazzy

# Disable interactive prompts during apt install
ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && \
# APT / certificates / repository management / GPG keys
# (apt-transport-https is usually unnecessary on Ubuntu 22.04)
apt-get install -y --no-install-recommends \
ca-certificates gnupg lsb-release software-properties-common && \
\
# Networking and troubleshooting tools
apt-get install -y --no-install-recommends \
iputils-ping iproute2 wget curl net-tools && \
\
# Build / debug / code management tools
apt-get install -y --no-install-recommends \
build-essential cmake gdb git git-lfs vim python3-vcstool && \
\
# Common C++ math and utility libraries
apt-get install -y --no-install-recommends \
libboost-all-dev libeigen3-dev libyaml-cpp-dev && \
\
# JSON processing / code formatting / miscellaneous utilities
apt-get install -y --no-install-recommends \
jq clang-format unzip ncdu && \
\
# GUI support (kept since GUI is required; gnome-terminal is large but convenient)
apt-get install -y --no-install-recommends \
dbus-x11 gnome-terminal && \
\
# Hardware / USB / joystick support
apt-get install -y --no-install-recommends \
udev usbutils joystick && \
\
# Python toolchain (pip)
apt-get install -y --no-install-recommends \
python3-pip && \
\
git lfs install


# ROS dependencies
RUN apt-get install -y --no-install-recommends \
ros-jazzy-plotjuggler-ros ros-jazzy-pinocchio

# Install additional dependencies
# unitree_sdk2
RUN apt-get install -y --no-install-recommends \
libspdlog-dev libfmt-dev && \
\
# unitree_ros2
apt-get install -y --no-install-recommends \
ros-jazzy-rmw-cyclonedds-cpp ros-jazzy-rosidl-generator-dds-idl && \
\
# unitree_mujoco
apt-get install -y --no-install-recommends \
libglfw3-dev

# Add apt source
RUN echo "deb [trusted=yes] https://github.com/qiayuanl/legged_buildfarm/raw/noble-jazzy-amd64/ ./" | sudo tee /etc/apt/sources.list.d/qiayuanl_legged_buildfarm.list && \
echo "yaml https://github.com/qiayuanl/legged_buildfarm/raw/noble-jazzy-amd64/local.yaml jazzy" | sudo tee /etc/ros/rosdep/sources.list.d/1-qiayuanl_legged_buildfarm.list && \
echo "deb [trusted=yes] https://github.com/qiayuanl/unitree_buildfarm/raw/noble-jazzy-amd64/ ./" | sudo tee /etc/apt/sources.list.d/qiayuanl_unitree_buildfarm.list && \
echo "yaml https://github.com/qiayuanl/unitree_buildfarm/raw/noble-jazzy-amd64/local.yaml jazzy" | sudo tee /etc/ros/rosdep/sources.list.d/1-qiayuanl_unitree_buildfarm.list && \
echo "deb [trusted=yes] https://github.com/qiayuanl/simulation_buildfarm/raw/noble-jazzy-amd64/ ./" | sudo tee /etc/apt/sources.list.d/qiayuanl_simulation_buildfarm.list && \
echo "yaml https://github.com/qiayuanl/simulation_buildfarm/raw/noble-jazzy-amd64/local.yaml jazzy" | sudo tee /etc/ros/rosdep/sources.list.d/1-qiayuanl_simulation_buildfarm.list && \
sudo apt-get update && \
sudo apt-get install -y --no-install-recommends \
ros-jazzy-legged-control-base ros-jazzy-mujoco-ros2-control ros-jazzy-unitree-description ros-jazzy-unitree-systems

# Copy workspace setup script into the container
COPY workspace-config.sh /workspace-config.sh

# Use workspace-config.sh as the container entrypoint
ENTRYPOINT ["/workspace-config.sh"]
Loading