Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
148 changes: 81 additions & 67 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,31 @@
![CMake Badge](https://github.com/flexivrobotics/flexiv_sim_plugin/actions/workflows/cmake.yml/badge.svg)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)

A middleware plugin to connect Flexiv Elements Studio to any external simulator so that the Flexiv robots in this simulator are controlled by the same force-torque controller used by the real robots.

## Environment Compatibility

| **OS** | **Platform** | **C++ compiler kit** | **Python interpreter** |
| --------------------- | ------------ | -------------------- | ---------------------- |
| Linux (Ubuntu 20.04+) | x86_64 | GCC v9.4+ | 3.10, 3.11, 3.12 |

## Tested External Simulators
## Overview

Flexiv Sim Plugin is a C++/Python library that bridges **Flexiv Elements Studio** and any external physics simulator, so that simulated Flexiv robots are driven by the same high-performance force-torque controller used on real robots — without requiring you to deal with the underlying IPC.

```
┌──────────────────────────────────────────┐
│ User Program with RDK Client │
│ (send robot commands) │
└────────────────────┬─────────────────────┘
│ RDK API
┌──────────────────────────────────────────┐
│ Flexiv Elements Studio │
│ (simulated controller + RDK Server) │
└────────────────────┬─────────────────────┘
│ Flexiv Sim Plugin API
┌────────────────────────────────────────────┐
│ External Simulator |
Comment thread
bsoe-flexiv marked this conversation as resolved.
│ (Flexiv Sim Plugin API + External sim API) │
└────────────────────────────────────────────┘
```

## Supported External Simulators

The following external simulators are tested and known to work with Flexiv Sim Plugin:

Expand All @@ -22,50 +38,53 @@ In theory, any simulator that meets the following criteria should work:

1. Has a C++ or Python interface.
2. Provides joint positions and velocities of the simulated robot.
3. The joints of the simulated robot are actuated by torque.
3. Actuates the joints of the simulated robot by torque.

## Demos

Below are some demos where Flexiv Elements Studio is connected to NVIDIA Isaac Sim to simulate applications where force control is required. The full demo videos can be found [here](https://github.com/flexivrobotics/isaac_sim_ws?tab=readme-ov-file#demos).
## Environment Compatibility

### Tower of Hanoi
| **OS** | **Platform** | **C++ compiler kit** | **Python interpreter** |
| --------------------- | ------------ | -------------------- | ---------------------- |
| Linux (Ubuntu 20.04+) | x86_64 | GCC v9.4+ | 3.10, 3.11, 3.12 |

![Tower of Hanoi](doc/demo/tower_of_hanoi.gif)

### Peg-in-hole
## Quick Start - Python

![Peg-in-hole](doc/demo/peg_in_hole.gif)
### Install the Python package

### Dual-robot polish
On all supported platforms, the Python package of Sim Plugin for a specific Python version can be installed using the `pip` module:

![Dual-robot polish](doc/demo/dual_arm_polish.gif)
python3 -m pip install flexivsimplugin

## Flexiv Elements Studio Setup
NOTE: replace python3 with a specific version (e.g. python3.10) if your default python3 is not one of the versions listed in the table above.

### Install Elements Studio
### Use the installed Python package

1. Prepare a Ubuntu 22.04 computer, all operations below are done on this computer.
2. [Contact Flexiv](https://www.flexiv.com/contact) to obtain the installation package of Elements Studio.
3. Extract the package to a non-root directory.
4. Install Elements Studio:
After the ``flexivsimplugin`` Python package is installed, it can be imported from any Python script. Test with the following commands in a new Terminal, which should start Flexiv Sim Plugin:

bash setup_FlexivElements.sh
python3
import flexivsimplugin
node = flexivsimplugin.UserNode("Rizon4-123456")
print("Connected = ", node.connected())

5. Switch physics engine from the default built-in to external:
The program should print some info messages and "Connected = False" at the end.

bash switch_physics_engine.sh
### Run the example Python script

Select *External* or *Isaac Sim* when prompted.
An example script that mocks an external simulator is provided and can be used to test the plugin with the following steps:

### Create a simulated robot in Elements Studio
1. Setup and run Flexiv Elements Studio simulation. See the relevant instructions at the end of this README.

2. Start the mock program:

cd flexiv_sim_plugin/example_py
python3 -m pip install spdlog
python3 ./mock_external_simulator.py [robot_serial_number]

NOTE: the robot serial number provided to the program is the same one you noted down when creating the simulated robot in Flexiv Elements Studio.

3. Wait for the connection to establish. If the connection is successful, you should see the visualized robot in Elements Studio moving every joint back and forth. NOTE: a software error should occur in Elements Studio which is expected because the mock external simulator did not close the loop by applying the calculated joint torques command to the simulated robot in it. This won't happen to real external simulators.

1. Start Flexiv Elements Studio from the application menu.
2. In the Robot Connection window, select *Simulator*, and click *CREATE*.
3. Choose "Create according to the selected robot type" and select one from the list, then click *CONFIRM*. A new simulated robot will be added to the simulator list.
4. Toggle on the *Connect* button for the newly added one, then wait for loading.
5. When loading is finished, you'll see a robot at its upright pose, with an "Exception" error at the bottom right corner. This is expected because the external simulator is not started yet. But if you see a normally operating robot, that again means you are running the wrong version of Elements Studio that only supports the built-in physics engine.
6. At the bottom of the window, click on the small robot icon with a "SIM" tag on it, then a small window will pop up, note down the displayed robot serial number.
7. In the same small pop-up window, click *CHANGE CONNECTION*, then toggle off the *Connect* button to close the simulated robot. We will restart it later. Note that you do NOT need to close the whole Elements Studio program.

## Quick Start - C++

Expand All @@ -86,6 +105,7 @@ Below are some demos where Flexiv Elements Studio is connected to NVIDIA Isaac S
The following steps are identical on all supported platforms.

1. Choose a directory for installing the C++ library of Sim Plugin and its dependencies. This directory can be under system path or not, depending on whether you want Sim Plugin to be globally discoverable by CMake. For example, a new folder named ``sim_plugin_install`` under the home directory.

2. In a new Terminal, run the provided script to compile and install all dependencies to the installation directory chosen in step 1:

cd flexiv_sim_plugin/thirdparty
Expand Down Expand Up @@ -119,56 +139,50 @@ NOTE: ``-D`` followed by ``CMAKE_PREFIX_PATH`` tells the user project's CMake wh

An example program that mocks an external simulator is provided and can be used to test the plugin with the following steps:

1. Start the mock program:
1. Setup and run Flexiv Elements Studio simulation. See the relevant instructions at the end of this README.

2. Start the mock program:

cd flexiv_sim_plugin/example/build
./mock_external_simulator [robot_serial_number]

NOTE: the robot serial number provided to the program is the same one you noted down when creating the simulated robot in Flexiv Elements Studio.

2. Go back to Elements Studio, then restart the exited simulator by toggling ON the *Connect* button.
3. Wait for the connection to establish. If the connection is successful, you should see the visualized robot in Elements Studio moving every joint back and forth. NOTE: a software error should occur in Elements Studio which is expected because the mock external simulator did not close the loop by applying the calculated joint torques command to the simulated robot in it. This won't happen to real external simulators.

## Quick Start - Python

### Install the Python package

On all supported platforms, the Python package of Sim Plugin for a specific Python version can be installed using the `pip` module:

python3.x -m pip install flexivsimplugin

NOTE: replace `3.x` with a specific Python version.

### Use the installed Python package
## API Documentation

After the ``flexivsimplugin`` Python package is installed, it can be imported from any Python script. Test with the following commands in a new Terminal, which should start Flexiv Sim Plugin:
The API documentation can be generated using Doxygen. For example, on Linux:

python3.x
import flexivsimplugin
node = flexivsimplugin.UserNode("Rizon4-123456")
print("Connected = ", node.connected())
sudo apt install doxygen-latex graphviz
cd flexiv_sim_plugin
doxygen doc/Doxyfile.in

The program should print some info messages and "Connected = False" at the end.
Open any html file under ``flexiv_sim_plugin/doc/html/`` with your browser to view the doc.

### Run the example Python script

An example script that mocks an external simulator is provided and can be used to test the plugin with the following steps:
## Flexiv Elements Studio Setup

1. Start the mock program:
### Install Elements Studio on Ubuntu

cd flexiv_sim_plugin/example_py
python3.x ./mock_external_simulator.py [robot_serial_number]
1. [Contact Flexiv](https://www.flexiv.com/contact) to obtain the installation package of Elements Studio.
2. Extract the package to a non-root directory.
3. Install Elements Studio:

NOTE: the robot serial number provided to the program is the same one you noted down when creating the simulated robot in Flexiv Elements Studio.
bash setup_FlexivElements.sh

2. The remaining steps are the same as documented in [Run the example C++ program](#run-the-example-c-program).
4. Switch physics engine from the default built-in to external:

## API Documentation
bash switch_physics_engine.sh

The API documentation can be generated using Doxygen. For example, on Linux:
Select *External* when prompted.

sudo apt install doxygen-latex graphviz
cd flexiv_sim_plugin
doxygen doc/Doxyfile.in
### Create a simulated robot in Elements Studio

Open any html file under ``flexiv_sim_plugin/doc/html/`` with your browser to view the doc.
1. Start Flexiv Elements Studio from the application menu.
2. In the Robot Connection window, select *Simulator*, and click *CREATE*.
3. Choose "Create according to the selected robot type" and select one from the list, then click *CONFIRM*. A new simulated robot will be added to the simulator list.
4. Toggle on the *Connect* button for the newly added one, then wait for loading.
5. When loading is finished, you'll see a robot at its upright pose, with an "Exception" error at the bottom right corner. This is expected because the external simulator is not started yet. But if you see a normally operating robot, that again means you are running the wrong version of Elements Studio that only supports the built-in physics engine.
6. At the bottom of the window, click on the small robot icon with a "SIM" tag on it, then a small window will pop up, note down the displayed robot serial number.
7. Toggle the virtual motion bar slider button to the "auto" position. In resulting popup window, select "AUTO (REMOTE)"
Binary file removed doc/demo/dual_arm_polish.gif
Binary file not shown.
Binary file removed doc/demo/peg_in_hole.gif
Binary file not shown.
Binary file removed doc/demo/tower_of_hanoi.gif
Binary file not shown.
Loading