Deploying learned motion tracking policies on legged robots often involves juggling multiple configuration files, runtime parameters, and complex middleware setups. The motion_tracking_controller repository tackles this head-on by embedding critical robot control parameters directly into the ONNX model metadata, streamlining deployment and keeping the policy self-contained. This approach simplifies the integration of reinforcement learning (RL) policies with real hardware and simulation environments.
what motion_tracking_controller does and its architecture
motion_tracking_controller is a C++ package designed to deploy RL-trained motion tracking policies on legged robots, part of the broader BeyondMimic project. It provides a complete inference pipeline that runs RL policies trained in simulation and applies them on real robots like the Unitree quadruped.
Under the hood, the system is built on ROS 2 Jazzy and integrates tightly with the legged_control2 framework, which handles low-level robot control abstractions. The package implements a custom controller that separates concerns clearly:
- MotionTrackingController: Manages observation collection and overall control loop logic.
- MotionOnnxPolicy: Handles neural network inference using the ONNX CPU runtime.
- MotionCommand: Defines the observation terms and commands sent to actuators.
One of the notable architectural decisions is embedding essential robot parameters such as joint order and impedance gains directly into the ONNX model’s metadata. This eliminates the need for separate configuration files, reducing the risk of mismatches and simplifying deployment.
The package supports both sim-to-sim testing using the MuJoCo physics engine and sim-to-real deployment on the Unitree robot. It also includes joystick-driven controller switching, allowing flexible testing and operation.
key technical strengths and tradeoffs
Embedding control parameters inside ONNX model metadata is an elegant solution that keeps the RL policy and its robot-specific parameters tightly coupled. This reduces DX (developer experience) friction when deploying policies across different robots or setups. It also enforces a convention-over-configuration pattern, minimizing external dependencies.
The modular separation between observation management, inference, and command generation makes the codebase easier to maintain and extend. The ROS 2 Jazzy base ensures compatibility with modern robotic middleware, while the legged_control2 framework provides a solid abstraction layer for robot-specific control.
However, this design comes with tradeoffs. The reliance on ROS 2 Jazzy and legged_control2 means users must be comfortable with these ecosystems, which can have a steep learning curve if unfamiliar. The ONNX CPU runtime is chosen for portability and simplicity, but inference speed might not match GPU-accelerated setups, which could matter for more complex models or faster control loops.
The code quality is solid, with clear separation of concerns and well-documented components. The joystick-based controller switching is a practical touch that facilitates real-world testing without code changes.
installation and quick start
dependencies
This software is built on the ROS 2 Jazzy, which needs to be installed first. Additionally, this code base depends on legged_control2.
install legged_control2
Pre-built binaries for legged_control2 are available on ROS 2 Jazzy. We recommend first reading the full documentation.
Specifically, for this repo, follow the Debian Source installation. Additionally, install Unitree-specific packages:
# Install packages
sudo apt-get install ros-jazzy-unitree-description
sudo apt-get install ros-jazzy-unitree-systems
build package
After installing legged_control2, you can build this package. You’ll also need the unitree_bringup repo, which contains utilities not included in the pre-built binaries.
Create a ROS 2 workspace if you don’t have one. Below we use ~/colcon_ws as an example.
mkdir -p ~/colcon_ws/src
Clone two repos into the src of workspace.
cd ~/colcon_ws/src
git clone https://github.com/qiayuanl/unitree_bringup.git
git clone https://github.com/HybridRobotics/motion_tracking_controller.git
cd ../
Install dependencies automatically:
rosdep install --from-paths src --ignore-src -r -y
Build the packages:
colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=RelwithDebInfo --packages-up-to unitree_bringup
colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=RelwithDebInfo --packages-up-to motion_tracking_controller
source install/setup.bash
basic usage
The repo provides a launch file to run the policy in MuJoCo simulation for sim-to-sim testing. This allows you to validate the RL policy behavior before deploying on real hardware.
verdict
motion_tracking_controller is a well-structured, practical implementation for deploying RL-trained motion tracking policies on legged robots within the ROS 2 ecosystem. Its key strength lies in embedding robot control parameters inside ONNX metadata, which is a neat architectural choice reducing configuration overhead and deployment errors.
The project is ideal for roboticists and researchers working with ROS 2 Jazzy and the legged_control2 framework who want to bridge simulation-trained policies to real robot control. It assumes familiarity with ROS 2 and some robotics middleware, so it might be challenging for those new to these tools.
While the ONNX CPU inference ensures portability, users needing ultra-low latency might need to consider GPU inference or other acceleration methods.
Overall, this repo offers a clear, maintainable approach to a tricky problem in robotics — deploying learned policies cleanly and reliably.
→ GitHub Repo: HybridRobotics/motion_tracking_controller ⭐ 490 · C++