Stanford University TWIST System Brings Natural Human Movement to Robots

AI Summary: Researchers from Stanford University and Simon Fraser University have developed TWIST (Teleoperated Whole-Body Imitation System), a novel system enabling humanoid robots to mimic diverse human movements with unprecedented naturalism and coordinated whole-body control. Unlike previous methods limited to partial or task-specific imitation, TWIST uses a motion capture system and a two-stage teacher-student neural network training approach to translate complex human actions into balanced robot movements.

May 16 2025 08:29
A team of researchers from Stanford University and Simon Fraser University has developed TWIST (Teleoperated Whole-Body Imitation System) that allows humanoid robots to mimic human movements with unprecedented naturalism and coordination. The system enables a single neural network controller to track and replicate a wide range of human movements in real time, from walking and manipulating objects to dancing and playing soccer.

Unlike previous approaches that often separated upper and lower body control or focused only on specific tasks like walking, TWIST represents the first system that enables coordinated whole-body control across diverse movements.



At its core, TWIST works by capturing human motion using a motion capture (MoCap) system, then translating those movements into commands that a humanoid robot can follow. This translation, called "retargeting," is necessary because robots have different proportions and capabilities than humans.


The innovation comes from how the system trains its controller—the brain that converts human movements into balanced robot movements. TWIST uses a two-stage approach:

• First, a "teacher" policy is trained using reinforcement learning with access to future motion frames, allowing it to anticipate and plan movements smoothly.
• Then, a "student" policy is trained using both reinforcement learning and behavior cloning, relying only on current data that would be available during real-world operation.

This approach allows the robot to move smoothly and naturally while maintaining balance—a significant challenge for bipedal robots.


Previous robot teleoperation systems have faced several limitations. Some used camera-based pose estimation, which struggles with accuracy. Others relied on VR keypoints that only capture upper-body movements. Many systems separated upper and lower body control entirely, making coordinated whole-body tasks impossible.

TWIST overcomes these limitations with several key innovations:

• Use of high-quality motion capture data for accurate whole-body tracking
• A teacher-student framework that allows the robot to learn smooth motions without having to see the future during deployment
• Incorporation of both online and offline motion data in training
• Joint optimization of 3D positions and orientations to improve retargeting quality
• End-effector perturbation training to help the robot learn to apply force effectively

The result is a system capable of performing tasks that require whole-body coordination, such as crouching down to lift a box, kicking a ball, or opening a door with a foot while holding objects in both hands.


The researchers demonstrated TWIST on a Unitree G1 humanoid robot with 29 degrees of freedom. The videos of the system in action show the robot performing a remarkable range of movements:

• Whole-Body Manipulation: Lifting boxes from the ground, opening refrigerator doors, pushing furniture
• Legged Manipulation: Kicking footballs, moving objects with feet
• Locomotion: Side-stepping, walking backward, crouching to cross obstacles
• Expressive Motion: Dancing, boxing, and other expressive movements

The research uncovered several important insights about robot movement:

• First, combining reinforcement learning with behavior cloning (RL+BC) achieves better results than either approach alone. Pure reinforcement learning often results in foot sliding, while behavior cloning struggles with unseen motions.
• Second, even a small amount of in-house motion capture data significantly improves performance on unseen motions. This is because real-world motion capture contains imperfections and noise that the system must learn to handle.
• Third, training with end-effector perturbations helps the robot learn to apply force effectively, which is crucial for tasks like lifting objects.

Despite its impressive capabilities, TWIST still faces some limitations:

• The system experiences a total teleoperation delay of approximately 0.9 seconds—with 0.7 seconds for generating tracking goals and 0.2 seconds for policy inference.
• The researchers also note challenges with visual occlusions during teleoperation, lack of tactile feedback, and motor overheating after 5-10 minutes of continuous operation, especially during demanding tasks like crouching.

The development of TWIST represents a significant step toward more capable and versatile humanoid robots. By enabling robots to perform coordinated whole-body movements similar to humans, the system opens the door to robots that can handle complex household tasks and operate in environments designed for people.

We want to focus on building a strong whole-body teleoperation system first, as there is not such a system yet. We will study how to learn visuomotor policies next with our system.

This progression suggests that TWIST could eventually lead to robots that not only move like humans but also perceive and interact with the world more naturally.

The ability for robots to perform coordinated whole-body movements is a fundamental requirement for developing general-purpose robotic intelligence. With systems like TWIST, we're getting closer to robots that can truly assist humans in everyday environments.

Imagine robots that can help elderly individuals with mobility challenges, assist in disaster relief operations requiring dexterous movement through difficult terrain, or simply handle household chores that require coordinated movement, like taking out the garbage while carrying groceries.

Perhaps most significantly, they aim to develop autonomous capabilities based on the teleoperation system: "We want to focus on building a strong whole-body teleoperation system first, as there is not such a system yet. We will study how to learn visuomotor policies next with our system."

By bridging the gap between human and robot movement, TWIST brings us one step closer to robots that can seamlessly integrate into human spaces and work alongside us effectively.

Research Paper: TWIST: Teleoperated Whole-Body Imitation System
Github: https://github.com/YanjieZe/TWIST