NVIDIA has recently unveiled HOVER (humanoid versatile controller), a 1.5 million parameter neural network designed to coordinate the motors of humanoid robots for locomotion and manipulation. This groundbreaking development was made possible by a diverse team of researchers from NVIDIA, Carnegie Mellon University, University of California, Berkeley, The University of Texas at Austin, and UC San Diego, led by Jim Fan, senior research manager and lead of embodied AI (GEAR Lab) at NVIDIA.
According to Fan, HOVER captures the subconscious processes involved in human movement, allowing robots to execute complex tasks without extensive programming. The model was trained in NVIDIA Isaac, a GPU-powered simulation suite that accelerates physics simulations by 10,000 times faster than real time. This allowed the model to undergo a year of virtual training in just 50 minutes of real-world time on a single GPU, making it highly efficient and easily transferable to real-world applications without requiring fine-tuning.
HOVER is capable of responding to various high-level motion instructions, including commands for head and hand poses using XR devices like the Apple Vision Pro, whole-body poses from motion capture or RGB cameras, joint angles from exoskeletons, and root velocity commands from joysticks. It also provides a unified interface for controlling robots with different input devices, making it easier to collect teleoperation data for training purposes. Additionally, HOVER integrates with an upstream Vision-Language-Action model to convert motion instructions into low-level motor signals at high frequency.
HOVER is compatible with any humanoid that can be simulated in Isaac, making it a versatile tool for bringing robots to life. This development follows NVIDIA’s earlier announcement of Project GR00T, a general-purpose foundation model for humanoid robots. Powered by GR00T, robots are able to understand natural language and mimic human movements by observing actions, allowing them to quickly learn coordination, dexterity, and other skills required to navigate, adapt, and interact effectively in the real world. With these advancements, NVIDIA is paving the way for more advanced and efficient humanoid robotics.
