Intrinsic and NVIDIA Collaborate to Advance Autonomous Robotic

Updated: May 06 2024 18:44

In an exciting development for the robotics industry, Intrinsic, a software and AI robotics company that spun out of Alphabet, has announced a collaboration with NVIDIA to integrate their AI and Isaac platform technologies. This partnership aims to push the boundaries of autonomous robotic manipulation by combining state-of-the-art dexterity and modular AI capabilities for robotic arms with NVIDIA's powerful foundation models and GPU-accelerated libraries.


NVIDIA's Isaac Manipulator, unveiled in March at GTC 2024, is a game-changer for industrial automation companies. It provides a collection of foundation models and modular GPU-accelerated libraries that enable the development of scalable and repeatable workflows for dynamic manipulation tasks. By accelerating AI model training and task reprogramming, Isaac Manipulator can significantly speed up path planning by up to 80x. Among early ecosystem partners are Yaskawa, Universal Robots, a Teradyne company, PickNik Robotics, Solomon, READY Robotics and Franka Robotics.


The collaboration between Intrinsic and NVIDIA leverages the power of foundation models, which are based on a transformer deep learning architecture. These models, trained on vast datasets, allow neural networks to learn by tracking relationships in data. They can process and understand sensor and robot information in a way that rivals the capabilities of ChatGPT for text. This breakthrough enables unprecedented robot perception and decision-making, including zero-shot learning, where robots can perform tasks without prior examples.

Enabling a robotic system to understand the next task and the physical objects involved requires a real-time, accurate, and semantic understanding of the environment. AI-enabled vision systems make intelligent robotic systems possible. Intrinsic have developed a foundation model, pre-trained with 130,000+ objects, that can perform one shot object and pose detection for a variety of objects in a few seconds. This is also a universal skill, meaning it can work well with many different objects and can contend with variables in the environment, including different cameras, lighting, objects, and orientations. The model is fast, generalized, and accurate.



NVIDIA's recent introduction of Project GROOT, a foundation model for humanoids, further highlights the potential of this technology. GR00T, which stands for "Generalist Robot 00 Technology," provides robotics developers with a generative AI platform to begin implementing large language models (LLMs) in humanoid robotics. Other companies, such as Covariant and Electric Sheep, are also developing foundation models for their respective industrial picking and outdoor landscaping robots.


Robots powered by GR00T will be designed to understand natural language and emulate movements by observing human actions — quickly learning coordination, dexterity and other skills in order to navigate, adapt and interact with the real world. Jensen Huang, founder and CEO of NVIDIA, demonstrated several such robots completing a variety of tasks at the GTC Keynote.


Wendy Tan White, CEO of Intrinsic, emphasized the profound impact of foundation models on the broader robotics industry. She stated, "For the broader industry, our work with NVIDIA shows how foundation models can have a profound impact, including making today's processing challenges easier to manage at scale, creating previously infeasible applications, reducing development costs, and increasing flexibility for end users."


One of the key challenges in robotics has been grasping, which has been time-consuming, expensive to program, and difficult to scale. However, the collaboration between Intrinsic and NVIDIA has demonstrated the potential to overcome these hurdles. Using NVIDIA Isaac Sim on the NVIDIA Omniverse platform, Intrinsic generated synthetic data for vacuum grasping using computer-aided design models of sheet metal and suction grippers. This allowed them to create a prototype for their customer Trumpf Machine Tools, a leading maker of industrial machine tools.

The prototype utilizes Intrinsic Flowstate, a developer environment for AI-based robotics solutions, for visualizing processes, associated perception, and motion planning. By incorporating Isaac Manipulator into the workflow, grasp poses and CUDA-accelerated robot motions can be generated and evaluated in simulation using Isaac Sim before being deployed in the real world with the Intrinsic platform. Below video visualizing the execution of a grasping skill, based on an Nvidia foundation model, using Intrinsic Flowstate:


Deepu Talla, vice president of robotics and edge computing at NVIDIA, highlighted the potential of AI foundation models in programming a variety of robot configurations that can generalize and interact with diverse objects in real-world environments. He stated, "As the collaboration between Intrinsic and NVIDIA deepens, we will be able to help companies scale and automate their industrial manufacturing operations."

It can be optimized on real hardware With just a few clicks, transfer solution from simulation to real hardware and test performance.


The collaboration between Intrinsic and NVIDIA represents a significant step forward in the advancement of autonomous robotic manipulation. By combining cutting-edge AI technologies, foundation models, and GPU-accelerated libraries, this partnership has the potential to revolutionize industrial manufacturing operations and pave the way for more intelligent, flexible, and scalable robotic solutions.