NVIDIA has just rolled out some exciting new tools: the NVIDIA Omniverse libraries and NVIDIA Cosmos world foundation models (WFMs). These innovations are set to supercharge the way we develop and deploy robotics and AI solutions. With the cutting-edge NVIDIA RTX PRO Servers and NVIDIA DGX Cloud backing them up, developers can now create incredibly accurate digital twins, capture and recreate real-world environments, generate synthetic training data, and build AI agents that truly grasp the physical world.
Rev Lebaredian, who serves as the VP of Omniverse and simulation technologies at NVIDIA, shared, “Computer graphics and AI are coming together to fundamentally change the landscape of robotics. By merging AI reasoning with scalable, physically accurate simulation, we’re empowering developers to create the robots and autonomous vehicles of the future, which will revolutionize industries worth trillions of dollars.”
NVIDIA Omniverse Libraries Enhance World Composition
The newly launched NVIDIA Omniverse libraries and SDKs provide developers with robust tools for industrial AI and robotics simulation. Here are some of the standout updates: -
· Data interoperability between MuJoCo (MJCF) and Universal Scene Description (OpenUSD), allowing over 250,000 MJCF robot learning developers to effortlessly simulate robots across different platforms.
· Omniverse NuRec libraries and AI models featuring RTX ray-traced 3D Gaussian splatting for high-fidelity reconstruction of real-world scenarios.
· NVIDIA Isaac Sim 5.0 and Isaac Lab 2.2 is now available on GitHub, with NuRec neural rendering and new OpenUSD-based robot and sensor schemas to help close the simulation-to-reality gap.
The integration with popular simulation platforms is also on the rise. CARLA, the open-source autonomous vehicle simulator boasting over 150,000 developers, now incorporates NuRec rendering. Meanwhile, Foretellix, a frontrunner in AV toolchains, is embracing Omniverse Sensor RTX and Cosmos Transfer to boost synthetic data generation.
The industry is already seeing strong adoption, with companies like Boston Dynamics, Figure AI, Hexagon, RAI Institute, Lightwheel, Skild AI, and Amazon Devices & Services leveraging NVIDIA Omniverse libraries to enhance their robotics and manufacturing solutions.
Cosmos WFMs are Advancing Synthetic Data Generation
The NVIDIA Cosmos platform has surpassed 2 million downloads, empowering developers to create varied training datasets for robotics using text, image, and video prompts. The upcoming Cosmos Transfer-2 model is set to simplify the creation of photorealistic synthetic data from 3D simulation scenes or spatial inputs like depth maps and HD maps.
A streamlined Cosmos Transfer model now condenses the previous 70-step distillation process into just one step, significantly boosting speed on NVIDIA RTX PRO Servers. Early adopters such as Lightwheel, Moon Surgical, and Skild AI are already harnessing these tools to accelerate the training of physical AI models.
Cosmos Reason Brings Human-Like Understanding to AI
While vision-language models (VLMs) have made great strides in recognition tasks, they often hit a wall when it comes to multi-step reasoning. Enter NVIDIA Cosmos Reason, a 7-billion-parameter reasoning VLM that’s here to shake things up by blending prior knowledge, an understanding of physics, and a touch of common sense.
Here are some of its applications: -
· Curating and annotating data for vast and varied datasets
· Planning and reasoning for robots, breaking down complex commands into manageable steps
· Video analytics AI agents that excel in search, summarization, and root-cause analysis
NVIDIA’s robotics teams, along with partners like Uber and Magna, are already weaving Cosmos Reason into systems for autonomous driving, traffic monitoring, and industrial inspections
NVIDIA AI Infrastructure to Power Robotics Anywhere
To fuel these advancements, NVIDIA has rolled out RTX PRO Blackwell Servers, designed to support every phase of robot development, from training to simulation. Plus, NVIDIA DGX Cloud is now available through the Microsoft Azure Marketplace, providing a fully managed environment for Omniverse developers and simplifying infrastructure challenges.
Driving the Robotics Developer Ecosystem
NVIDIA is also launching new educational and collaborative initiatives:
- An OpenUSD Curriculum and Certification in collaboration with Adobe, Amazon Robotics, Autodesk, Pixar, Siemens, and others to meet the rising demand for USD expertise.
- Open-source partnerships with Lightwheel to enhance NVIDIA Isaac Lab with reinforcement learning training tools and simulation-ready robot assets.
With NVIDIA Omniverse libraries and Cosmos WFMs, developers are equipped with faster, more realistic, and smarter tools to create the next wave of robots and AI agents. From industrial manufacturing to autonomous delivery, these innovations are poised to supercharge robotics development on a global scale.
