AI agents no longer just write code — now they control robots too

Until now, we've known agent AI mainly in its software form: Claude Code writes pull requests, ChatGPT generates text, Gemini searches documents. But NVIDIA is now opening up a completely new front. Its open-source agent tools for physical AI allow AI agents to manage the complex development processes behind robots, self-driving cars, or visual inspection in manufacturing.

"AI agents are revolutionizing software development, and this shift is now coming to physical AI — expanding into systems that will transform transportation, manufacturing, healthcare, and robotics," said Jensen Huang, NVIDIA founder and CEO, during the announcement at GTC Taipei.

In practice, this means developers no longer need to manually configure simulations, generate training data, or fine-tune models for specific hardware — these tasks are taken over by AI agents that understand NVIDIA's tools and orchestrate them autonomously.

Five platforms, one agent ecosystem

NVIDIA has opened its complete physical AI stack for agent control. Specifically, this involves five key platforms:

• NVIDIA Cosmos — world foundation models for physical reasoning and generating realistic scenarios
• NVIDIA Omniverse — libraries for simulation and creating digital twins at industrial scale
• NVIDIA Isaac — simulation tools for robotics and robot machine learning
• NVIDIA Metropolis — platform for Vision AI, i.e. visual inspection and image analysis
• NVIDIA Jetson — platform for edge AI, i.e. AI running directly on devices in the field

These libraries, models, and frameworks are now being transformed into agent-callable tools — i.e. tools that any coding agent can address and utilize. In other words: Claude Code, OpenAI's Codex, or open-source agents can now directly call NVIDIA functions for simulation, data generation, or model training.

NemoClaw and OpenShell — safety first

Alongside the agent skills, NVIDIA also introduced two tools for secure deployment:

NVIDIA NemoClaw is a blueprint for building and deploying autonomous agents that use physical AI skills. It acts as a protective layer ensuring the agent doesn't do anything unexpected. NVIDIA OpenShell then adds policy-based governance — whether the agent runs locally or in the cloud.

This is particularly important for European companies that must comply with the EU AI Act. The ability to define security policies and run agents on their own hardware gives companies the control that regulation demands.

Concrete results: 67% faster training, 17% better detection

These aren't theoretical demos — initial deployments are already delivering measurable results:

• Pegatron reduced the time for training and deploying visual inspection models by 67% thanks to synthetic data generated using the new Defect Image Generation skill.
• Delta Electronics used the same skill to detect excess soldering on metal busbars and improved detection rate by 17%.
• Inventec deployed its Observation Agent pipeline and shortened defect data collection for laptop chassis manufacturing by 30%.
• Foxconn, together with DeepHow, increased first pass yield by approximately 3% thanks to early error detection.

In the autonomous vehicle space, Li Auto, Afari, and DeepRoute.ai are generating over 1,000 neural reconstructions and more than 300,000 simulations per day using NVIDIA Omniverse NuRec.

Healthcare: robotic assistants in hospitals

For the very first time, NVIDIA is showing detailed deployment of physical AI in hospitals. Foxconn is scaling its Nurabot robot across several hospitals and long-term care facilities in Taiwan — the robot assists with patient care. Foxconn also introduced a new Scrub Nurse Collaborative Robot, which optimizes workflows in operating rooms.

Compal is developing the PolyMedX platform, which connects simulation, AI, and real-world operations for hospital robot orchestration.

For European healthcare, this is a signal of where development is heading — robotic assistance during surgeries or elderly care is an area where even Czech hospitals may benefit from these technologies in the future.

Who's already riding the NVIDIA stack

The list of partners already using NVIDIA's physical AI is impressive:

• Manufacturing and industry: TSMC, Siemens, Cadence, Dassault Systèmes, Synopsys, PTC, SK hynix
• Robotics: 1x, Agile Robots, Agility, FieldAI, Hexagon Robotics, NEURA Robotics, Skild AI, Universal Robots
• Cloud: Microsoft, CoreWeave, and Nebius are integrating agent skills into their cloud services

"When agents can directly use NVIDIA's libraries, models, and frameworks, physical AI development will accelerate at a pace that allows developers to build the robots, autonomous vehicles, and industrial systems of the future at incredible speed," Huang added.

What this means for the Czech Republic and Europe

For Czech companies and developers, the opening of these tools brings two key advantages:

1. Open source means access for everyone. The tools are freely available on GitHub and via skills.sh. Czech startups or research teams don't have to pay licensing fees — just download and start using with any coding agent.

2. Synthetic data and simulation save money and time. The ability to generate synthetic training data and test robots in simulation before deploying them in real-world operations dramatically reduces development costs. This is crucial for smaller European companies that can't afford expensive physical testing.

Moreover, the Czech Republic has a strong industrial base — from automotive to mechanical engineering to electronics. Tools like Metropolis for visual inspection or Isaac for robotics are directly usable in Czech factories. And with the Czech AI Factory in Ostrava, which provides computing capacity for AI projects, there is also infrastructure on which these models can be trained.

How to get started

NVIDIA has published not only the tools themselves but also Physical AI Launchables on the NVIDIA Brev platform — pre-configured environments that include agent skills and tools for faster synthetic data generation. Three key skills — Neural Reconstruction, Video Augmentation, and Defect Image Generation — can be tried out immediately.