NVIDIA Introduces Cosmos World Models for Physical AI Simulation
NVIDIA has released Cosmos, a platform of world foundation models and development tools designed for physical AI developers working with interactive simulation and deployment.
Physical AI developers building world models often struggle with two main hurdles: tailoring models to specific sensors, tasks, and datasets, and deploying those models reliably into production applications. NVIDIA has rolled out its Cosmos platform to address both challenges, offering a suite of world foundation models and dedicated development tools.
The platform consists of two primary components: the Cosmos Framework, which provides agent skills and recipes for training, optimizing, and running inference, and Cosmos NIM microservices, which enable GPU-accelerated serving via application APIs. Together, these tools are designed to streamline the lifecycle of physical AI models across autonomous vehicles, robotics, and surgical scenarios.
Exploring the Cosmos Framework and Simulation Workflows
The Cosmos Framework is a modular environment that supports model training, optimization, and inference. It handles multi-modal data including text, images, video, audio, and actions, and accommodates training methods ranging from LoRA parameter-efficient adaptation to full supervised fine-tuning. The framework also supports diverse reinforcement learning paths, such as DPO, GRPO, PPO, OPD, and DiffusionNFT.
For simulation, the platform introduces specific workflows like Cosmos3-Nano-Transfer-Auto and Cosmos3-Nano-Sim-Auto for autonomous driving scenarios. These workflows generate synchronized camera video and LiDAR observations using World Scenario Map controls. Other variants cater to robotics—such as Sim-Transfer and Sim-Bimanual—and complex surgical environments, where generative simulation learns visual dynamics directly from video and kinematics rather than explicitly modeling complex tissue deformation and physics.
What it means for developers
Developers can leverage agent-driven skills within the Cosmos Framework to post-train, optimize, and evaluate checkpoints across different precisions and GPU configurations. For production deployment, Cosmos NIM microservices offer optimized containers featuring quantization, speculative decoding, and KV-cache-aware routing to accelerate inference and reduce memory overhead. Additionally, developers can try top AI models cheaply through one API at https://apixoai.online.
Deployment and Accelerated Inference
Deploying models via Cosmos NIM involves selecting a GPU and model configuration, setting up the required environment on Linux, and launching the appropriate container using an NGC API key. Configurations can be tuned for either latency—prioritizing faster individual response times for interactive use—or throughput, which maximizes completed requests over time for dataset generation or concurrent users.
Source: Build World Models for Interactive Simulation with NVIDIA Cosmos — NVIDIA Technical Blog. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
