The era of robotics has arrived, as researchers and companies around the world are developing robots powered by physical AI. These physical AIs are models that can understand instructions and autonomously perform complex tasks in the real world. Multimodal large language models (LLMs) are breakthroughs that enable robots to learn, perceive, and understand the world around them, and plan how they'll act. From human demonstrations, robots can now learn the skills required to interact with the world using gross and fine motor skills.

One of the integral technologies for advancing robotics is reinforcement learning. Just as LLMs need reinforcement learning from human feedback to learn particular skills, generative physical AI can learn skills using reinforcement learning from physics feedback in a simulated world. These simulation environments, or "robot gyms," are where robots learn to make decisions by performing actions in a virtual world that obeys the laws of physics. In these environments, robots can learn to perform complex and dynamic tasks safely and quickly, refining their skills through millions of acts of trial and error.

Nvidia Omniverse is the operating system where these physical AIs can be created. Omniverse is a development platform for virtual world simulation, combining real-time physically based rendering, physics simulation, and generative AI technologies. In Omniverse, robots can learn how to autonomously manipulate objects with precision, such as grasping and handling objects, or navigate environments autonomously, finding optimal paths while avoiding obstacles and hazards. Learning in Omniverse minimizes the "sim-to-real" gap and maximizes the transfer of learned behavior.

Building robots with generative physical AI requires three key components: Nvidia AI supercomputers to train the models, Nvidia Jetson Orin and next-generation Jetson Thor robotic supercomputers to run the models, and Nvidia Omniverse where robots can learn and refine their skills in simulated worlds. Nvidia provides the platforms, acceleration libraries, and AI models needed by developers and companies, allowing them to use any or all of the stacks that suit them best. The next wave of AI is here, and robotics powered by physical AI will revolutionize industries.

Imagine a future where computers interact with us just like humans can. This is the incredible reality of digital humans. Digital humans will revolutionize industries from customer service to advertising and gaming. The possibilities are endless.

Using advanced AI models, digital humans can see, understand, and interact with us in human-like ways. The foundation of digital humans are AI models built on multilingual speech recognition and synthesis, and large language models that understand and generate natural conversation. These AI models connect to other generative AI to dynamically animate a lifelike 3D mesh of a face, and AI models that reproduce lifelike appearances enabling real-time path traced subsurface scattering to simulate the way light penetrates the skin, scatters, and exits at various points, giving skin its soft and translucent appearance.

NVIDIA Ace is a suite of digital human technologies packaged as easy-to-deploy, fully optimized microservices. Developers can integrate Ace microservices into their existing frameworks, engines, and digital human experiences. These include neotron SLM and LLM microservices to understand intent and orchestrate other models, Reva speech microservices for interactive speech and translation, and audio-to-face and gesture microservices for facial and body animation. Ace microservices run on NVIDIA's global network of NVIDIA Exel-accelerated infrastructure, delivering low-latency digital human processing to over 100 regions.

Digital humans will enable new and enhanced experiences across industries. They can serve as AI customer service agents, making interactions more engaging and personalized. They can be AI healthcare workers, checking on patients and providing timely, personalized care. They can even be AI brand ambassadors, setting the next trends in marketing and advertising. The rise of digital humans is a glimpse into the future, where AI-powered virtual beings will seamlessly integrate into our lives.

Demand for NVIDIA accelerated Computing is skyrocketing as the world modernizes traditional data centers into generative AI factories. Foxconn, the world's largest electronics manufacturer, is gearing up to meet this demand by building robotic factories with Nvidia Omniverse.

AI Factory planners use Omniverse to integrate facility and Equipment data from leading industry applications like Siemens Teamcenter, Siemens NX, and Autodesk Revit in the digital twin. They optimize floor layout and line configurations, and locate optimal camera placements to monitor future operations with Nvidia Metropolis-powered Vision AI.

Virtual integration saves planners on the enormous cost of physical change orders during construction. The Foxconn teams use the digital twin as the source of Truth to communicate and validate accurate equipment layout. The Omniverse digital twin is also the robot gym where Foxconn developers train and test Nvidia Isaac AI applications for robotic perception and manipulation, and Metropolis AI applications for Sensor Fusion.

In Omniverse, Foxconn simulates two robot AIs before deploying runtimes to Jetson computers on the assembly line. They simulate Isaac manipulator libraries and AI models for automated Optical inspection for object identification, defect detection, and trajectory planning. To transfer HGX systems to the test pods, they simulate Isaac Perceptor-powered FBOT AMRs as they perceive and move about their environment with 3D mapping and reconstruction.

With Omniverse, Foxconn builds their robotic factories that orchestrate robots running on Nvidia Isaac to build Nvidia AI supercomputers, which in turn train Foxconn robots.

Nvidia's new gaming assistant is a fascinating development that aims to transform the way gamers engage with their favorite games and applications. This AI-powered assistant is designed to be a conduit for game-specific knowledge, providing players with valuable insights and recommendations to enhance their gaming experience.

The key features of Nvidia's gaming assistant include:

1. In-Game Assistance: The assistant can answer questions about game mechanics, item locations, and optimal strategies, helping new players get up to speed and seasoned veterans explore the game's depth more effectively.

2. Context-Aware Recommendations: By understanding the context of the player's in-game actions and progress, the assistant can tailor its recommendations to the specific playthrough, suggesting the best early-game weapons, crafting materials, or next steps to take.

3. System Optimization: The assistant can also provide insights into system performance metrics, such as FPS and PC latency, and offer recommendations to optimize settings and configurations for better efficiency and responsiveness.

4. Seamless Integration: Nvidia has partnered with game studios, like Studio Wildcard for the demonstration, to integrate the assistant seamlessly into the gaming experience, making it a natural and intuitive part of the player's journey.

This gaming assistant represents a significant step forward in the integration of AI technology into the gaming industry. By providing players with personalized guidance and system optimization, the assistant has the potential to enhance the overall gaming experience, making it more accessible, engaging, and rewarding for both new and experienced players.

As the gaming landscape continues to evolve, the integration of AI-powered assistants like Nvidia's could become a defining feature of the future of gaming, transforming the way players interact with and explore their favorite virtual worlds.