Google's Genie 4 adds real-time audio to its AI-generated worlds

Google DeepMind released Genie 4, its newest world model, as a research preview on Tuesday. The system builds the most photorealistic and interactive simulated worlds the company has shown yet, and it lets users type a prompt mid-simulation to steer where the scene goes. It also generates real-time audio — music, ambient noise, and interactive dialogue — matched to the simulated world. Genie 4 adds free-form edits, such as changing the weather or camera angle on the fly or dropping in new people and objects, and it can hold a scene steady for up to two minutes, twice as long as Genie 3, the research preview Google put out a little over a year earlier. Google also says the new model handles physics better and renders more humanlike characters.

The launch follows AI video startup Runway’s own second world model, GWM Worlds 2, which shipped a couple of months earlier. World models are proliferating fast enough that the field has already started consolidating: AMD is buying Fei-Fei Li’s World Labs, Amazon is negotiating to acquire Decart, and Rivian recently picked up Third Dimension to bring simulation work in-house for autonomous-driving tests.

Gemini, Google’s AI model, now plays a tighter role alongside Genie 4, sketching out steps and reviewing frames as Genie produces the visuals. Google also ran a robotics experiment built on Gemini Robotics, where a Gemini 4 Argon model split tasks into steps for Genie 4 to act out in real time — one early sign that the system could let engineers rehearse robot behavior in simulation before testing it physically. “What we’ll increasingly see in the future are these hybrid world models, where you have a combination of Gemini writing code and text, and Genie ingesting that in real time and simulating reality at different levels of abstraction,” said Tejas Kulkarni, a Google research director, during a briefing.

Google is pitching Genie 4 for gaming, interactive learning, and robotics, but early trusted testers are also using it to prototype films, Google DeepMind engineer Diego Rivas said — useful, he said, for staying in a creative flow during ideation and storyboarding. Rivas said users could, in time, turn to Gemini Omni — Google’s multimodal model — to build finished creative work from those drafts, though he cautioned that capability is still a ways off. The team flagged real gaps too: details can wander the longer a scene runs, and reliable in-world audio and legible on-screen text remain unsolved. Genie 4 has already opened to trusted testers as a research preview, and Google says it will reach Google AI Ultra subscribers inside Project Genie, as well as developers with early access to the Gemini API and Google AI Studio, within the next few weeks.

Why it matters

Genie 4’s tighter coupling with Gemini and its use in a robotics-simulation experiment point to world models becoming a testing ground for physical AI, not just a creative tool — a shift that lines up with the wave of acquisitions already reshaping the field.