DeepMind, Google’s AI research division, has introduced Genie 2, a cutting-edge model capable of generating an infinite array of playable 3D worlds. Building on its predecessor Genie, released earlier this year, Genie 2 can create interactive, real-time scenes from just a single image and a text description, such as “A cute humanoid robot in the woods.” This innovation mirrors the efforts of companies like Fei-Fei Li’s World Labs and the Israeli startup Decart.

Genie 2 is designed to generate immersive 3D environments where users can interact through actions like jumping and swimming using a keyboard or mouse. Trained on video data, it simulates intricate details such as object interactions, animations, lighting, physics, reflections, and non-playable character (NPC) behavior. Many of these simulations resemble high-quality AAA video games, likely due to the model being trained on gameplay footage. However, DeepMind has kept the specifics of its training data under wraps, citing competitive concerns.

This raises questions about intellectual property, particularly given Google’s access to YouTube content and its terms of service, which might allow such data usage. Critics wonder whether Genie 2’s outputs could be considered unauthorized reproductions of games it analyzed – a matter for legal debate.

The model offers consistent rendering of 3D worlds in different perspectives, such as first-person and isometric views, for up to a minute. Most simulations last between 10 and 20 seconds but retain spatial memory, ensuring unseen parts of the scene are accurately rendered when revisited – a feature it shares with models from World Labs.

However, these generated worlds are currently limited in scope and unsuitable for creating full-fledged games due to their ephemeral nature, which resets progress frequently. Therefore, rather than being a gaming engine, DeepMind sees Genie 2 as a tool for study and creative development. It enables rapid development of interactive environments and unique evaluation tasks for testing AI agents, leveraging its ability to turn concept art into dynamic, playable scenes.

While some creatives, particularly in the gaming industry, may view such advancements with skepticism amid fears of job displacement, DeepMind continues to expand its research. Recent high-profile hires, including Tim Brooks from OpenAI and Tim Rocktäschel from Meta, underscore Google’s commitment to pushing the boundaries of AI-driven world modeling and simulation technologies.