Image source: Generated by Unbounded AI

Launched by many core contributors from Carnegie Mellon University, Stanford University, MIT Computer Science and Artificial Intelligence Laboratory, NVIDIA and Tsinghua University Genesis, a new physics simulation platform designed for general-purpose robotics, embodied artificial intelligence, and physical AI applications.

The Genesis project has been in development for 2 years, combining the creativity of generative AI with the accuracy of real-world physics. Being able to create virtual environments and conduct training helps machines understand and better interact with the physical world in a way that is not only realistic, but also very fast and efficient.

Genesis has multiple features at the same time:

- Universal physics engine: A universal physics engine built from scratch, capable of simulating a wide range of materials and physical phenomena.

- Robot simulation platform: a lightweight, ultra-fast, Pythonic and user-friendly robot simulation platform.

According to project contributor Zhou Xian, a doctoral student at the CMU Robotics Institute, Genesis's physics engine is completely developed in pure Python, but its speed is faster than frameworks widely used in robot simulation and reinforcement learning research, such as Isaac Gym and MJX is 10 to 80 times faster without sacrificing accuracy at all. It only takes 26 seconds to train a robot motion strategy that can be transferred to the real world on an RTX4090.

Using a single RTX 4090, Genesis can solve IK for 10,000 robot arms like the advanced Franka arm simultaneously in less than 2 milliseconds. This speed and efficiency is unprecedented, opening up new possibilities for large-scale robotic simulations, industrial automation and complex research projects.

- Photorealistic Rendering System: A powerful and fast photorealistic rendering system.

- Generative data engine: It can convert the natural language description prompted by the user into various forms of data.

At its core, Genesis integrates a variety of advanced physics solvers into a unified framework, providing unparalleled simulation capabilities. This powerful foundation, complemented by a generative agent framework, aims to automate data generation and push the boundaries of robotics research and the broader field.

#01. Main features of Genesis

- Unified physical simulation : Based on a universal physics engine, Genesis integrates cutting-edge solvers to simulate a wide range of physics scenarios with exceptional accuracy and fidelity.

- Generative framework: The generative agent framework operates as a modular system and integrates multiple generation modules to process different data forms. These modules are routed by an advanced agent, seamlessly integrating existing researchresearch and continued progress.

The Genesis project introduces a generative agent that handles every step of teaching a robot to operate in a real environment with full autonomy. First, it independently designs virtual environments that simulate real-world spaces, such as kitchens, living rooms, and other everyday environments. It then proposes tasks for the robot to perform, such as opening a microwave, picking up items, or navigating between furniture.

- Open source accessibility: The physics engine and simulation platform are open source, allowing researchers and developers to freely explore, experiment, and innovate. Generating features will be rolled out gradually to expand access.

- Photorealistic rendering: Genesis combines ultra-fast rendering capabilities with visually stunning effects to make simulations more immersive and realistic.

Genesis also supports character action generation, such as a little Wukong who can make acrobatic movements, and can also generate various robot control strategies, such as robotic arms that organize books, drones that can flip synchronously, etc. wait.

Currently, the Genesis team is open-sourcing the underlying physics engine and simulation platform, and officials say access to the generation framework will be gradually rolled out in the near future.

However, some netizens said that using a completely non-existent generate function to demonstrate is suspected of trying to make a big pie. "Even if this part of the content is not open source, I hope the author team can produce a real-machine demonstration video."

#02, Another world simulator? It is more about solving the data bottleneck of embodied intelligence

The goal of the Genesis team is to build a universal data engine, use the upper generation framework to independently create the physical world, and Accompanied by a variety of data models, including environment, camera movement, robot task proposals, reward functions, robot strategies, character actions, fully interactive 3D scenes, open world joint assets, etc., it aims to realize the complete integration of robots, physics AI and other applications. Automated data generation.

Obviously, it is not a world simulator/Sora. Genesis is a hybrid of physics engine and generative AI. Its main function is to provide a unified simulation platform for general robot learning. By lowering the threshold of physical simulation, it can virtually reproduce the real world, thereby reducing human investment in data generation. , enabling an automated and self-sufficient data ecosystem in robotics and related fields.

Robot data collection has always been a thorny problem. One method is to collect data by manually operating the robot. Because you are interacting with the robot in the real physical world, you can know exactly where the robot is and how to interact with it. World interactions, and how missions succeed or fail. However, real-world data collection is costly, inefficient, difficult to expand, and has various limitations (incomplete data modalities, difficulty in collecting closed-loop data, etc.). GenesisThe solution is to use physical simulation and generative AI technology to independently generate a large amount of diverse training data without relying on expensive real machine data collection, thus lowering the threshold for data acquisition.

In addition, there is a gap between the physical characteristics and rendering effects of many simulation environments and the real world, making it difficult for trained robots to be applied in the real world, forming a "Sim2Real Gap". Genesis' solution is to provide a general physics engine that can simulate various materials and physical phenomena, and adopt a realistic rendering system to make the simulation environment as close to the real world as possible and support tactile sensor simulation based on physical principles. Projects such as NVIDIA's Isaac Sim or Isaac Gym have accelerated the development of the robotics field through simulation technology.

Judging from the demo shown, Genesis supports natural language description to generate corresponding scenes, uses generative AI technology to generate sensor data and strategies, and does a good job in rendering simulations of physical properties, which is very useful for solving specific problems. Personal data is scarce and of great significance.

Regarding the simulation issue in the robot industry, Qin Yusen, vice president of the Digua Robot Cloud Platform, expressed his views to Silicon Stars:

Take the Apollo moon landing program as an example, even if we have It is difficult for all the data assets of that year to fully replicate this feat today. This is not just because many original electronic components have been discontinued, but more importantly, simply relying on more powerful computing power and more data cannot solve the fundamental problem. Although the computing power of modern smartphones has far exceeded the supercomputers of the past, we still face many challenges in the face of complex projects such as manned moon landing.

“Because simplification becomes complex and complex becomes simple, and there is a lack of elegant simplification in mathematics and engineering, simulation only increases the amount of calculation.” "To make the simulation realistic enough requires computing power and various resources. For robots, why not just build it directly in the real world?"

Therefore, he believes that Genesis is the engine of The value is that it can make the simulation project relatively elegant in the "engineering architecture", simplifying the various physical properties that need to be understood when calling the engine, and turning it into an engine that can be built through natural language.

He said that the essence of Genesis is to prove that a certain path is feasible, but before it can truly create actual value, it still requires the cooperation of many downstream industry personnel.

“For example, NVIDIA’s Issac SIM should have been released in 2018. It has been 6 years since it was used by some people in the industry, but there are still some problems with ease of use.”

Genesis’s Team Chuang Gan said on worry, but I firmly believe that we can’t create a good thing just becauseBypass the physics simulator if the physics simulator is challenging! ” He said, “Please join the Genesis community! We hope to convince the robotics community that ‘Generative Physics Simulator is all You Need! ’”

Genesis may not be able to achieve “Genesis” yet, but AI is indeed getting closer to the real physical world.