AI Replicates a Video Game Engine After Watching Just 2 Minutes of Video

Modern video game engines, like Unreal, Unity, and CRYENGINE, are incredibly complex. These are the systems that power the graphics, physics, and gameplay of the boundary-pushing games we enjoy today. With state of the art engines like this, it’s easy to think of retro 2D platformers as simplistic. In reality, they were anything but that: game designers from that era had to work within very tight limitations and with primitive tools (by today’s standards).

Even now, with so many incredible design and programming tools at our disposal, creating an engine for a game like Super Mario Bros. is a very difficult task. But, as difficult as it is, a new AI from Georgia Tech students Matthew Guzdial and Boyang Li, along with associate professor Mark O. Riedl, has been able to do just that after watching just a couple minutes of gameplay video. The AI was able to look at footage of a game (like Super Mario Bros. or Sonic the Hedgehog) being played, and then create its own engine to match it.

What’s really important to note here is that the AI did not look at any of the game’s code (or assets), just the gameplay footage. Guzdial says “Our AI creates the predictive model without ever accessing the game’s code, and makes significantly more accurate future event predictions than those of convolutional neural networks.” It does this by studying the video, and making inferences about the rules of gameplay — things like gravity, player movement, and enemy interaction.

The results aren’t perfect, and there are a number of visible glitches. But, it’s still very impressive. Further training of the AI, and more time for it to study, should improve this further.

The benefits for video game development are pretty substantial, as game designers could potentially train the AI to do the heavy lifting of initial game prototyping and programming.