Chronosphero

Ever experienced a really poorly designed level boundary? Ever got frustrated by invisible walls and annoying messages in midair? We are tired of these design decisions that’s why we have committed ourselves to develop a simple-to-use jet powerful AI to create endless game worlds. We also developed a simple side-scroller to test and evaluate the capability of this AI.

Chronosphero

We developed a simple jump 'n' run platform game called Chronosphero to test different techniques of procedural level generation.

In Chronosphero the player controls a little robot (left/right + jump) in a simple 2D-world with the objective to collect as many coins as possible. At the beginning a timer is set to 10 Seconds and is running all the time. To reset the timer again to 10 Seconds, the player has to collect a clock-item. The player can move freely in all directions and has to explore the infinite level to find new coins and clocks. The game ends when the timer reaches 0:00.

What are Generative Adversarial Networks (GANs) ?

Generative Adversarial Networks are one of the most interesting ideas in computer science today. Two models are trained simultaneously by an adversarial process. A generator ("the artist") learns to create images that look real, while a discriminator("the art critic") learns to tell real images apart from fakes.

During training, the generator progressively becomes better at creating images that look real, while the discriminatorbecomes better at telling them apart. The process reaches equilibrium when the discriminatorcan no longer distinguish real images from fakes. The generator tries to estimate the distribution of the training data from which it learns. Labeling of the data is not required.

GANs and Chronsphero

Convolutional GANs are great at working with images and we can represent our game level as matrices and therefor also as images. Each block in the level corresponds with a ‘1’ in a matrix respectively with a black pixel in an image. The free space between the blocks corresponds with a ‘0’ in a matrix or a white pixel in an image. Each of these images represents a small piece of a large level.

This data representation is quite flexible and can easily be extended with further game objects like the clocks and coins the player have to collect.

You could train a GAN with such images, and it would consistently generate new pieces. If you join these pieces together, you will get a potentially infinite large level.

What problems we are facing?

We have not a large amount of training data but only a view handcrafted levels. This is not enough for a GAN to learn the distribution and create good levels. That's why we looked at yet another technology:

What about Reinforcement Learning?

While GANs work well on existing Datasets, which weren't available to us, we came up with a different approach suited for the Unity Environment we were working with: Machine Learning Agents with Reinforcement Learning.

Agency as a philosophical Term describes the ability to act in a certain way. The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents, licensed under the Apache 2.0 License.

Originally ML-Agents are used for Training Non personal characters (NPCs) on specific tasks to enrich gaming experiences. The ML-Agents Toolkit provides training environments and machine learning algorithms suited for our needs.

RL and Procedural Content Generation

EA states their model as Adversarial Reinforcement Learning for Procedural Content Generation. Three Main components build the model. Two Agents, a Generator (Environment Builder) and a Solver (NPC), as well as an Environment.

Getting to know ML Agents

Getting started with ML Agents is relatively easy compared to setting up a greenfield project. There is a lot of content out there to get a hold on how everything has link together for a project.

Especially helpful was a fixed solution to get to know all the tools necessary for ML-Agents by Adam Kelly of Immersive Limit. Nevertheless, as time isn't limitless here is a speeded up description on what we needed to do:

1. Setup the world and the task to solve

Build a little platform game, with basic features such as a timer and reward system. To keep complexity low:

• The solver agent needs to run through the world and collect coins within a set amount of time.
• The generator needs to add new platforms in ether direction left and right and generates and places the coins.

2. Create the agent logic with scripts

The main difficulty is to get the thought of how the agent would solve the problem translated to code. You need to tell the agent within a rewardsystem that is coupled via heuristics to the training enviornment, how well it did on a specific task and additionally tell how it behaves once hooked to the learning algorithms of ML-Agents.

While all of this is done within Unity you don't have to wrap your head around physics too much but a basic understanding of vector descriptions and movement are needed.

3. Train the agents with the Python API and Tensorflow

After you are done with the transfer of your physical model to the agent scripts you can start training the agents with anaconda via the python API ML-Agents provides.

Training even for simple tasks can require some time, and setting up the Unity environment such as, that there several instances of the same agent running might be required.

What's next and why the hassle?

The familiarity with the Unity Environment and the tools to train our agent models enables us to create agent-models for different tasks in video games. If successful gaming environments could be generated and made to fit the skill set of different players. While gaming 'against' a set environment or conventional AI can seem boring after an amount of time, NPCs in Form of intelligent agents create a more vivid experience with even collaborative tasks to master.

With this in mind, as not only the gaming industry but big tech companies as well shift their business processes from the real world into virtual environments, the adaption for other uses such as assistance in work environments for example seems far from unrealistic.

Attribution

This project was developed as part of the lecture 'Applied Artificial Intelligence' at the University of Applied Sciences Esslingen.

Portions of this page are reproduced from work created and shared by Google and used according to terms described in the Creative Commons 4.0 Attribution License.