Microsoft Minecraft is an experimental place where humans and AI work together

"The idea of ​​turning Minecraft into an AI Olympic Games is great."

Microsoft's Project Malmo project began in June 2015. Its predecessor was Project AIX. This project is a mature AI experiment platform based on Minecraft and aims to support basic research in the field of artificial intelligence. The Project Malmo platform consists of two parts: the Java version of the game module and the code that helps the AI ​​agent to perceive and act. Both of these parts can run under Windows, Linux, or Mac OS environments, and researchers can use any language to program AI agents.

The specific scenario is that researchers use Minecraft to train AI agents to manipulate the characters in the game and perform various tasks, such as crossing bridges or building complex objects. In the past, manipulating the game characters was often done by human players. Not only that, the Project Malmo platform also offers several ways for human players and artificial agents to collaborate with each other : the game interface will display a chat window, and human players can talk to a new born agent.

Microsoft once stated in its official blog that because Minecraft involves the solution of various problems, the establishment of space, the deduction of time, and even cooperation, it is very suitable for training artificial intelligence.

Katja Hoffman is a research fellow at the Microsoft Cambridge Research Institute and the project leader of Project Malmo. She said: "In the long run, I hope to develop artificial intelligence that can be taught by any user and help users accomplish some of the goals and tasks."

(The picture shows the Project Malmo team, the first row of right blonde girls is Katja Hoffman), from: Microsoft Blog

Last week, Hoffman demonstrated the project to researchers in the AI ​​field at an academic conference in New York and stated that achieving human and AI cooperation is a key goal of this project. In addition, Project Malmo is now open source on GitHub, and related tools have been freely available for download to help developers more easily enhance and improve the field of artificial intelligence. Hoffman said: "We have built in enough features for researchers to develop collaborative AI."

Project Malmo is currently focusing on testing reinforcement learning algorithms that use a simulated reward mechanism to train computers to perform specific tasks. For example, use an intensive algorithm to train an AI agent to control a character in Minecraft, to cross a room full of obstacles and give it a reward if the task is successful. Human players can participate in this process and provide some helpful suggestions for the agents. Agents will gradually understand and use these suggestions.

As shown in the figure below, the character named human 1 is controlled by a human player and the character named malmo 1 is controlled by an algorithm.

Picture from: MIT Technology Review

Microsoft CEO Satya Nadella wrote the article "The Partnership of the Future" on June 28, depicting the future vision of human and AI cooperation in solving social problems and stressing that Increasingly clever machine learning algorithms have great potential in helping humans improve their work efficiency, and AI is an especially important development direction for Microsoft.

Hoffman believes that this project will eventually be applied to regular games. She said: "I can imagine that in the future you will let the agent deal with the monotonous part of the game."

AI assists Minecraft players in playing games. This is just the first step in implementing computer software to help humans handle daily tasks. "We need to develop something that really benefits humans and empowers humans," Hoffman said.

AI researchers have generally begun to transform computer games into testing grounds for testing mature machine learning methods. At the beginning of last year, Google’s DeepMind Company demonstrated an AI system. The system autonomously studied 49 games on the Atari 2600 console, and scored more than the game experts in 29 games, causing a stir. The algorithm used in this AI system incorporates deep learning and reinforcement learning. DeepMind then demonstrated to the outside world that the AI ​​system was learning to manipulate the more complicated 3D games, but so far no further work has been exposed.

Minecraft is an indie game that was acquired by Microsoft in 2014. Despite its extremely simple picture and lack of clear game goals, it is widely popular for its inspiring creativity. The game fans used it to build a series of incomparable complex buildings and mechanical devices, and the players used their "Redstone" elements to create CPUs, which are truly computer computers.

Picture from: MIT Technology Review

Although Minecraft is relatively simple, it can provide a useful testing ground for many robotic algorithms. In July last year, the team led by Professor Stefanie Tellex of Brown University developed an algorithm that allowed the robot to quickly determine a sequence of actions. At that time, it used Minecraft, the "testing ground." She said that one of the great uses of the Malmo platform is to make it easier for researchers to compare their research methods . She said: "The idea of ​​turning Minecraft into an Olympic Games for the AI ​​community is very good." She also pointed out that the platform can also collect large amounts of data more effectively, such as data on the interaction between humans and AI, and these data are exactly modern. Machine learning is very much needed.

Although the Malmo platform targets professionals in the fields of artificial intelligence, machine learning, and robotics, anyone with sufficient technical skills can download this platform to experiment with the game's built-in AI agent. The game contains several examples of machine learning packages and AI agents. In fact, the current download of the game does include a group of amateurs and software engineers who do not specialize in AI.

Minecraft's somewhat awkwardly coarse pixel game may become an excellent platform for humans to learn and work together with artificial intelligence (AI).

Via MIT Technology Review