technology that ChatGPT supports the ability to do more than just talk. Linxi “Jim” Fan, an AI researcher at chip maker Nvidia, has worked with some colleagues to devise a way to put the powerful GPT-4 language model — the “brains” behind ChatGPT and a growing number of other apps and services — incomprehensible inside a blocky video game. Maine Craft.
The Nvidia team, which included Anima Anandkumar, the company’s machine learning director and professor at Caltech, created a Minecraft bot called Voyager that uses GPT-4 to solve in-game problems. The language model generates goals that help the agent to explore the game, and code that improves the bot’s skill in the game over time.
Voyager doesn’t play the game like a person, but it can read the state of the game directly, via an API. You might see a fishing rod in its inventory and a river nearby, for example, and use GPT-4 to suggest a goal to do some fishing to gain experience. It will then use this intent to have GPT-4 generate the code required to make the character achieve this.
Courtesy of NVIDIA
The most recent part of the project is the code that GPT-4 generates to add behaviors to Voyager. If the initially suggested code doesn’t work perfectly, Voyager will try to improve it using error messages, feedback from the game, and a description of the code generated by GPT-4.
Over time, Voyager builds a library of code to learn how to make increasingly complex things and explore more of the game. The graph created by the researchers shows how capable it is compared to other Minecraft factors. Voyager gets more than three times as many items; explores more than twice the distance; And it builds tools 15 times faster than other AI agents. Fan says the approach may improve in the future by adding a way for the system to integrate visual information from the game.
While chatbots like ChatGPT have dazzled the world with their eloquence and obvious knowledge—even if they often make things up—Voyager shows the huge potential of language models to perform useful actions on computers. Using language models in this way could automate many routine office tasks, which could be one of the biggest economic impacts of technology.
Courtesy of NVIDIA
The process that Voyager uses might be adapted to GPT-4 for figuring out how to do things in Minecraft for a software assistant working out how to automate tasks across the operating system on a computer or phone. OpenAI, the startup that created ChatGPT, has added “plugins” to the bot that allow it to interact with online services such as grocery delivery app Instacart. Microsoft, which owns Minecraft, also trains AI software to run it, and the company recently announced Windows 11 Copilot, a feature in the operating system that will use machine learning and APIs to automate certain tasks. It might be a good idea to try this kind of technology inside a game like Minecraft, where faulty code can do relatively little harm.
Video games have always been a testing ground for AI algorithms, of course. AlphaGo, the machine learning software that perfected the hit board game Go Back in 2016, cut its teeth playing simple Atari video games. AlphaGo used a technique called reinforcement learning, which trains an algorithm to play a game by giving it positive and negative feedback, for example from an in-game score.
This method is difficult to direct an agent in an open-ended game like Minecraft, where there are no points or set goals and where the player’s actions may not bear fruit until much later. Whether or not you think we should prepare to contain the existential threat from artificial intelligence right now, Minecraft looks like an excellent playground for technology.
