NeuralKart - A Real-time Mario Kart AI using CNNs, Offline Search, and DAGGER
Set-up
The project currently only works on Windows.
Install 64-bit Python 3
This project was written for Python 3. Furthermore, Tensorflow requires 64-bit Python.
Install Python Dependencies
The following Python dependencies need to be installed.
- Tensorflow
- Keras
- Pillow
- matplotlib
- mkdir_p
- h5py
Get BizHawk (1.12.2)
Our scripts are all written for the BizHawk emulator (tested in version 1.12.2), which has embedded Lua scripting. To get BizHawk you first need to install the prerequisites - https://github.com/TASVideos/BizHawk-Prereqs/releases/tag/1.4. Then you can download BizHawk (version 1.12.2) and unzip it to any directory - https://github.com/TASVideos/BizHawk/releases/tag/1.12.2
Download Our Pre-trained Weights and Recordings
These should be unzipped into the folder of the repo.
- Save States - The states should be accessible as
states/[mode]/[track].state
. - Weights - The weights should be accessible as
weights/[track].hdf5
- Recordings (Optional) - The recordings should be accessible as
recordings/[track]/[mode]/[recording]/[frame].png
.
Usage Instructions
Running a Live Demo
These instructions can be used to run a demo of three tracks that the AI performs well on.
- Download the save states and pre-trained weights.
- Run
predict-server.py
using Python 3 - this starts a server on port36296
which actually runs the model.- You can pass a
--cpu
to force Tensorflow to run on the CPU.
- You can pass a
- Open BizHawk and Load the MarioKart 64 ROM.
- Turn off messages (View > Display Messages).
- You don't have to do this, but they get in the way.
- Open the BizHawk Lua console (Tools > Lua Console).
- Load
Demo.lua
This should automatically play three tracks in a loop. You can hit Esc
to switch to the next track. You can also hit the arrow keys to manually steer the player. This can be used to demonstrate the AI's stability.
Note that the clipboard is used to pass frames from the emulator to the Python script. It's a hack, but it seems to work - just don't try to copy or paste anything while the scripts are running.
Run the AI on another Track
Once you have the demo working, you can use these instructions to play on other tracks. Note that you can only play on a track if there are weights trained for it.
First, navigate to another track from the menu, or use one of our save states (File > Load State > Load Named State). These states are set to be the frame after the race starts. Then load Play.lua
from the Lua console.
Training the Model on Recordings
Once you have the AI running, you probably want to try retraining the weights based off our recordings. First download our weights from the link above, then run train.py [track]
. You can also use --cpu
to force it to use the CPU.
Creating new Recordings from the Search AI
Load a state and then load SearchAI.lua
in order to generate a recording using the search AI. Recordings consist of a series of frames and a steering.txt
file that contains the recorded steering values.
Running the Iterative Improvement Loop
As mentioned in the paper, we ran an iterative improvement loop that swaps between playing and generating new recordings. To bootstrap the process, you must first generate a recording using the search AI and create an initial weights file using train.py
. Now start predict-server.py
using the --cpu
flag (so that you can train on the GPU).
Now you can load a state and run PlayAndSearch.lua
which alternates between playing and searching. It retrains every other run. You probably need to edit the code that calls train.py
on line 90 so that it works in your environment.
Other Projects + Links
- TensorKart - The first MarioKart deep learning project, which we started from as our baseline.
- Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning - The idea for using a search-based AI for teaching the Convnet AI came from this paper.
- A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning - The DAGGER algorithm was first introduced in this paper.
- MarioKart 64 NEAT - This AI uses the NEAT algorithm to genetically evolve a shallow neural network
- weatherton/BizHawkMarioKart64 - Some MarioKart 64 scripts which we used as a reference for memory locations.