• Stars
    star
    663
  • Rank 67,991 (Top 2 %)
  • Language
    Lua
  • Created over 7 years ago
  • Updated almost 3 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A Real-time Mario Kart 64 AI using ConvNets.

NeuralKart - A Real-time Mario Kart AI using CNNs, Offline Search, and DAGGER

Set-up

The project currently only works on Windows.

Install 64-bit Python 3

This project was written for Python 3. Furthermore, Tensorflow requires 64-bit Python.

Install Python Dependencies

The following Python dependencies need to be installed.

  • Tensorflow
  • Keras
  • Pillow
  • matplotlib
  • mkdir_p
  • h5py

Get BizHawk (1.12.2)

Our scripts are all written for the BizHawk emulator (tested in version 1.12.2), which has embedded Lua scripting. To get BizHawk you first need to install the prerequisites - https://github.com/TASVideos/BizHawk-Prereqs/releases/tag/1.4. Then you can download BizHawk (version 1.12.2) and unzip it to any directory - https://github.com/TASVideos/BizHawk/releases/tag/1.12.2

Download Our Pre-trained Weights and Recordings

These should be unzipped into the folder of the repo.

  • Save States - The states should be accessible as states/[mode]/[track].state.
  • Weights - The weights should be accessible as weights/[track].hdf5
  • Recordings (Optional) - The recordings should be accessible as recordings/[track]/[mode]/[recording]/[frame].png.

Usage Instructions

Running a Live Demo

These instructions can be used to run a demo of three tracks that the AI performs well on.

  1. Download the save states and pre-trained weights.
  2. Run predict-server.py using Python 3 - this starts a server on port 36296 which actually runs the model.
    • You can pass a --cpu to force Tensorflow to run on the CPU.
  3. Open BizHawk and Load the MarioKart 64 ROM.
  4. Turn off messages (View > Display Messages).
    • You don't have to do this, but they get in the way.
  5. Open the BizHawk Lua console (Tools > Lua Console).
  6. Load Demo.lua

This should automatically play three tracks in a loop. You can hit Esc to switch to the next track. You can also hit the arrow keys to manually steer the player. This can be used to demonstrate the AI's stability.

Note that the clipboard is used to pass frames from the emulator to the Python script. It's a hack, but it seems to work - just don't try to copy or paste anything while the scripts are running.

Run the AI on another Track

Once you have the demo working, you can use these instructions to play on other tracks. Note that you can only play on a track if there are weights trained for it.

First, navigate to another track from the menu, or use one of our save states (File > Load State > Load Named State). These states are set to be the frame after the race starts. Then load Play.lua from the Lua console.

Training the Model on Recordings

Once you have the AI running, you probably want to try retraining the weights based off our recordings. First download our weights from the link above, then run train.py [track]. You can also use --cpu to force it to use the CPU.

Creating new Recordings from the Search AI

Load a state and then load SearchAI.lua in order to generate a recording using the search AI. Recordings consist of a series of frames and a steering.txt file that contains the recorded steering values.

Running the Iterative Improvement Loop

As mentioned in the paper, we ran an iterative improvement loop that swaps between playing and generating new recordings. To bootstrap the process, you must first generate a recording using the search AI and create an initial weights file using train.py. Now start predict-server.py using the --cpu flag (so that you can train on the GPU).

Now you can load a state and run PlayAndSearch.lua which alternates between playing and searching. It retrains every other run. You probably need to edit the code that calls train.py on line 90 so that it works in your environment.

Other Projects + Links

More Repositories

1

netplayjs

Make P2P multiplayer browser games, no server hosting or synchronization code required. Powered by rollback netcode + WebRTC.
TypeScript
420
star
2

blog-cells

Add interactive code snippets to any blog or webpage.
TypeScript
330
star
3

Dygra

A real-time non-euclidean ray-tracer.
C++
94
star
4

ipfs-share

A simple IPFS-powered Pastebin / Image Host / File Sharing application.
HTML
74
star
5

showdownbot

A Pokemon Showdown battling AI.
JavaScript
68
star
6

love-ide

Tools for programming LÖVE games in Atom.
JavaScript
56
star
7

ups

Command line tools for manipulating UPS binary patches, written in Go.
Go
30
star
8

fault

An obstacle avoidance game for Android and iOS.
C
26
star
9

Azathoth

An editor for non-euclidean worlds.
Python
20
star
10

binary-inspector

Web-based inspectors for packets and binary files.
TypeScript
19
star
11

love-console

An easy to integrate in-game console for Love2D games.
Lua
15
star
12

snippets-editor

Manage your Atom snippets using a visual interface.
JavaScript
10
star
13

marble-mouse

A Super Monkey Ball-inspired game made using THREE.js and CANNON.js.
TypeScript
9
star
14

codec2-emscripten

The Codec 2 speech codec, compiled to WASM using Emscripten.
JavaScript
9
star
15

Hnefatafl

Android Hnefatafl game using Google Play Games Services.
Java
6
star
16

sox-emscripten

WASM port of SoX (Sound eXchange) using Emscripten, for use in the browser.
JavaScript
6
star
17

hain-plugin-chrome-bookmarks

A Hain plugin for searching Chrome bookmarks.
JavaScript
6
star
18

todo-list

Find TODOs, NOTEs, and FIXMEs in your project. Sort by assignee and file.
JavaScript
4
star
19

Ulfhednar

A co-op bullet hell shooter, inspired by Norse mythology.
Lua
3
star
20

random-mbmbam

Listen to a Random MBMBAM epsiode.
HTML
3
star
21

pinboard-tools

Miscellaneous utilities for manipulating my Pinboard links.
Python
3
star
22

proctor

A horror game based on 3D-sound, created in Love2D.
Lua
3
star
23

misbehaving-receiver

Replicating the paper "TCP Congestion Control with a Misbehaving Receiver" for CS 244.
C
3
star
24

feedvid-live

A horror game with text-adventure inspired gameplay, created using Three.js and React.
TypeScript
3
star
25

protoworld

[WIP] A live object system for JavaScript, inspired by Self and Smalltalk.
JavaScript
3
star
26

love2d-template

My template for making games in LOVE2D.
Lua
2
star
27

got

[Incomplete] A VCS written in Go.
Go
2
star
28

github-network-analysis

GitHub network analysis for CS224W.
Python
2
star
29

RetroBoy

A cross-platform, GameBoy inspired, fantasy console.
Java
2
star
30

TheSixthSun

[Abandoned] A turn-based, asynchronous, space strategy game that takes place both in space, and across the surface of planets.
C#
2
star
31

Math4D

A tool for visualizing a variety of mathematical constructs, including 3 and 4-variable functions.
Python
2
star
32

linter-luaparse

Linter for Atom that finds Lua syntax errors.
JavaScript
2
star
33

dumblisp

A lisp interpreter written in pure C.
C
2
star
34

hump.timer.actions

Composable actions, powered by Hump.Timer.
Lua
1
star
35

marble-mouse-custom-mapping

Marble Mouse custom mapping guide.
1
star
36

MemChewIonic

Ionic-based client for MemChew.
JavaScript
1
star
37

MemChewAndroid

MemChew Android client.
Java
1
star
38

lolreplay

Python code for parsing League of Legends replay files.
Python
1
star
39

libsrtp-js

[Incomplete] TypeScript binding for libSRTP, using WebAssembly.
TypeScript
1
star
40

jamestown-ai

(Incomplete) AI bot for the bullet hell game Jamestown.
Python
1
star
41

BlueScreen

A first-person stealth / 6DOF shooter mashup. Created in Unity in 48 hours for Ludum Dare 30.
C#
1
star
42

cs234-project

Python
1
star
43

escape-template-string

Escape a string so that it can be eval-ed in as a template literal.
JavaScript
1
star
44

hyperclick-love

A Hyperclick provider for LÖVE.
JavaScript
1
star
45

tldr-ml

The TLDR Machine Learning dictionary.
HTML
1
star
46

love-ide-data

Shared data between autocomplete-love and hyperclick-love.
Lua
1
star
47

user-scripts

User scripts that I use.
JavaScript
1
star
48

ws-relay

A simple WebSocket relay client and server. Can be used for WebRTC signaling, or to send data directly between browsers.
TypeScript
1
star
49

snp

[WIP] An enet-like UDP reliability layer for NodeJS, KaiOS, and other JavaScript environments.
TypeScript
1
star
50

topkappa

Vote on Twitch chat messages in real-time.
JavaScript
1
star