Discounted Future Prediction (DFP) implemented in Keras

This repo includes implementation of Discounted Future Prediction (DFP) Algorithm describe in this paper. The implementation is tested on the VizDoom Health Gathering scenario, which is a 3D partially observable environment.

For a general introduction of DFP and step-by-step walkthrough of the implementation, please check out my blog post at https://flyyufelix.github.io/2017/11/17/direct-future-prediction.html.

Results

Below is the performance chart of 40,000 episodes of DFP and DDQN running on Health Gathering. Y-axis is the average survival time (moving average over 50 episodes).

Usage

First follow this instruction to install VizDoom. If you use python, you can simply do pip install:

$ pip install vizdoom

Second, clone ViZDoom to your machine, copy the python files provided in this repo over to examples/python.

Next, edit scenarios/health_gathering.cfg file. Replace this line

doom_scenario_path = health_gathering.wad

with

doom_scenario_path = health_gathering_supreme.wad

To test if the environment is working, run

$ cd examples/python
$ python dfp.py

You should see some printouts indicating that the DFP is running successfully. Errors will be thrown otherwise.

Dependencies

• Keras 1.2.2 / 2.0.5
• Tensorflow 0.12.0 / 1.2.1
• VizDoom Environment