Annotated graph of Q-values over time for an episode of Breakout.

For my dissertation project I chose to explore the topic of game AI. Google DeepMind had recently released a paper about a novel technique of creating general game AIs using machine learning. They then demonstrated the efficacy of their method on a variety of Atari 2600 games (Breakout, Space Invaders, Pacman, etc.).

The method is explained more fully in my dissertation paper, but it essentially combines the traditional technique of Q-learning with neural networks. The aim of the disseration was twofold. Firstly, to try to actually reproduce these results, and potentially improve upon them. Secondly, to develop a visualization tool to try and understand what was going on "behind the scenes". Neural networks are famously opaque, so the aim was to try and understand why each agent took the actions it did.

Since the method requires a large amount of computation, I wrote a program that distributed the workload over 150 lab computers in the computer science building at the university. I also used Google Cloud Storage to store the results.

This was a challenging project, and I was proud at the end of it when I achieved similar results to the Google DeepMind team with a much smaller amount of resources.