A Standard Interface for Reinforcement Learning Software

by Richard S. Sutton and Juan Carlos Santamaria

This standard is being replaced by the one described at rlai.net.

This document presents a standard interface for programming reinforcement learning simulations. The interface is meant to facilitate reinforcement learning research and the development of widely useable software. In particular, the interface should facilitate a plug and play approach in which learning agents and environments can be designed and implemented separately and then interconnected relatively easily in a standard, uniform fashion. The interface has been kept as simple as possible and follows the terminology in the book by Sutton and Barto.

The software implementing the interface is available in C, C++ and Common Lisp (CLOS). There is basically one interface, but the documentation is provided separately for each language in order to facilitate use in any one language. Examples of agents, environments and simulations are also available.