Reinforcement Learning Software and Stuff

• Amir Hesami's matlab version of the cart-pole system


• G, a graphics package for Macintosh Common Lispe

• A simple graphing tool for Macintosh Common Lisp


• Tor Aamodt's simulation of a swing-up pendulum with OpenGL display and its RL solution. Includes C++ code, BASc thesis, and executable for Windows NT 4.0 or Windows 98.

• Proposed Standard for RL Software

• Tile Coding, a.k.a. CMACs (Univ New Hampshire, my form simplified for RL.)

• TD Model of Classical Conditioning in C++ and Lisp

• C++ Code for the Mountain Car (S. Mahadevan)

• Lisp code for Acrobot (a code fragment, FYI)

• Code demonstrating a RL network-routing algorithm by Justin Boyan and Michael Littman. You need a C compiler and the "wish" windowing shell to run this. The ideas are described in their papers at NIPS'93 and IWANNT'93. There was also a paper at NIPS'95 extending this work.

• C code for the pole-balancing simulator and controller as in the Barto, Sutton, and Anderson (1983) paper.

• Pseudo-code for combining TD(lambda) and backpropagation as in Tesauro's (1992) backgammon player TD-Gammon. See also the following paper.

• Technical report giving detailed equations for the combination of TD(lambda) and backpropagation. See also above pseudo-code.

• Lisp code for a simple illustrative example of an early version of Dyna-AHC (also known as Dyna-PI). This is a pure commonlisp program without graphics.

• The schedule of the workshop "Reinforcement Learning: What We Know, What we Need" held at the Univ. of Mass. as part of the 1993 Int. Machine Learning Conference. Serves as one view of the key topics in RL.