	Reinforcement Learning and Artificial Intelligence (RLAI)
	Downloading and Installing the Tile Coding Software

Tile coding is a good way of representing continuous state variables as binary features for online machine learning such as is common in reinforcement learning. For further information, see the reference manual, or the textbook description. This page lists the various implementations of tile coding that follow the specification in the reference manual (version 2.1) and compares their efficiency.

Timing Comparisons

These tests were run on a Mac powerbook with 1 G memory, in 2005 or thereabouts. For 10,000 calls to tiles, each with 4 tilings and 2 floats:

Tiles Implementation	Python	Lisp	C++	Python calling C++
Regular: no collision table	1.03 sec	0.19 sec	0.02 sec	0.09 sec
unsafe collision table	1.56 sec	0.26 sec	0.06 sec	0.10 sec
safe collision table	1.73 sec	0.29 sec	0.04 sec	0.10 sec
super safe collision table	2.19 sec	0.24 sec	0.06 sec	0.10 sec
Wrap: no collision table	1.50 sec	1.38 sec(GC)	0.02 sec	0.12 sec
unsafe collision table	1.86 sec	0.29 sec	0.05 sec	0.11 sec
safe collision table	1.93 sec	0.29 sec	0.03 sec	0.13 sec
super safe collision table	1.64 sec	0.25 sec	0.05 sec	0.12 sec
Load: no collision table	1.22 sec	0.27 sec	0.0003 sec	5.9 sec
unsafe collision table	1.81 sec	1.57 sec(GC)	0.0004 sec	5.8 sec
safe collision table	1.96 sec	0.31 sec	0.0004 sec	5.9 sec
super safe collision table	1.78 sec	0.27 sec	0.0004 sec	6.4 sec
Load wrap: no collision table	1.39 sec	0.21 sec	0.0002 sec	5.8 sec
unsafe collision table	1.87 sec	0.30 sec	0.0003 sec	5.9 sec
safe collision table	1.95 sec	0.29 sec	0.0003 sec	5.8 sec
super safe collision table	1.62 sec	0.35 sec	0.0005 sec	5.9 sec

C++ Implementation of Tiles

C++ header, C++ source Timing code is here

Python Implementation

This has been tested using Python version 2.3...
For help in using Python, see here.

Installation:

The Tiles code was last updated January 25, 2005. For previous versions, see information on RLtoolkit versions.
Download python source code here. Right click or control click to save this tarball.
The file will unzip into a folder called Tiles. Move this folder to your Python site packages folder, or to wherever you want to run Python from.
Tiles can also be acquired by installing the Reinforcement Learning toolkit.

Usage:

Move to the folder you put Tiles in (not necessary if it is installed in your python site packages folder)
Start up Python and do one of the following:
- from Tiles.tiles import *
  (then you can call the basic tiles functions tiles, loadtiles, tileswrap, loadtileswrap and CollisionTable with no module name required)
- import Tiles.tiles as tiles
  (then make sure you prefix all basic tiles commands with tiles - eg. tiles.loadtiles(...)
- from Tiles import *
  Then you can use the various modules in the Tiles package as follows by prefixing the module name to the desired function. For example:
  - tiles.tiles(...)
  - fancytiles.diamondtiles(...)
  - tilesdemo.showtiles(...)

Python Calling C Implementation of Tiles

This version allows you to have the extra speed of the C version of tiles but still write your programs in Python.

Requirements:

You must have the following installed on your machine to use the c version of tiles:
- Python (e.g., version 2.3)
- C++
For help in using Python, see here.

Installation:

The Python calling C Tiles code was last updated January 25, 2005.
Download C and python source code here. Right click or control click to save this tarball.
The file will unzip into a folder called CTiles. Move this folder to your Python site packages, or to wherever you want to run Python from.
There is a Makefile for Macintosh in this folder. You must run this to compile the c version of tiles, and the python to c interface for tiles. If you are not using a Macintosh, look at the Makefile to see what needs to be done and compile the code appropriately for your platform.
Tiles, including this version, can also be acquired by installing the Reinforcement Learning toolkit.

Usage:

Move to the folder you put CTiles in (not necessary if it is installed in your python site packages folder)
Start up Python and do one of the following:
- from CTiles.tiles import *
  (then you can call the basic tiles functions tiles, loadtiles, tileswrap, loadtileswrap and CollisionTable with no module name required)
- import CTiles.tiles as tiles
  (then make sure you prefix all basic tiles commands with tiles - eg. tiles.loadtiles(...)
- from CTiles import *
  Then you can use the various modules in the CTiles package as follows by prefixing the module name to the desired function. For example:
  - tiles.tiles(...)
  - fancytiles.diamondtiles(...)
  - tilesdemo.showtiles(...)

Lisp Installation of Tiles

Requirements:

This runs under Macintosh Common Lisp

Installation:

The tiles.lisp source code is available here. Right click or control click to save this file. Timing code is here.
(Previous version is here)