• Sorted by Date • Classified by Publication Type •
Shimon Whiteson. Evolutionary Function Approximation for Reinforcement Learning. In GECCO 2006: Proceedings of the Genetic and Evolutionary Computation Conference Graduate Student Workshop, July 2006. Best Paper Award, Graduate Student Workshop.
Temporal difference methods are theoretically grounded and empirically effective methods for addressing reinforcement learning problems. In most real-world reinforcement learning tasks, TD methods require a function approximator to represent the value function. However, using function approximators requires manually making crucial representational decisions. This thesis investigates evolutionary function approximation, a novel approach to automatically selecting function approximator representations that enable efficient individual learning. This method evolves individuals that are better able to learn. I present a fully implemented instantiation of evolutionary function approximation which combines NEAT, a neuroevolutionary optimization technique, with Q-learning, a popular TD method. The resulting NEAT+Q algorithm automatically discovers effective representations for neural network function approximators. This thesis also presents on-line evolutionary computation, which improves the on-line performance of evolutionary computation by borrowing selection mechanisms used in TD methods to choose individual actions and using them in evolutionary computation to select policies for evaluation. I evaluate these contributions with extended empirical studies in two domains: 1) the mountain car task, a standard reinforcement learning benchmark on which neural network function approximators have previously performed poorly and 2) server job scheduling, a large probabilistic domain drawn from the field of autonomic computing. The results demonstrate that evolutionary function approximation can significantly improve the performance of TD methods and on-line evolutionary computation can significantly improve evolutionary methods.
@InProceedings{whiteson:gecco06ws,
author = "Shimon Whiteson",
title = "Evolutionary Function Approximation for Reinforcement Learning",
booktitle = "GECCO 2006: Proceedings of the Genetic and
Evolutionary Computation Conference Graduate Student Workshop",
month = "July",
year = 2006,
note = "\textbf{Best Paper Award, Graduate Student Workshop}.",
}
Generated by bib2html.pl (written by Patrick Riley ) on Wed Mar 07, 2012 15:56:00