Surrogate models for enhancing the efficiency of neuroevolution in reinforcement learning

Jörg Stork, Thomas Bartz-Beielstein, Martin Zaefferer, A. E. Eiben

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

In the last years, reinforcement learning received a lot of attention. One method to solve reinforcement learning tasks is Neuroevolution, where neural networks are optimized by evolutionary algorithms. A disadvantage of Neuroevolution is that it can require numerous function evaluations, while not fully utilizing the available information from each fitness evaluation. This is especially problematic when fitness evaluations become expensive. To reduce the cost of fitness evaluations, surrogate models can be employed to partially replace the fitness function. The difficulty of surrogate modeling for Neuroevolution is the complex search space and how to compare different networks. To that end, recent studies showed that a kernel based approach, particular with phenotypic distance measures, works well. These kernels compare different networks via their behavior (phenotype) rather than their topology or encoding (genotype). In this work, we discuss the use of surrogate model-based Neuroevolution (SMB-NE) using a phenotypic distance for reinforcement learning. In detail, we investigate a) the potential of SMB-NE with respect to evaluation efficiency and b) how to select adequate input sets for the phenotypic distance measure in a reinforcement learning problem. The results indicate that we are able to considerably increase the evaluation efficiency using dynamic input sets.

Original languageEnglish
Title of host publicationGECCO 2019 - Proceedings of the 2019 Genetic and Evolutionary Computation Conference
PublisherAssociation for Computing Machinery, Inc
Pages934-942
Number of pages9
ISBN (Electronic)9781450361118
DOIs
Publication statusPublished - 13 Jul 2019
Event2019 Genetic and Evolutionary Computation Conference, GECCO 2019 - Prague, Czech Republic
Duration: 13 Jul 201917 Jul 2019

Conference

Conference2019 Genetic and Evolutionary Computation Conference, GECCO 2019
CountryCzech Republic
CityPrague
Period13/07/1917/07/19

Keywords

  • Neuroevolution
  • Reinforcement Learning
  • Surrogate Models

Fingerprint Dive into the research topics of 'Surrogate models for enhancing the efficiency of neuroevolution in reinforcement learning'. Together they form a unique fingerprint.

  • Cite this

    Stork, J., Bartz-Beielstein, T., Zaefferer, M., & Eiben, A. E. (2019). Surrogate models for enhancing the efficiency of neuroevolution in reinforcement learning. In GECCO 2019 - Proceedings of the 2019 Genetic and Evolutionary Computation Conference (pp. 934-942). Association for Computing Machinery, Inc. https://doi.org/10.1145/3321707.3321829