Surrogate models for enhancing the efficiency of neuroevolution in reinforcement learning

Jörg Stork, Thomas Bartz-Beielstein, Martin Zaefferer, A. E. Eiben

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review


In the last years, reinforcement learning received a lot of attention. One method to solve reinforcement learning tasks is Neuroevolution, where neural networks are optimized by evolutionary algorithms. A disadvantage of Neuroevolution is that it can require numerous function evaluations, while not fully utilizing the available information from each fitness evaluation. This is especially problematic when fitness evaluations become expensive. To reduce the cost of fitness evaluations, surrogate models can be employed to partially replace the fitness function. The difficulty of surrogate modeling for Neuroevolution is the complex search space and how to compare different networks. To that end, recent studies showed that a kernel based approach, particular with phenotypic distance measures, works well. These kernels compare different networks via their behavior (phenotype) rather than their topology or encoding (genotype). In this work, we discuss the use of surrogate model-based Neuroevolution (SMB-NE) using a phenotypic distance for reinforcement learning. In detail, we investigate a) the potential of SMB-NE with respect to evaluation efficiency and b) how to select adequate input sets for the phenotypic distance measure in a reinforcement learning problem. The results indicate that we are able to considerably increase the evaluation efficiency using dynamic input sets.

Original languageEnglish
Title of host publicationGECCO 2019 - Proceedings of the 2019 Genetic and Evolutionary Computation Conference
PublisherAssociation for Computing Machinery, Inc
Number of pages9
ISBN (Electronic)9781450361118
Publication statusPublished - 13 Jul 2019
Event2019 Genetic and Evolutionary Computation Conference, GECCO 2019 - Prague, Czech Republic
Duration: 13 Jul 201917 Jul 2019


Conference2019 Genetic and Evolutionary Computation Conference, GECCO 2019
Country/TerritoryCzech Republic


  • Neuroevolution
  • Reinforcement Learning
  • Surrogate Models


Dive into the research topics of 'Surrogate models for enhancing the efficiency of neuroevolution in reinforcement learning'. Together they form a unique fingerprint.

Cite this