Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

Amjad Yousef Majid; Serge Saaybi; Vincent Francois-Lavet; R. Venkatesha Prasad; Chris Verhoeven

doi:10.1109/TNNLS.2023.3264540

Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

Amjad Yousef Majid
, Serge Saaybi
, Vincent Francois-Lavet
, R. Venkatesha Prasad
, Chris Verhoeven

Research output: Contribution to Journal › Article › Academic › peer-review

Abstract

Deep reinforcement learning (DRL) and evolution strategies (ESs) have surpassed human-level control in many sequential decision-making problems, yet many open challenges still exist. To get insights into the strengths and weaknesses of DRL versus ESs, an analysis of their respective capabilities and limitations is provided. After presenting their fundamental concepts and algorithms, a comparison is provided on key aspects, such as scalability, exploration, adaptation to dynamic environments, and multiagent learning. Current research challenges are also discussed, including sample efficiency, exploration versus exploitation, dealing with sparse rewards, and learning to plan. Then, the benefits of hybrid algorithms that combine DRL and ESs are highlighted.

Original language	English
Pages (from-to)	11939-11957
Number of pages	19
Journal	IEEE Transactions on Neural Networks and Learning Systems
Volume	35
Issue number	9
Early online date	2 May 2023
DOIs	https://doi.org/10.1109/TNNLS.2023.3264540
Publication status	Published - Sept 2024

Bibliographical note

Publisher Copyright:
Author

Funding

This work was supported in part by Cognizant Technology Solutions through the Internet of Swarms Project and in part by Rijksdienst voor Ondernemend Nederland under PPS O&I.

Funders
Rijksdienst voor Ondernemend Nederland

Keywords

Deep learning
Deep reinforcement learning (DRL)
Evolution (biology)
evolution strategies (ESs)
exploration
Games
meta-learning
multiagent
Optimization
parallelism
Q-learning
Robots
Scalability

Access to Document

10.1109/TNNLS.2023.3264540

Persistent URL (handle)

Persistent link

Cite this

@article{fa03b103356148feb2e7789e4992c6e6,

title = "Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey",

abstract = "Deep reinforcement learning (DRL) and evolution strategies (ESs) have surpassed human-level control in many sequential decision-making problems, yet many open challenges still exist. To get insights into the strengths and weaknesses of DRL versus ESs, an analysis of their respective capabilities and limitations is provided. After presenting their fundamental concepts and algorithms, a comparison is provided on key aspects, such as scalability, exploration, adaptation to dynamic environments, and multiagent learning. Current research challenges are also discussed, including sample efficiency, exploration versus exploitation, dealing with sparse rewards, and learning to plan. Then, the benefits of hybrid algorithms that combine DRL and ESs are highlighted.",

keywords = "Deep learning, Deep reinforcement learning (DRL), Evolution (biology), evolution strategies (ESs), exploration, Games, meta-learning, multiagent, Optimization, parallelism, Q-learning, Robots, Scalability",

author = "Majid, \{Amjad Yousef\} and Serge Saaybi and Vincent Francois-Lavet and Prasad, \{R. Venkatesha\} and Chris Verhoeven",

note = "Publisher Copyright: Author",

year = "2024",

month = sep,

doi = "10.1109/TNNLS.2023.3264540",

language = "English",

volume = "35",

pages = "11939--11957",

journal = "IEEE Transactions on Neural Networks and Learning Systems",

issn = "2162-237X",

publisher = "IEEE Computational Intelligence Society",

number = "9",

}

TY - JOUR

T1 - Deep Reinforcement Learning Versus Evolution Strategies

T2 - A Comparative Survey

AU - Majid, Amjad Yousef

AU - Saaybi, Serge

AU - Francois-Lavet, Vincent

AU - Prasad, R. Venkatesha

AU - Verhoeven, Chris

N1 - Publisher Copyright: Author

PY - 2024/9

Y1 - 2024/9

N2 - Deep reinforcement learning (DRL) and evolution strategies (ESs) have surpassed human-level control in many sequential decision-making problems, yet many open challenges still exist. To get insights into the strengths and weaknesses of DRL versus ESs, an analysis of their respective capabilities and limitations is provided. After presenting their fundamental concepts and algorithms, a comparison is provided on key aspects, such as scalability, exploration, adaptation to dynamic environments, and multiagent learning. Current research challenges are also discussed, including sample efficiency, exploration versus exploitation, dealing with sparse rewards, and learning to plan. Then, the benefits of hybrid algorithms that combine DRL and ESs are highlighted.

AB - Deep reinforcement learning (DRL) and evolution strategies (ESs) have surpassed human-level control in many sequential decision-making problems, yet many open challenges still exist. To get insights into the strengths and weaknesses of DRL versus ESs, an analysis of their respective capabilities and limitations is provided. After presenting their fundamental concepts and algorithms, a comparison is provided on key aspects, such as scalability, exploration, adaptation to dynamic environments, and multiagent learning. Current research challenges are also discussed, including sample efficiency, exploration versus exploitation, dealing with sparse rewards, and learning to plan. Then, the benefits of hybrid algorithms that combine DRL and ESs are highlighted.

KW - Deep learning

KW - Deep reinforcement learning (DRL)

KW - Evolution (biology)

KW - evolution strategies (ESs)

KW - exploration

KW - Games

KW - meta-learning

KW - multiagent

KW - Optimization

KW - parallelism

KW - Q-learning

KW - Robots

KW - Scalability

UR - https://www.scopus.com/pages/publications/85159827870

UR - https://www.scopus.com/pages/publications/85159827870#tab=citedBy

U2 - 10.1109/TNNLS.2023.3264540

DO - 10.1109/TNNLS.2023.3264540

M3 - Article

AN - SCOPUS:85159827870

SN - 2162-237X

VL - 35

SP - 11939

EP - 11957

JO - IEEE Transactions on Neural Networks and Learning Systems

JF - IEEE Transactions on Neural Networks and Learning Systems

IS - 9

ER -

Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

Abstract

Bibliographical note

Funding

Keywords

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this