Learning to Coordinate with Coordination Graphs in Repeated Single-Stage Multi-Agent Decision Problems

Eugenio Bargiacchi, Timothy Verstraeten, D.M. Roijers, Ann Nowé, Hado Van Hasselt

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

Learning to coordinate between multiple agents is an important problem in many reinforcement learning problems. Key to learning to coordinate is exploiting loose couplings, i.e., conditional independences between agents. In this paper we study learning in repeated fully cooperative games, multi-agent multi-armed bandits (MAMABs), in which the expected rewards can be expressed as a coordination graph. We propose multi-agent upper confidence exploration (MAUCE), a new algorithm for MAMABs that exploits loose couplings, which enables us to prove a regret bound that is logarithmic in the number of arm pulls and only linear in the number of agents. We empirically compare MAUCE to sparse cooperative Q-learning, and a state-of-the-art combinatorial bandit approach, and show that it performs much better on a variety of settings, including learning control policies for wind farms.
Original languageEnglish
Title of host publicationInternational Conference on Machine Learning, 10-15 July 2018, Stockholmsmässan, Stockholm Sweden
Subtitle of host publicationProceedings of the 35th International Conference on Machine Learning, ICML 2018
Place of PublicationStockholm
Pages482-490
Number of pages9
Publication statusPublished - 2018
EventInternational Conference on Machine Learning - Stockholmsmässan, Stockholm, Sweden
Duration: 9 Jul 2018 → …
Conference number: 35
https://icml.cc/

Publication series

NameProceedings of Machine Learning Research
Volume80
ISSN (Electronic)1938-7228

Conference

ConferenceInternational Conference on Machine Learning
Abbreviated titleICML
CountrySweden
CityStockholm
Period9/07/18 → …
Internet address

Keywords

  • Multi-agent
  • reinforcement learning
  • Coordination
  • Wind Energy

Fingerprint Dive into the research topics of 'Learning to Coordinate with Coordination Graphs in Repeated Single-Stage Multi-Agent Decision Problems'. Together they form a unique fingerprint.

  • Cite this

    Bargiacchi, E., Verstraeten, T., Roijers, D. M., Nowé, A., & Van Hasselt, H. (2018). Learning to Coordinate with Coordination Graphs in Repeated Single-Stage Multi-Agent Decision Problems. In International Conference on Machine Learning, 10-15 July 2018, Stockholmsmässan, Stockholm Sweden : Proceedings of the 35th International Conference on Machine Learning, ICML 2018 (pp. 482-490). (Proceedings of Machine Learning Research; Vol. 80). Stockholm.