Improving Confidence in the Estimation of Values and Norms

Luciano Cavalcante Siebert; Rijk Mercuur; Virginia Dignum; Jeroen van den Hoven; Catholijn Jonker

doi:10.1007/978-3-030-72376-7_6

Improving Confidence in the Estimation of Values and Norms

Luciano Cavalcante Siebert
, Rijk Mercuur
, Virginia Dignum
, Jeroen van den Hoven
, Catholijn Jonker

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Autonomous agents (AA) will increasingly be interacting with us in our daily lives. While we want the benefits attached to AAs, it is essential that their behavior is aligned with our values and norms. Hence, an AA will need to estimate the values and norms of the humans it interacts with, which is not a straightforward task when solely observing an agent’s behavior. This paper analyses to what extent an AA is able to estimate the values and norms of a simulated human agent (SHA) based on its actions in the ultimatum game. We present two methods to reduce ambiguity in profiling the SHAs: one based on search space exploration and another based on counterfactual analysis. We found that both methods are able to increase the confidence in estimating human values and norms, but differ in their applicability, the latter being more efficient when the number of interactions with the agent is to be minimized. These insights are useful to improve the alignment of AAs with human values and norms.

Original language	English
Title of host publication	Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII
Subtitle of host publication	International Workshops COIN 2017 and COINE 2020, Sao Paulo, Brazil, May 8-9, 2017 and Virtual Event, May 9, 2020, Revised Selected Papers
Editors	Andrea Aler Tubella, Stephen Cranefield, Christopher Frantz, Felipe Meneguzzi, Wamberto Vasconcelos
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	98-113
Number of pages	16
ISBN (Electronic)	9783030723767
ISBN (Print)	9783030723750
DOIs	https://doi.org/10.1007/978-3-030-72376-7_6
Publication status	Published - 2021
Externally published	Yes
Event	International Workshop on Coordination, Organizations, Institutions, and Norms for Governance of Multi-Agent Systems, COIN 2017, co-located with AAMAS 2017 and the International Workshop on Coordination, Organizations, Institutions, Norms and Ethics for Governance of Multi-Agent Systems, COINE 2020, co-located with AAMAS 2020 - Virtual, Online Duration: 9 May 2020 → 9 May 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Publisher	Springer
Volume	12298 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Name	COINE: International Workshop on Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems
Publisher	Springer
Volume	COIN 2017. COINE 2020

Conference

Conference	International Workshop on Coordination, Organizations, Institutions, and Norms for Governance of Multi-Agent Systems, COIN 2017, co-located with AAMAS 2017 and the International Workshop on Coordination, Organizations, Institutions, Norms and Ethics for Governance of Multi-Agent Systems, COINE 2020, co-located with AAMAS 2020
City	Virtual, Online
Period	9/05/20 → 9/05/20

Funding

Acknowledgements. This work was supported by the AiTech initiative of the Delft University of Technology.

Funders
Technische Universiteit Delft

Access to Document

10.1007/978-3-030-72376-7_6

Persistent URL (handle)

Persistent link

Cite this

Siebert, L. C., Mercuur, R., Dignum, V., van den Hoven, J., & Jonker, C. (2021). Improving Confidence in the Estimation of Values and Norms. In A. Aler Tubella, S. Cranefield, C. Frantz, F. Meneguzzi, & W. Vasconcelos (Eds.), Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII: International Workshops COIN 2017 and COINE 2020, Sao Paulo, Brazil, May 8-9, 2017 and Virtual Event, May 9, 2020, Revised Selected Papers (pp. 98-113). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12298 LNAI), (COINE: International Workshop on Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems; Vol. COIN 2017. COINE 2020). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-72376-7_6

Siebert, Luciano Cavalcante ; Mercuur, Rijk ; Dignum, Virginia et al. / Improving Confidence in the Estimation of Values and Norms. Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII: International Workshops COIN 2017 and COINE 2020, Sao Paulo, Brazil, May 8-9, 2017 and Virtual Event, May 9, 2020, Revised Selected Papers. editor / Andrea Aler Tubella ; Stephen Cranefield ; Christopher Frantz ; Felipe Meneguzzi ; Wamberto Vasconcelos. Springer Science and Business Media Deutschland GmbH, 2021. pp. 98-113 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). (COINE: International Workshop on Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems).

@inproceedings{62354538e7614485875227c613c7d135,

title = "Improving Confidence in the Estimation of Values and Norms",

abstract = "Autonomous agents (AA) will increasingly be interacting with us in our daily lives. While we want the benefits attached to AAs, it is essential that their behavior is aligned with our values and norms. Hence, an AA will need to estimate the values and norms of the humans it interacts with, which is not a straightforward task when solely observing an agent{\textquoteright}s behavior. This paper analyses to what extent an AA is able to estimate the values and norms of a simulated human agent (SHA) based on its actions in the ultimatum game. We present two methods to reduce ambiguity in profiling the SHAs: one based on search space exploration and another based on counterfactual analysis. We found that both methods are able to increase the confidence in estimating human values and norms, but differ in their applicability, the latter being more efficient when the number of interactions with the agent is to be minimized. These insights are useful to improve the alignment of AAs with human values and norms.",

author = "Siebert, \{Luciano Cavalcante\} and Rijk Mercuur and Virginia Dignum and \{van den Hoven\}, Jeroen and Catholijn Jonker",

year = "2021",

doi = "10.1007/978-3-030-72376-7\_6",

language = "English",

isbn = "9783030723750",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "98--113",

editor = "\{Aler Tubella\}, Andrea and Stephen Cranefield and Christopher Frantz and Felipe Meneguzzi and Wamberto Vasconcelos",

booktitle = "Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII",

note = "International Workshop on Coordination, Organizations, Institutions, and Norms for Governance of Multi-Agent Systems, COIN 2017, co-located with AAMAS 2017 and the International Workshop on Coordination, Organizations, Institutions, Norms and Ethics for Governance of Multi-Agent Systems, COINE 2020, co-located with AAMAS 2020 ; Conference date: 09-05-2020 Through 09-05-2020",

}

Siebert, LC, Mercuur, R, Dignum, V, van den Hoven, J & Jonker, C 2021, Improving Confidence in the Estimation of Values and Norms. in A Aler Tubella, S Cranefield, C Frantz, F Meneguzzi & W Vasconcelos (eds), Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII: International Workshops COIN 2017 and COINE 2020, Sao Paulo, Brazil, May 8-9, 2017 and Virtual Event, May 9, 2020, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12298 LNAI, COINE: International Workshop on Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems, vol. COIN 2017. COINE 2020, Springer Science and Business Media Deutschland GmbH, pp. 98-113, International Workshop on Coordination, Organizations, Institutions, and Norms for Governance of Multi-Agent Systems, COIN 2017, co-located with AAMAS 2017 and the International Workshop on Coordination, Organizations, Institutions, Norms and Ethics for Governance of Multi-Agent Systems, COINE 2020, co-located with AAMAS 2020, Virtual, Online, 9/05/20. https://doi.org/10.1007/978-3-030-72376-7_6

Improving Confidence in the Estimation of Values and Norms. / Siebert, Luciano Cavalcante; Mercuur, Rijk; Dignum, Virginia et al.
Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII: International Workshops COIN 2017 and COINE 2020, Sao Paulo, Brazil, May 8-9, 2017 and Virtual Event, May 9, 2020, Revised Selected Papers. ed. / Andrea Aler Tubella; Stephen Cranefield; Christopher Frantz; Felipe Meneguzzi; Wamberto Vasconcelos. Springer Science and Business Media Deutschland GmbH, 2021. p. 98-113 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12298 LNAI), (COINE: International Workshop on Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems; Vol. COIN 2017. COINE 2020).

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Improving Confidence in the Estimation of Values and Norms

AU - Siebert, Luciano Cavalcante

AU - Mercuur, Rijk

AU - Dignum, Virginia

AU - van den Hoven, Jeroen

AU - Jonker, Catholijn

PY - 2021

Y1 - 2021

N2 - Autonomous agents (AA) will increasingly be interacting with us in our daily lives. While we want the benefits attached to AAs, it is essential that their behavior is aligned with our values and norms. Hence, an AA will need to estimate the values and norms of the humans it interacts with, which is not a straightforward task when solely observing an agent’s behavior. This paper analyses to what extent an AA is able to estimate the values and norms of a simulated human agent (SHA) based on its actions in the ultimatum game. We present two methods to reduce ambiguity in profiling the SHAs: one based on search space exploration and another based on counterfactual analysis. We found that both methods are able to increase the confidence in estimating human values and norms, but differ in their applicability, the latter being more efficient when the number of interactions with the agent is to be minimized. These insights are useful to improve the alignment of AAs with human values and norms.

AB - Autonomous agents (AA) will increasingly be interacting with us in our daily lives. While we want the benefits attached to AAs, it is essential that their behavior is aligned with our values and norms. Hence, an AA will need to estimate the values and norms of the humans it interacts with, which is not a straightforward task when solely observing an agent’s behavior. This paper analyses to what extent an AA is able to estimate the values and norms of a simulated human agent (SHA) based on its actions in the ultimatum game. We present two methods to reduce ambiguity in profiling the SHAs: one based on search space exploration and another based on counterfactual analysis. We found that both methods are able to increase the confidence in estimating human values and norms, but differ in their applicability, the latter being more efficient when the number of interactions with the agent is to be minimized. These insights are useful to improve the alignment of AAs with human values and norms.

UR - https://www.scopus.com/pages/publications/85107492756

UR - https://www.scopus.com/pages/publications/85107492756#tab=citedBy

U2 - 10.1007/978-3-030-72376-7_6

DO - 10.1007/978-3-030-72376-7_6

M3 - Conference contribution

SN - 9783030723750

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 98

EP - 113

BT - Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII

A2 - Aler Tubella, Andrea

A2 - Cranefield, Stephen

A2 - Frantz, Christopher

A2 - Meneguzzi, Felipe

A2 - Vasconcelos, Wamberto

PB - Springer Science and Business Media Deutschland GmbH

T2 - International Workshop on Coordination, Organizations, Institutions, and Norms for Governance of Multi-Agent Systems, COIN 2017, co-located with AAMAS 2017 and the International Workshop on Coordination, Organizations, Institutions, Norms and Ethics for Governance of Multi-Agent Systems, COINE 2020, co-located with AAMAS 2020

Y2 - 9 May 2020 through 9 May 2020

ER -

Siebert LC, Mercuur R, Dignum V, van den Hoven J, Jonker C. Improving Confidence in the Estimation of Values and Norms. In Aler Tubella A, Cranefield S, Frantz C, Meneguzzi F, Vasconcelos W, editors, Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII: International Workshops COIN 2017 and COINE 2020, Sao Paulo, Brazil, May 8-9, 2017 and Virtual Event, May 9, 2020, Revised Selected Papers. Springer Science and Business Media Deutschland GmbH. 2021. p. 98-113. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). (COINE: International Workshop on Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems). Epub 2021 Apr 2. doi: 10.1007/978-3-030-72376-7_6

Improving Confidence in the Estimation of Values and Norms

Abstract

Publication series

Conference

Funding

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this