Automating failure detection in cognitive agent programs

Vincent J. Koeman, Koen V. Hindriks, Catholijn M. Jonker

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

Debugging is notoriously difficult and extremely time consuming but also essential for ensuring the reliability and quality of a software system. In order to reduce debugging effort and enable automated failure detection, we propose an automated testing framework for detecting failures in cognitive agent programs. Our approach is based on the assumption that modules within such programs are a natural unit for testing. We identify a minimal set of temporal operators that enable the specification of test conditions and show that the test language is sufficiently expressive for detecting all failures in an existing failure taxonomy. We also introduce an approach for specifying test templates that supports a programmer in writing tests. Furthermore, empirical analysis of agent programs allows us to evaluate whether our approach using test templates detects all failures.

Original languageEnglish
Title of host publicationAAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems
PublisherInternational Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
Pages1237-1246
Number of pages10
ISBN (Electronic)9781450342391
Publication statusPublished - 1 Jan 2016
Externally publishedYes
Event15th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2016 - Singapore, Singapore
Duration: 9 May 201613 May 2016

Conference

Conference15th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2016
CountrySingapore
CitySingapore
Period9/05/1613/05/16

Fingerprint

Testing
Taxonomies
Specifications

Keywords

  • Multi-agent systems
  • Testing
  • Verification

Cite this

Koeman, V. J., Hindriks, K. V., & Jonker, C. M. (2016). Automating failure detection in cognitive agent programs. In AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems (pp. 1237-1246). International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).
Koeman, Vincent J. ; Hindriks, Koen V. ; Jonker, Catholijn M. / Automating failure detection in cognitive agent programs. AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2016. pp. 1237-1246
@inproceedings{8cbb8e9df42548e9b47844eb54b40e50,
title = "Automating failure detection in cognitive agent programs",
abstract = "Debugging is notoriously difficult and extremely time consuming but also essential for ensuring the reliability and quality of a software system. In order to reduce debugging effort and enable automated failure detection, we propose an automated testing framework for detecting failures in cognitive agent programs. Our approach is based on the assumption that modules within such programs are a natural unit for testing. We identify a minimal set of temporal operators that enable the specification of test conditions and show that the test language is sufficiently expressive for detecting all failures in an existing failure taxonomy. We also introduce an approach for specifying test templates that supports a programmer in writing tests. Furthermore, empirical analysis of agent programs allows us to evaluate whether our approach using test templates detects all failures.",
keywords = "Multi-agent systems, Testing, Verification",
author = "Koeman, {Vincent J.} and Hindriks, {Koen V.} and Jonker, {Catholijn M.}",
year = "2016",
month = "1",
day = "1",
language = "English",
pages = "1237--1246",
booktitle = "AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems",
publisher = "International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)",

}

Koeman, VJ, Hindriks, KV & Jonker, CM 2016, Automating failure detection in cognitive agent programs. in AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), pp. 1237-1246, 15th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2016, Singapore, Singapore, 9/05/16.

Automating failure detection in cognitive agent programs. / Koeman, Vincent J.; Hindriks, Koen V.; Jonker, Catholijn M.

AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2016. p. 1237-1246.

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Automating failure detection in cognitive agent programs

AU - Koeman, Vincent J.

AU - Hindriks, Koen V.

AU - Jonker, Catholijn M.

PY - 2016/1/1

Y1 - 2016/1/1

N2 - Debugging is notoriously difficult and extremely time consuming but also essential for ensuring the reliability and quality of a software system. In order to reduce debugging effort and enable automated failure detection, we propose an automated testing framework for detecting failures in cognitive agent programs. Our approach is based on the assumption that modules within such programs are a natural unit for testing. We identify a minimal set of temporal operators that enable the specification of test conditions and show that the test language is sufficiently expressive for detecting all failures in an existing failure taxonomy. We also introduce an approach for specifying test templates that supports a programmer in writing tests. Furthermore, empirical analysis of agent programs allows us to evaluate whether our approach using test templates detects all failures.

AB - Debugging is notoriously difficult and extremely time consuming but also essential for ensuring the reliability and quality of a software system. In order to reduce debugging effort and enable automated failure detection, we propose an automated testing framework for detecting failures in cognitive agent programs. Our approach is based on the assumption that modules within such programs are a natural unit for testing. We identify a minimal set of temporal operators that enable the specification of test conditions and show that the test language is sufficiently expressive for detecting all failures in an existing failure taxonomy. We also introduce an approach for specifying test templates that supports a programmer in writing tests. Furthermore, empirical analysis of agent programs allows us to evaluate whether our approach using test templates detects all failures.

KW - Multi-agent systems

KW - Testing

KW - Verification

UR - http://www.scopus.com/inward/record.url?scp=85014137631&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85014137631&partnerID=8YFLogxK

M3 - Conference contribution

SP - 1237

EP - 1246

BT - AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems

PB - International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)

ER -

Koeman VJ, Hindriks KV, Jonker CM. Automating failure detection in cognitive agent programs. In AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS). 2016. p. 1237-1246