Using automatic failure detection for cognitive agents in eclipse (AAMAS 2016 DEMONSTRATION)

Vincent J. Koeman, Koen V. Hindriks, Catholijn M. Jonker

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

In order to reduce debugging effort and enable automated failure detection, we proposed an automated testing framework for detecting failures in cognitive agent programs in previous work. This approach is based on a minimal set of temporal operators that enable the specification of test conditions with sufficient expressiveness for detecting all failures in an existing failure taxonomy. We also introduced an according approach for specifying test templates that supports a programmer in writing tests. In this demonstration paper, the automated test framework for the Goal agent programming language that has been created for the Eclipse platform is introduced, with a focus on its practical aspects, i.e., how to use it to detect failures in cognitive agents. As fault localization is an important follow-up to failure detection, the integration of the test framework in the existing source-level debugger for Goal is discussed as well. In addition, an empirical evaluation of the automated testing framework implementation for Goal is presented based on the work of almost 200 novice agent programmers.

Original languageEnglish
Title of host publicationEngineering Multi-Agent Systems - 4th International Workshop, EMAS 2016, Revised, Selected, and Invited Papers
EditorsJorg P. Muller, Ingrid Nunes, Matteo Baldoni, Rym Zalila-Wenkstern
PublisherSpringer/Verlag
Pages59-80
Number of pages22
ISBN (Print)9783319509822
DOIs
Publication statusPublished - 1 Jan 2016
Externally publishedYes
Event4th International Workshop on Engineering Multi-Agent Systems, EMAS 2016 - Singapore, Singapore
Duration: 9 May 201610 May 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10093 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference4th International Workshop on Engineering Multi-Agent Systems, EMAS 2016
CountrySingapore
CitySingapore
Period9/05/1610/05/16

Fingerprint

Failure Detection
Testing
Taxonomies
Computer programming languages
Debugging
Minimal Set
Expressiveness
Taxonomy
Demonstrations
Programming Languages
Template
Specifications
Fault
Specification
Sufficient
Framework
Evaluation
Operator

Cite this

Koeman, V. J., Hindriks, K. V., & Jonker, C. M. (2016). Using automatic failure detection for cognitive agents in eclipse (AAMAS 2016 DEMONSTRATION). In J. P. Muller, I. Nunes, M. Baldoni, & R. Zalila-Wenkstern (Eds.), Engineering Multi-Agent Systems - 4th International Workshop, EMAS 2016, Revised, Selected, and Invited Papers (pp. 59-80). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10093 LNAI). Springer/Verlag. https://doi.org/10.1007/978-3-319-50983-9_4
Koeman, Vincent J. ; Hindriks, Koen V. ; Jonker, Catholijn M. / Using automatic failure detection for cognitive agents in eclipse (AAMAS 2016 DEMONSTRATION). Engineering Multi-Agent Systems - 4th International Workshop, EMAS 2016, Revised, Selected, and Invited Papers. editor / Jorg P. Muller ; Ingrid Nunes ; Matteo Baldoni ; Rym Zalila-Wenkstern. Springer/Verlag, 2016. pp. 59-80 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{71c95025523e46bcb9a8db220bea964c,
title = "Using automatic failure detection for cognitive agents in eclipse (AAMAS 2016 DEMONSTRATION)",
abstract = "In order to reduce debugging effort and enable automated failure detection, we proposed an automated testing framework for detecting failures in cognitive agent programs in previous work. This approach is based on a minimal set of temporal operators that enable the specification of test conditions with sufficient expressiveness for detecting all failures in an existing failure taxonomy. We also introduced an according approach for specifying test templates that supports a programmer in writing tests. In this demonstration paper, the automated test framework for the Goal agent programming language that has been created for the Eclipse platform is introduced, with a focus on its practical aspects, i.e., how to use it to detect failures in cognitive agents. As fault localization is an important follow-up to failure detection, the integration of the test framework in the existing source-level debugger for Goal is discussed as well. In addition, an empirical evaluation of the automated testing framework implementation for Goal is presented based on the work of almost 200 novice agent programmers.",
author = "Koeman, {Vincent J.} and Hindriks, {Koen V.} and Jonker, {Catholijn M.}",
year = "2016",
month = "1",
day = "1",
doi = "10.1007/978-3-319-50983-9_4",
language = "English",
isbn = "9783319509822",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer/Verlag",
pages = "59--80",
editor = "Muller, {Jorg P.} and Ingrid Nunes and Matteo Baldoni and Rym Zalila-Wenkstern",
booktitle = "Engineering Multi-Agent Systems - 4th International Workshop, EMAS 2016, Revised, Selected, and Invited Papers",

}

Koeman, VJ, Hindriks, KV & Jonker, CM 2016, Using automatic failure detection for cognitive agents in eclipse (AAMAS 2016 DEMONSTRATION). in JP Muller, I Nunes, M Baldoni & R Zalila-Wenkstern (eds), Engineering Multi-Agent Systems - 4th International Workshop, EMAS 2016, Revised, Selected, and Invited Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10093 LNAI, Springer/Verlag, pp. 59-80, 4th International Workshop on Engineering Multi-Agent Systems, EMAS 2016, Singapore, Singapore, 9/05/16. https://doi.org/10.1007/978-3-319-50983-9_4

Using automatic failure detection for cognitive agents in eclipse (AAMAS 2016 DEMONSTRATION). / Koeman, Vincent J.; Hindriks, Koen V.; Jonker, Catholijn M.

Engineering Multi-Agent Systems - 4th International Workshop, EMAS 2016, Revised, Selected, and Invited Papers. ed. / Jorg P. Muller; Ingrid Nunes; Matteo Baldoni; Rym Zalila-Wenkstern. Springer/Verlag, 2016. p. 59-80 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10093 LNAI).

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Using automatic failure detection for cognitive agents in eclipse (AAMAS 2016 DEMONSTRATION)

AU - Koeman, Vincent J.

AU - Hindriks, Koen V.

AU - Jonker, Catholijn M.

PY - 2016/1/1

Y1 - 2016/1/1

N2 - In order to reduce debugging effort and enable automated failure detection, we proposed an automated testing framework for detecting failures in cognitive agent programs in previous work. This approach is based on a minimal set of temporal operators that enable the specification of test conditions with sufficient expressiveness for detecting all failures in an existing failure taxonomy. We also introduced an according approach for specifying test templates that supports a programmer in writing tests. In this demonstration paper, the automated test framework for the Goal agent programming language that has been created for the Eclipse platform is introduced, with a focus on its practical aspects, i.e., how to use it to detect failures in cognitive agents. As fault localization is an important follow-up to failure detection, the integration of the test framework in the existing source-level debugger for Goal is discussed as well. In addition, an empirical evaluation of the automated testing framework implementation for Goal is presented based on the work of almost 200 novice agent programmers.

AB - In order to reduce debugging effort and enable automated failure detection, we proposed an automated testing framework for detecting failures in cognitive agent programs in previous work. This approach is based on a minimal set of temporal operators that enable the specification of test conditions with sufficient expressiveness for detecting all failures in an existing failure taxonomy. We also introduced an according approach for specifying test templates that supports a programmer in writing tests. In this demonstration paper, the automated test framework for the Goal agent programming language that has been created for the Eclipse platform is introduced, with a focus on its practical aspects, i.e., how to use it to detect failures in cognitive agents. As fault localization is an important follow-up to failure detection, the integration of the test framework in the existing source-level debugger for Goal is discussed as well. In addition, an empirical evaluation of the automated testing framework implementation for Goal is presented based on the work of almost 200 novice agent programmers.

UR - http://www.scopus.com/inward/record.url?scp=85009758493&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85009758493&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-50983-9_4

DO - 10.1007/978-3-319-50983-9_4

M3 - Conference contribution

SN - 9783319509822

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 59

EP - 80

BT - Engineering Multi-Agent Systems - 4th International Workshop, EMAS 2016, Revised, Selected, and Invited Papers

A2 - Muller, Jorg P.

A2 - Nunes, Ingrid

A2 - Baldoni, Matteo

A2 - Zalila-Wenkstern, Rym

PB - Springer/Verlag

ER -

Koeman VJ, Hindriks KV, Jonker CM. Using automatic failure detection for cognitive agents in eclipse (AAMAS 2016 DEMONSTRATION). In Muller JP, Nunes I, Baldoni M, Zalila-Wenkstern R, editors, Engineering Multi-Agent Systems - 4th International Workshop, EMAS 2016, Revised, Selected, and Invited Papers. Springer/Verlag. 2016. p. 59-80. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-50983-9_4