Using linked data to help robots understand product-related actions

Michaela Kümpel*, Anna de Groot, Ilaria Tiddi, Michael Beetz

*Corresponding author for this work

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review


Household robots need semantics to understand that a detergent is a cleaning product that can be used to clean physical objects like a table, but laundry detergent is only used to clean/wash laundry. A safely acting autonomous robot should also know that both will not be used as ingredients for meal preparation. We propose a new approach to connect robot sensor data to Linked Data in order to give robotic agents semantic product information about objects that can be found in their environment so that the action to be performed with a given object can be inferred. For this, we use the robot’s belief state when recognizing a product and link it to a product ontology that follows Semantic Web standards. We then use the product class information to fetch further information from external sources like Wikidata or ConceptNet that contain action information (e.g. laundry detergent is used for laundering). At last, the action results are mapped to internally known actions of the robotic agent so that it knows which action can be performed with the perceived object.

Original languageEnglish
Title of host publicationJOWO 2020 The Joint Ontology Workshops
Subtitle of host publicationProceedings of the Joint Ontology Workshops co-located with the Bolzano Summer of Knowledge (BOSK 2020) Virtual & Bozen-Bolzano, Italy, August 31st to October 7th, 2020
EditorsKarl Hammar, Oliver Kutz, Anastasia Dimou, Torsten Hahmann, Robert Hoehndorf, Claudio Masolo, Randi Vita
Number of pages7
Publication statusPublished - 24 Oct 2020
Event2020 Joint Ontology Workshops, JOWO 2020 - Virtual, Bozen-Bolzano, Italy
Duration: 31 Aug 20207 Oct 2020

Publication series

NameCEUR Workshop Proceedings
PublisherCEUR Workshop Proceedings
ISSN (Print)1613-0073


Conference2020 Joint Ontology Workshops, JOWO 2020
CityVirtual, Bozen-Bolzano


The research reported in this paper has been partially supported by the Federal Ministry for Economic Affairs and Energy BMWi within the Knowledge4Retail project, subproject semantic Digital Twin 01MK20001M (, the European Commission within the H2020 REFILLS projectID n. 731590, as well as the German Research Foundation DFG, as part of Collaborative Research Center (Sonderforschungsbereich) 1320 ?EASE - Everyday Activity Science and Engineering?, University of Bremen (

FundersFunder number
Everyday Activity Science and Engineering?
Federal Ministry for Economic Affairs and Energy BMWi
German Research Foundation DFG
European Commission731590
Deutsche Forschungsgemeinschaft
Bundesministerium für Wirtschaft und Energie
Universität Bremen


    • Knowledge acquisition
    • Knowledge graph
    • Knowledge representation
    • Linked data
    • Product ontology


    Dive into the research topics of 'Using linked data to help robots understand product-related actions'. Together they form a unique fingerprint.

    Cite this