Human-interactive Agent Learning

URL study guide

https://studiegids.vu.nl/en/courses/2024-2025/XM_0146

Course Objective

At the end of the course, students will be:
- familiar with basic knowledge of some of the core aspects of human-interactive agent learning (HIAL), including imitation learning, learning from different kinds of human feedback, preference-based learning, teaching signals, interaction problems in HIAL, and evaluation metrics. (Knowledge and understanding)
- able to program, evaluate, and discuss results of basic HIAL algorithms, as well as critically design an interaction between a human teacher and a learning robot. (Applying knowledge and understanding, Making judgements)
- able to engage in groupwork and orally present results of an open-ended research-flavored assignment. (Communication skills)
- trained in acquiring a set of interdisciplinary HIAL-related topics at the intersection of machine learning, human factors, and robotics in a restricted period of time, making them equipped for engaging in research in this emerging field (Learning skills)

Course Content

Human-Interactive Agent Learning (HIAL) is an area of interactive AI that focuses on developing agents (e.g., robots, artificial assistants) that can improve their task performance through interaction with humans. This course aims to cover basic HIAL principles and techniques, drawing from the fields of Machine Learning, Human Factors, Behavior Modelling, Robotics, and Design. The course will delve into different classes of interactive machine learning algorithms that can be used to build HIAL systems, including imitation learning (e.g., behavioral cloning, inverse reinforcement learning), learning from evaluative feedback (e.g., TAMER, policy shaping), preference-based learning (e.g., learning from ranked trajectories), and learning from corrective feedback (e.g, Expert Intervention Learning). In addition to learning algorithms, students will be exposed to human-centered concepts related to situated interaction with such systems, including teaching signals, usability, human bias and limitations, transparency, alignment and synergy. The course is meant to provide students with a good algorithmic foundation while fostering broader critical thinking about human-agent interactions with teacher-learner dynamics.

Teaching Methods

The course will combine lectures with individual in-lab practical assignments. There will be a small group assignment towards the end of the course, as well as a final exam. There will be a significant amount of self-study from lecture materials and readings.

Method of Assessment

Digital exam (50%) at least 5.5/10 to pass the course Practicals (30%) Group assignment presentation (20%) There will be a resit exam, but NO resit for the group assignment.

Literature

Academic papers and other literature will be shared during the course to complement lecture content.

Target Audience

MSc Artificial Intelligence (elective)

Recommended background knowledge

Basic concepts of Reinforcement Learning Basic concepts of Probability Theory and Statistics Basic concepts of Calculus