The effect of video exemplars on interrater reliability in assessment of parental behavior

Research output: Contribution to ConferenceAbstractAcademic

Abstract

In the process of developing an observational instrument for assessing parental behavior for use in daily practice of child protection services and other institutions working with (at risk) families with young children, it is important to make sure that the scale instructions will reduce time of training and becoming a reliable coder as much as possible, that the format allows quick and efficient application in clinical use cases, and that repeated reliability checks can be easily configured to reduce coder drift. Establishing a positive user experience will also contribute to quick and efficient application. In order to gain information about what type of instruction, visual or written, will fit these requirements, the current study is being performed. The study tests whether videobased descriptions of scale anchors may contribute towards adapting rating scale instruments to become more efficient. Our hypothesis is that in the condition where raters use a video-scale, interrater reliability will be higher on average, interrater reliability will increase faster and the user experience will be more positive than in the condition where raters use a traditional written scale. The research is conducted with two groups of participants assigned to two research conditions, video-based anchors and text-based anchors. Each participant in both groups will observe six target videos and then rate parental sensitivity according to either the video-anchor scale or the text-anchor scale. Also, they will answer a questionnaire about their user experience. Rater accuracy, interrater reliability, mean time needed for coding, and user experience will be calculated for each condition, and compared between the two conditions to establish whether they differ significant. Results of this study are expected beginning of 2018.
Original languageEnglish
Pages56-57
Publication statusPublished - 26 May 2018
EventWAIMH: 2018 - Rome, Italy
Duration: 26 May 201830 May 2018
http://www.waimh2018.org

Conference

ConferenceWAIMH
CountryItaly
CityRome
Period26/05/1830/05/18
Internet address

Fingerprint

Anchors
Instrument scales

Bibliographical note

Published in INFANT MENTAL HEALTH JOURNAL Supplement to Volume 39

Cite this

@conference{e22081c5b3f64488af4c4ef0224a1d01,
title = "The effect of video exemplars on interrater reliability in assessment of parental behavior",
abstract = "In the process of developing an observational instrument for assessing parental behavior for use in daily practice of child protection services and other institutions working with (at risk) families with young children, it is important to make sure that the scale instructions will reduce time of training and becoming a reliable coder as much as possible, that the format allows quick and efficient application in clinical use cases, and that repeated reliability checks can be easily configured to reduce coder drift. Establishing a positive user experience will also contribute to quick and efficient application. In order to gain information about what type of instruction, visual or written, will fit these requirements, the current study is being performed. The study tests whether videobased descriptions of scale anchors may contribute towards adapting rating scale instruments to become more efficient. Our hypothesis is that in the condition where raters use a video-scale, interrater reliability will be higher on average, interrater reliability will increase faster and the user experience will be more positive than in the condition where raters use a traditional written scale. The research is conducted with two groups of participants assigned to two research conditions, video-based anchors and text-based anchors. Each participant in both groups will observe six target videos and then rate parental sensitivity according to either the video-anchor scale or the text-anchor scale. Also, they will answer a questionnaire about their user experience. Rater accuracy, interrater reliability, mean time needed for coding, and user experience will be calculated for each condition, and compared between the two conditions to establish whether they differ significant. Results of this study are expected beginning of 2018.",
author = "Mirte Forrer and Carlo Schuengel and M. Oosterman",
note = "Published in INFANT MENTAL HEALTH JOURNAL Supplement to Volume 39; WAIMH : 2018 ; Conference date: 26-05-2018 Through 30-05-2018",
year = "2018",
month = "5",
day = "26",
language = "English",
pages = "56--57",
url = "http://www.waimh2018.org",

}

The effect of video exemplars on interrater reliability in assessment of parental behavior. / Forrer, Mirte; Schuengel, Carlo; Oosterman, M.

2018. 56-57 Abstract from WAIMH, Rome, Italy.

Research output: Contribution to ConferenceAbstractAcademic

TY - CONF

T1 - The effect of video exemplars on interrater reliability in assessment of parental behavior

AU - Forrer, Mirte

AU - Schuengel, Carlo

AU - Oosterman, M.

N1 - Published in INFANT MENTAL HEALTH JOURNAL Supplement to Volume 39

PY - 2018/5/26

Y1 - 2018/5/26

N2 - In the process of developing an observational instrument for assessing parental behavior for use in daily practice of child protection services and other institutions working with (at risk) families with young children, it is important to make sure that the scale instructions will reduce time of training and becoming a reliable coder as much as possible, that the format allows quick and efficient application in clinical use cases, and that repeated reliability checks can be easily configured to reduce coder drift. Establishing a positive user experience will also contribute to quick and efficient application. In order to gain information about what type of instruction, visual or written, will fit these requirements, the current study is being performed. The study tests whether videobased descriptions of scale anchors may contribute towards adapting rating scale instruments to become more efficient. Our hypothesis is that in the condition where raters use a video-scale, interrater reliability will be higher on average, interrater reliability will increase faster and the user experience will be more positive than in the condition where raters use a traditional written scale. The research is conducted with two groups of participants assigned to two research conditions, video-based anchors and text-based anchors. Each participant in both groups will observe six target videos and then rate parental sensitivity according to either the video-anchor scale or the text-anchor scale. Also, they will answer a questionnaire about their user experience. Rater accuracy, interrater reliability, mean time needed for coding, and user experience will be calculated for each condition, and compared between the two conditions to establish whether they differ significant. Results of this study are expected beginning of 2018.

AB - In the process of developing an observational instrument for assessing parental behavior for use in daily practice of child protection services and other institutions working with (at risk) families with young children, it is important to make sure that the scale instructions will reduce time of training and becoming a reliable coder as much as possible, that the format allows quick and efficient application in clinical use cases, and that repeated reliability checks can be easily configured to reduce coder drift. Establishing a positive user experience will also contribute to quick and efficient application. In order to gain information about what type of instruction, visual or written, will fit these requirements, the current study is being performed. The study tests whether videobased descriptions of scale anchors may contribute towards adapting rating scale instruments to become more efficient. Our hypothesis is that in the condition where raters use a video-scale, interrater reliability will be higher on average, interrater reliability will increase faster and the user experience will be more positive than in the condition where raters use a traditional written scale. The research is conducted with two groups of participants assigned to two research conditions, video-based anchors and text-based anchors. Each participant in both groups will observe six target videos and then rate parental sensitivity according to either the video-anchor scale or the text-anchor scale. Also, they will answer a questionnaire about their user experience. Rater accuracy, interrater reliability, mean time needed for coding, and user experience will be calculated for each condition, and compared between the two conditions to establish whether they differ significant. Results of this study are expected beginning of 2018.

M3 - Abstract

SP - 56

EP - 57

ER -