A Practical Procedure for the Construction and Reliability Analysis of Fixed Length Tests with Random Drawn Test Items

S. Draaijer, S. Klinkenberg

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

A procedure to construct valid and fair fixed-length tests with randomly drawn items from an item bank is described. The procedure provides guidelines for the set-up of a typical achievement test with regard to the number of items in the bank and the number of items for each position in a test. Further, a procedure is proposed to calculate the relative difficulty for individual tests and to correct the obtained score for each student based on the mean difficulty for all students and the particular test of a student. Also, two procedures are proposed for the problem to calculate the reliability of tests with randomly drawn items. The procedures use specific interpretations of regularly used methods to calculate Cronbach’s alpha and KR20 and the Spearman-Brown prediction formula. A simulation with R is presented to illustrate the accuracy of the calculation procedures and the effects on pass-fail decisions.
Original languageEnglish
Title of host publicationComputer assisted assessment: research into e-assessment
EditorsE. Ras, D. Joosten-Ten Brinke
PublisherSpinger
Pages47-60
DOIs
Publication statusPublished - 2015
EventComputer assisted assessment: research into e-assessment -
Duration: 1 Jan 20151 Jan 2015

Conference

ConferenceComputer assisted assessment: research into e-assessment
Period1/01/151/01/15

Fingerprint

bank
student
achievement test
simulation
interpretation

Cite this

Draaijer, S., & Klinkenberg, S. (2015). A Practical Procedure for the Construction and Reliability Analysis of Fixed Length Tests with Random Drawn Test Items. In E. Ras, & D. Joosten-Ten Brinke (Eds.), Computer assisted assessment: research into e-assessment (pp. 47-60). Spinger. https://doi.org/10.1007/978-3-319-27704-2_6
Draaijer, S. ; Klinkenberg, S. / A Practical Procedure for the Construction and Reliability Analysis of Fixed Length Tests with Random Drawn Test Items. Computer assisted assessment: research into e-assessment. editor / E. Ras ; D. Joosten-Ten Brinke. Spinger, 2015. pp. 47-60
@inproceedings{a8fe888c981c4b4d8c52c755f4850251,
title = "A Practical Procedure for the Construction and Reliability Analysis of Fixed Length Tests with Random Drawn Test Items",
abstract = "A procedure to construct valid and fair fixed-length tests with randomly drawn items from an item bank is described. The procedure provides guidelines for the set-up of a typical achievement test with regard to the number of items in the bank and the number of items for each position in a test. Further, a procedure is proposed to calculate the relative difficulty for individual tests and to correct the obtained score for each student based on the mean difficulty for all students and the particular test of a student. Also, two procedures are proposed for the problem to calculate the reliability of tests with randomly drawn items. The procedures use specific interpretations of regularly used methods to calculate Cronbach’s alpha and KR20 and the Spearman-Brown prediction formula. A simulation with R is presented to illustrate the accuracy of the calculation procedures and the effects on pass-fail decisions.",
author = "S. Draaijer and S. Klinkenberg",
year = "2015",
doi = "10.1007/978-3-319-27704-2_6",
language = "English",
pages = "47--60",
editor = "E. Ras and {Joosten-Ten Brinke}, D.",
booktitle = "Computer assisted assessment: research into e-assessment",
publisher = "Spinger",

}

Draaijer, S & Klinkenberg, S 2015, A Practical Procedure for the Construction and Reliability Analysis of Fixed Length Tests with Random Drawn Test Items. in E Ras & D Joosten-Ten Brinke (eds), Computer assisted assessment: research into e-assessment. Spinger, pp. 47-60, Computer assisted assessment: research into e-assessment, 1/01/15. https://doi.org/10.1007/978-3-319-27704-2_6

A Practical Procedure for the Construction and Reliability Analysis of Fixed Length Tests with Random Drawn Test Items. / Draaijer, S.; Klinkenberg, S.

Computer assisted assessment: research into e-assessment. ed. / E. Ras; D. Joosten-Ten Brinke. Spinger, 2015. p. 47-60.

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - A Practical Procedure for the Construction and Reliability Analysis of Fixed Length Tests with Random Drawn Test Items

AU - Draaijer, S.

AU - Klinkenberg, S.

PY - 2015

Y1 - 2015

N2 - A procedure to construct valid and fair fixed-length tests with randomly drawn items from an item bank is described. The procedure provides guidelines for the set-up of a typical achievement test with regard to the number of items in the bank and the number of items for each position in a test. Further, a procedure is proposed to calculate the relative difficulty for individual tests and to correct the obtained score for each student based on the mean difficulty for all students and the particular test of a student. Also, two procedures are proposed for the problem to calculate the reliability of tests with randomly drawn items. The procedures use specific interpretations of regularly used methods to calculate Cronbach’s alpha and KR20 and the Spearman-Brown prediction formula. A simulation with R is presented to illustrate the accuracy of the calculation procedures and the effects on pass-fail decisions.

AB - A procedure to construct valid and fair fixed-length tests with randomly drawn items from an item bank is described. The procedure provides guidelines for the set-up of a typical achievement test with regard to the number of items in the bank and the number of items for each position in a test. Further, a procedure is proposed to calculate the relative difficulty for individual tests and to correct the obtained score for each student based on the mean difficulty for all students and the particular test of a student. Also, two procedures are proposed for the problem to calculate the reliability of tests with randomly drawn items. The procedures use specific interpretations of regularly used methods to calculate Cronbach’s alpha and KR20 and the Spearman-Brown prediction formula. A simulation with R is presented to illustrate the accuracy of the calculation procedures and the effects on pass-fail decisions.

U2 - 10.1007/978-3-319-27704-2_6

DO - 10.1007/978-3-319-27704-2_6

M3 - Conference contribution

SP - 47

EP - 60

BT - Computer assisted assessment: research into e-assessment

A2 - Ras, E.

A2 - Joosten-Ten Brinke, D.

PB - Spinger

ER -

Draaijer S, Klinkenberg S. A Practical Procedure for the Construction and Reliability Analysis of Fixed Length Tests with Random Drawn Test Items. In Ras E, Joosten-Ten Brinke D, editors, Computer assisted assessment: research into e-assessment. Spinger. 2015. p. 47-60 https://doi.org/10.1007/978-3-319-27704-2_6