Evaluating municipal websites: A methodological comparison of three think-aloud variants

M.J. van den Haak, M.D.T. de Jong, P.J. Schellens

    Research output: Contribution to JournalArticleAcademicpeer-review

    Abstract

    Usability methods have received relatively little methodological attention within the field of E-Government. This paper aims to address this gap by reporting on a usability test of the municipal website of Deventer (the Netherlands), carried out by means of three variants of the think-aloud method (concurrent/retrospective think-aloud protocols and constructive interaction). These three methods had proved successful in a previous evaluation of a different municipal website, yet we decided to replicate our study in order to investigate whether the three methods would reveal different results when applied to another municipal website with a different information architecture. The results of our study showed that, as in the previous municipal website evaluation, the three evaluation methods were largely comparable in terms of output. Nevertheless, we did find a number of differences between the present and previous municipal website evaluation regarding the workings of the three methods-differences that could be explained by the different information architectures of the municipal websites tested. This suggests that the three evaluation methods might indeed work differently depending on the nature of the website that is being evaluated, and calls for more research into the effect of task type on the validity of evaluation methods. © 2008 Elsevier Inc. All rights reserved.
    Original languageDutch
    Pages (from-to)193-202
    Number of pages10
    JournalGovernment Information Quarterly
    Volume26
    Issue number1
    DOIs
    Publication statusPublished - 2009

    Cite this

    @article{d6592f7202de458cbbf7e12e158cc600,
    title = "Evaluating municipal websites: A methodological comparison of three think-aloud variants",
    abstract = "Usability methods have received relatively little methodological attention within the field of E-Government. This paper aims to address this gap by reporting on a usability test of the municipal website of Deventer (the Netherlands), carried out by means of three variants of the think-aloud method (concurrent/retrospective think-aloud protocols and constructive interaction). These three methods had proved successful in a previous evaluation of a different municipal website, yet we decided to replicate our study in order to investigate whether the three methods would reveal different results when applied to another municipal website with a different information architecture. The results of our study showed that, as in the previous municipal website evaluation, the three evaluation methods were largely comparable in terms of output. Nevertheless, we did find a number of differences between the present and previous municipal website evaluation regarding the workings of the three methods-differences that could be explained by the different information architectures of the municipal websites tested. This suggests that the three evaluation methods might indeed work differently depending on the nature of the website that is being evaluated, and calls for more research into the effect of task type on the validity of evaluation methods. {\circledC} 2008 Elsevier Inc. All rights reserved.",
    author = "{van den Haak}, M.J. and {de Jong}, M.D.T. and P.J. Schellens",
    year = "2009",
    doi = "10.1016/j.giq.2007.11.003",
    language = "Dutch",
    volume = "26",
    pages = "193--202",
    journal = "Government Information Quarterly",
    issn = "0740-624X",
    publisher = "Elsevier Limited",
    number = "1",

    }

    Evaluating municipal websites: A methodological comparison of three think-aloud variants. / van den Haak, M.J.; de Jong, M.D.T.; Schellens, P.J.

    In: Government Information Quarterly, Vol. 26, No. 1, 2009, p. 193-202.

    Research output: Contribution to JournalArticleAcademicpeer-review

    TY - JOUR

    T1 - Evaluating municipal websites: A methodological comparison of three think-aloud variants

    AU - van den Haak, M.J.

    AU - de Jong, M.D.T.

    AU - Schellens, P.J.

    PY - 2009

    Y1 - 2009

    N2 - Usability methods have received relatively little methodological attention within the field of E-Government. This paper aims to address this gap by reporting on a usability test of the municipal website of Deventer (the Netherlands), carried out by means of three variants of the think-aloud method (concurrent/retrospective think-aloud protocols and constructive interaction). These three methods had proved successful in a previous evaluation of a different municipal website, yet we decided to replicate our study in order to investigate whether the three methods would reveal different results when applied to another municipal website with a different information architecture. The results of our study showed that, as in the previous municipal website evaluation, the three evaluation methods were largely comparable in terms of output. Nevertheless, we did find a number of differences between the present and previous municipal website evaluation regarding the workings of the three methods-differences that could be explained by the different information architectures of the municipal websites tested. This suggests that the three evaluation methods might indeed work differently depending on the nature of the website that is being evaluated, and calls for more research into the effect of task type on the validity of evaluation methods. © 2008 Elsevier Inc. All rights reserved.

    AB - Usability methods have received relatively little methodological attention within the field of E-Government. This paper aims to address this gap by reporting on a usability test of the municipal website of Deventer (the Netherlands), carried out by means of three variants of the think-aloud method (concurrent/retrospective think-aloud protocols and constructive interaction). These three methods had proved successful in a previous evaluation of a different municipal website, yet we decided to replicate our study in order to investigate whether the three methods would reveal different results when applied to another municipal website with a different information architecture. The results of our study showed that, as in the previous municipal website evaluation, the three evaluation methods were largely comparable in terms of output. Nevertheless, we did find a number of differences between the present and previous municipal website evaluation regarding the workings of the three methods-differences that could be explained by the different information architectures of the municipal websites tested. This suggests that the three evaluation methods might indeed work differently depending on the nature of the website that is being evaluated, and calls for more research into the effect of task type on the validity of evaluation methods. © 2008 Elsevier Inc. All rights reserved.

    U2 - 10.1016/j.giq.2007.11.003

    DO - 10.1016/j.giq.2007.11.003

    M3 - Article

    VL - 26

    SP - 193

    EP - 202

    JO - Government Information Quarterly

    JF - Government Information Quarterly

    SN - 0740-624X

    IS - 1

    ER -