Development of an automated tool to score the Headache Screening Questionnaire: Agreement between automated and manual scoring

Background: Migraines and tension-type headaches (TTH) are primary headaches that can be screened with the Headache Screening Questionnaire (HSQ). However, the HSQ scoring algorithms rely on manual calculation, which is laborious and carries a risk of human error. Objective: To develop an automated tool to calculate the output of the HSQ scoring algorithm and to determine the agreement between the automated and manual calculation. Design: A cross-sectional design was used. Methods: The automated tool was developed as a Microsoft Excel spreadsheet that was tested with all possible answers for the HSQ. An experienced researcher had access to answers to the HSQ from 163 people with headaches and manually applied the migraine and TTH algorithms to obtain the final scores and classifications. After that, the same answers were uploaded into the spreadsheet and scored by the automated algorithm. The agreement between manual and automated scoring was calculated for the total score using Intraclass Correlation Coefficients (ICC_2,1), Standard Error of Measurement (SEM), and Limits of Agreement. The agreement between the classification obtained by the automated tool and the classification obtained by manual calculation for migraine and TTH was calculated using weighted Kappas (k-values). Results: The total score showed excellent agreement for migraine (ICC = 0.97, 95% CI = 0.96–0.98, SEM = 0.36) and good agreement for TTH (ICC = 0.87, 95% CI = 0.82–0.90, SEM = 0.55). The classification demonstrated excellent agreement for migraine (k-value = 0.93, 95% CI = 0.89–0.97) and for TTH (k-value = 0.78, 95% CI = 0.70–0.86). Conclusion: Implementation of the automated tool in clinical practice is suggested when using the HSQ to screen patients with primary headaches.