The present field study compared pretest and posttest face validity perceptions, predictive validity perceptions, and fairness perceptions regarding a paper-and-pencil version and a computerized version of an in-basket exercise. Furthermore, the nature of these reactions and their relationship with test performance were examined. Data were collected among 205 applicants. Contrary to our expectations, minimal differences were found between applicant reactions to the computerized version and the paper-and-pencil version of the in-basket exercise. Results from structural equation modeling showed that applicants' general beliefs in tests affected pretest reactions. Applicants' test performance influenced posttest reactions via self-assessed test performance. Theoretical and practical implications of these results are discussed. © 2012 Taylor and Francis Group, LLC.