Defining useful metrics to measure the goals of a software organisation is difficult. Defining useful metrics to measure the causes of the (failure) to fulfil those organisational goals is even more difficult, as the diversity of potential causes makes their measurement illusive. In this article, we describe a method to select useful software metrics based on findings from qualitative research. In a case study, we apply this method to a previously conducted study of project post-mortem reviews to assess the validity of our prior claims. For this we collected data on 109 new software projects in the organisation in which we conducted the previous case study. © 2007 Elsevier Inc. All rights reserved.