This study presents the results of a meta-analysis of the peer reviewed literature on water harvesting technologies, with a focus on the crop yield impacts of water harvesting in semi-arid Africa and Asia. Main aim of the analysis is to assess whether water harvesting significantly improves crop yields, and whether the type of water harvesting technology and the quality of the rainy season correlate with the change in yield. We find that water harvesting improves crop yields significantly, and that the relative impact of water harvesting on crop yields is largest in low rainfall years. Smallholder farmers may still be reluctant to invest in water harvesting, however, as in regions with low agricultural productivity the returns to investment are limited. Finally, our review of the literature suggests that there is only a limited number of studies that has systematically evaluated the crop yield impacts of water harvesting technologies. More work is needed to strengthen the scientific knowledge base.