The Earth's climate response to the rapid vegetation collapse at the termination of the African Humid Period (AHP) (5.5-5.0 kyr BP) is still lacking a comprehensive investigation. Here we discuss the sensitivity of mid-Holocene Arctic climate to changes in albedo brought by a rapid desertification of the Sahara. By comparing a network of surface temperature reconstructions with output from a coupled global climate model, we find that, through a system of land-atmosphere feedbacks, the end of the AHP reduced the atmospheric and oceanic poleward heat transport from tropical to high northern latitudes. This entails a general weakening of the mid-latitude Westerlies, which results in a shift towards cooling over the Arctic and North Atlantic regions, and a change from positive to negative Arctic Oscillation-like conditions. This mechanism would explain the sign of rapid hydro-climatic perturbations recorded in several reconstructions from high northern latitudes at 5.5-5.0 kyr BP, suggesting that these regions are sensitive to changes in Saharan land cover during the present interglacial. This is central in the debate surrounding Arctic climate amplification and future projections for subtropical precipitation changes.