People prioritize those aspects of the visual environment that match their attentional set. In the present study, we investigated whether switching from one attentional set to another is associated with a cost. We asked observers to sequentially saccade toward two color-defined targets, one on the left side of the display, the other on the right, each among a set of heterogeneously colored distractors. The targets were of the same color (no attentional set switch required) or of different colors (switch of attentional sets necessary), with each color consistently tied to a side, to allow observers to maximally prepare for the switch. We found that saccades were less accurate and slower in the switch condition than in the no-switch condition. Furthermore, whenever one of the distractors had the color associated with the other attentional set, a substantial proportion of saccades did not end on the target, but on this distractor. A time course analysis revealed that this distractor preference turned into a target preference after about 250-300 ms, suggesting that this is the time required to switch attentional sets. © 2011 The Author(s).