Recent research has shown that reward influences visual perception and cognition in a way that is distinct from the well-documented goal-directed mechanisms. In the current study we explored how task-irrelevant stimulus-reward associations affect processing of stimuli when attention is constrained and reward no longer delivered. During a training phase, participants performed a choice game, exposing them to different reward schedules for different semantic categories of natural scene pictures. In a separate test session in which no additional reward was provided, the differentially rewarded scene categories were used as task-irrelevant non-targets in a rapid serial visual presentation (RSVP) task. Participants performed a detection task on a previously non-rewarded target category. Results show that participants' sensitivity index (d′) for target detection decreased as a function of whether the reward was coupled to the distractor category during training. The semantic category of natural scenes associated with high reward caused more interference in target detection than the semantic category associated with a low reward. The same was found when new, unseen distractor pictures of the same semantic categories were used. The present findings suggest that reward can be selectively associated with high level scene semantics. We argue that learned stimulus-reward associations persistently bias perceptual processing independently of spatial shifts in attention and the immediate prospect of reward.