Tailoring the past to the future: how prospective actions shape visual working memory

Many researchers and philosophers contend that human cognition is rooted in sensorimotor processes, and that therefore actions should play a key role in determining how information that is used for complex mental operations is represented in the brain. One such operation is visual working memory (VWM), the ability of storing past visual information for an imminent task. A large body of work points towards the existence of a tight link between the motor system and VWM, and recent theoretical accounts have underscored the importance of action in shaping VWM representations. However, so far, the influence of action on VWM has remained relatively unexplored. In this thesis, we addressed this research gap by systematically investigating how variations in the strength of action-VWM links influence the storage of visual information in working memory. In Chapters 2 and 3, we manipulated the strength of the VWM-action link by varying the functional coupling between actions and memorized items. We asked whether objects that are the direct target of an action plan are more strongly represented in working memory than objects that are relevant for the planning of an action but not their direct target. Using eye-tracking (Chapter 2) and EEG (Chapter 3), we assessed the action-induced strengthening of the VWM representation by estimating the interference that holding such representation in memory caused during an intermittent perceptual task. We found that the VWM representations of items that are the target of an action are strengthened compared to those of non-target items. In Chapter 4, we again manipulated the strength of the VWM-action link, this time by varying the specificity of the action to be planned. Using eye-tracking, we investigated whether VWM representations are attended more in a situation in which participants know in advance how to act on a memorized object compared to a situation in which participants intend to act on that same object, without yet knowing how. Across three experiments, we found overall evidence that that the more defined the action-VWM link, the greater the VWM-guidance of attention during the intermediate visual selection task. In Chapter 5, we examined whether action plans not only strengthen VWM representations but also alter them, by inducing adaptive memory distortions. We manipulated the strength of the VWM-action link by varying the univocity of this coupling, and we asked whether two objects that are linked to different action plans (i.e., one-to-one sensorimotor mapping) are remembered as being more dissimilar compared to two objects that are assigned to the same action (i.e., many-to-one sensorimotor mapping). By leveraging on the finding that VWM interactions can lead to both repulsive and attraction biases between memorized items, we discovered that perceptually similar objects are repelled more from each other in memory when they are assigned to different actions rather than to the same action. Our findings clearly show that action planning modulates how information is represented within VWM, both in terms of representational strength and adaptive distortions, highlighting the importance of considering action contexts in the study of VWM.