The effect of temporal asynchrony on the multisensory integration of letters and speech sounds

Nienke M van Atteveldt, Elia Formisano, Leo Blomert, Rainer Goebel

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

Temporal proximity is a critical determinant for cross-modal integration by multisensory neurons. Information content may serve as an additional binding factor for more complex or less natural multisensory information. Letters and speech sounds, which form the basis of literacy acquisition, are not naturally related but associated through explicit learning. We investigated the relative importance of temporal proximity and information content on the integration of letters and speech sounds by manipulating both factors within the same functional magnetic resonance imaging (fMRI) design. The results reveal significant interactions between temporal proximity and content congruency in anterior and posterior auditory association cortex, indicating that temporal synchrony is critical for the integration of letters and speech sounds. The temporal profiles for multisensory integration in the auditory association cortex resemble those demonstrated for single multisensory neurons in different brain structures and animal species. This similarity suggests that basic neural integration rules apply to the binding of multisensory information that is not naturally related but overlearned during literacy acquisition. Furthermore, the present study shows the suitability of fMRI to study temporal aspects of multisensory neural processing.

Original languageEnglish
Pages (from-to)962-74
Number of pages13
JournalCerebral Cortex
Volume17
Issue number4
DOIs
Publication statusPublished - Apr 2007

Keywords

  • Acoustic Stimulation
  • Adult
  • Auditory Cortex
  • Auditory Perception
  • Evoked Potentials, Auditory
  • Evoked Potentials, Visual
  • Female
  • Humans
  • Male
  • Photic Stimulation
  • Speech Perception
  • Time Factors
  • Visual Cortex
  • Journal Article
  • Research Support, Non-U.S. Gov't

Fingerprint Dive into the research topics of 'The effect of temporal asynchrony on the multisensory integration of letters and speech sounds'. Together they form a unique fingerprint.

  • Cite this