Multisensory integration in hemianopia and unilateral spatial neglect: Evidence from the sound induced flash illusion

Abstract

Recent neuropsychological evidence suggests that acquired brain lesions can, in some instances, abolish the ability to integrate inputs from different sensory modalities, disrupting multisensory perception. We explored the ability to perceive multisensory events, in particular the integrity of audio-visual processing in the temporal domain, in brain-damaged patients with visual field defects (VFD), or with unilateral spatial neglect (USN), by assessing their sensitivity to the 'Sound-Induced Flash Illusion' (SIFI). The study yielded two key findings. Firstly, the 'fission' illusion (namely, seeing multiple flashes when a single flash is paired with multiple sounds) is reduced in both left- and right-brain-damaged patients with VFD, but not in right-brain-damaged patients with left USN. The disruption of the fission illusion is proportional to the extent of the occipital damage. Secondly, a reliable 'fusion' illusion (namely, seeing less flashes when a single sound is paired with multiple flashes) is evoked in USN patients, but neither in VFD patients nor in healthy participants. A control experiment showed that the fusion, but not the fission, illusion is lost in older participants (>50 year-old), as compared with younger healthy participants (<30 year-old). This evidence indicates that the fission and fusion illusions are dissociable multisensory phenomena, altered differently by impairments of visual perception (i.e. VFD) and spatial attention (i.e. USN). The occipital cortex represents a key cortical site for binding auditory and visual stimuli in the SIFI, while damage to right-hemisphere areas mediating spatial attention and awareness does not prevent the integration of audio-visual inputs in the temporal domain.