Bayesian inference underlies human auditory-visual perception: a unifying ac-count of sensory integration-segregation
Poster Presentation

Ladan Shams
UCLA Department of Psychology

Wei Ji Ma
Brain and Cognitive Science, University of Rochester

     Abstract ID Number: 73
     Full text: Not available
     Last modified: March 18, 2005

Abstract
Temporally coincident signals in the different sensory modalities do not always originate from the same source, and thus, should not—and do not—always get integrated. However, previous models of cross-modal interactions have exclusively focused on conditions in which the signals of the different modalities do get fused, and are unable to account for conditions in which the signals do not get integrated. We developed an ideal observer model which uses Bayesian inference to make inference about the causes of the various sensory signals. We tested the model in a paradigm in which subjects were asked to report the number of brief flashes and beeps. The human observers’ auditory-visual perception was surprisingly consistent with the ideal observer, indicating that the rule used by the nervous system for when and how to combine auditory and visual signals is statistically optimal. These results provide the first unifying account for the entire spectrum of cue combination, ranging from no integration, to partial interactions, to complete fusion. Our findings also show that the sound-induced flash illusion (in which a single flash is perceived as multiple when accompanied by multiple sounds) is an epiphenomenon of this general, statistically optimal strategy, as opposed to a processing error.