Integration of semantically unrelated and semantically contingent object features reveals cortical hierarchy in human audio-visual object recognition

Grit Hein, Brain Imaging Center, Cognitive Neurology Unit

Abstract
Object recognition often requires the neuronal integration of auditory and visual features. In many cases, for example human machine interfaces, these object features are abstract and semantically unrelated. Using functional magnetic resonance imaging we investigated audio-visual integration of abstract, semantically unrelated object features. Neuronal integration sites for semantically unrelated abstract object images and sounds were compared to integration of animal sounds and images linked by varying semantic contingencies. Our results revealed a cortical hierarchy in audio-visual object recognition. Integration in precentral sulcus, the cortically highest integration level, was based on temporal and spatial contingencies, independently of semantic contingency. Posterior superior temporal sulcus (pSTS) preferred semantically contingent material. Semantically strongly related pairs were preferably integrated in dorsal portions of pSTS, object pairs with weaker semantic relationship in ventral pSTS parts. Auditory cortex, the cortically lowest integration level, was highest specialized, i.e., preferably integrated semantically strongly related object sounds and images.

Not available

Back to Abstract