Cross-modal Recognition of Human Body Pose: Interaction of Motor and Visual Information
Poster Presentation

Yasuyuki Inoue
Department of Knowledge-based Information Engineering, Toyohashi University of Technology

Michiteru Kitazaki
Research Center for Future Vehicle / Department of Knowledge-based Information Engineering, Toyohashi University of Technology

     Abstract ID Number: 87
     Full text: Not available
     Last modified: March 21, 2005

Abstract
Recognition of human body is a special case of object recognition, since human body is very familiar and can be encoded with motor (self-body movement) as well as visual information. We aimed to investigate the effects of visual and motor encoding of body poses on visual recognition. Half of the subjects learned 16 different poses (8 front views and 8 back views) with visual observation. The others learned the same poses with visual observation and body action in that they imitated the presented posture by own body. 4 repetitions of the learning session were followed by a visual recognition test, in which a learned pose and a novel pose were presented and the subject judged which was learned one. We varied the view difference (0-180deg) of the learned poses between the learning session and the recognition test. For visual-encoding subjects the recognition performance with front-view encoding was significantly better than with back-view encoding, but for visual-and-motor-encoding subjects there was no difference. It is suggested that the visual encoding of human body has advantage in front-view encoding while the motor encoding has advantage in back-view encoding. Thus, we can learn novel postures view-independently by using both visual and motor information.