Acting within the environment involves a complex interaction between visual and nonvisual information resulting from bodily and internal states. Since actions, multisensory processing and internal states are tightly linked I studied them in a closed action/perception loop, focusing on the understanding of how sensing the body internal/external states allows humans to aid the interpretation of visual stimulation to achieve a stable and coherent representation of the world which in turn influence motor and perceptual judgments. To improve measures of perception-action I develop and test computational models and integrate new technology. This is all possible at my Laboratories (the Active vision, Eye Movement and VR Lab) where a variety of methods are used including kinematic-analysis, VR, reaction time, psychophysics, with the aid of a sophisticated rendering and Eye and motion tracking system. Our activities are organized in 3 main lines of research:

Active vision: Insights on the perception of a stable world in VR

One of the real mystery of visual perception resides in the sensation of visual stability that 3D objects convey when viewed during body movements. When an observer moves, the retinal projections of 3D objects in the world change accordingly but we tend to perceive the outside world as almost stable. This investigation is aimed at understanding how the sensing of body movements allows human observers to aid the interpretation of visual stimulation in order to achieve visual stability, to reduce motion sikness and to improve the detection of environmental features.

Part of this program utilize our Active Vision Apparatus, which by means of an OPTOTRAK Certus allows for the precise localization of infrared markers that are rigidly attached to the observers’ head. This allows to precisely track the 3D position of the observers’ head. This information is transmitted in real time to PC that generates the images of 3D objects on a Hi-Res computer display. Immersive VR in single and shared modalities are instead tested by a stereo projector driven by a standard PC equipped with a Core i7 6800K CPU, 16 GB RAM and an nVidia GeForce GTX 1080 graphics card that, for multi-user purpose, can be networked with other PCs. Each PC, through our custom-made modules can be interfaced with different hardware devices to enable deep immersive experience, joined-presence and interactivity, such as the Oculus Rift head-mounted display, the Oculus Touch controllers, the Leap Motion sensor, and 3Dconnexion devices.

Visual completion & non-visual information

Biological systems recover the structure of the world by (1) picking-up projective invariants that specify 3D properties of distal objects which are partially specified in the image (Fantoni 2008), and (2) using interpolation/approximation processes to overcome spatial and temporal fragmentation due to occlusion, both at the level of contours and at the level of surfaces (Fantoni, Gerbino et al., 2003; Fantoni, Gerbino et al., 2008). Although the perception of wholes from visual fragments has traditionally been considered only as the results of the processing of sensory data, growing evidences suggest it could be a component of the action–perception cycle and that it might depends on expressive qualities and multisensory integration from the body signals or from non visual senses like acoustic. In this project we measured how visual completion depends on these form of sensory integration

Action for perception: Insights on how goal directed actions impact perception and vice-versa

Classic research on prehensile movements focused on the role of vision for the control of fundamental motor action that humans perform with great dexterity, such as reaching and grasping (Goodale, Milner, Jakobson, & Carey, 1991). On the other hand, 3D vision has traditionally been considered as an optimal cue integration process of sensory data leading into an almost metric 3D representation of the environment.

Some recent works from my Lab are challenging these views as showing that hand proprioception induced by congruent/incongruent visuomotor adaptation might alter basic perceived features of objects, such as shape, position, and size as well as tactile sensitivity (Volcic et al., 2013). Furthermore we have recently discovered that the perceived quality of our global experience of the environment can be modulated by the valence of the action. This result provide a first evidence that expressive qualities of the social environment can be altered by subjective feelings associated to motor actions (Fantoni & Gerbino, 2014). Vice-versa we have recently shown that contextual factors linked to social norms systematically affects the way hand are directed towards objects (Fantoni et al., 2016).

This program thus seeks to broaden both leading views on vision and motor control by providing and testing evidences showing that: (1) goal directed actions are not only visually driven but also emotionally and socially driven; (2) visual perception is not only stimulus dependent but also action as well as emotion dependent. The way in which different kinds of perceptual alterations (both visual and emotional) are induced by comfort/discomfort will be further tested through a paradigm inspired to our novel Motor Action Mood Induction Procedure - MAMIP - (Fantoni & Gerbino, in press). This is necessary for the ultimate understanding of how the different sources of mood alterations like sound, light, and vibrations might penetrate perception (Lorenzino, Rigutti, Bovenzi, Fantoni, Bregant, in press).