Audiovisual Interaction and grouping in apparent motion
Audiovisual interval interaction and Ternus apparent motion
Ternus apparent motion (or Ternus illusion) was first discovered by Josef Ternus in 1926. Using two frames of multiple dots, when overlaid, share one or several common dots at the center. Depending on the spatial configuration and inter-stimulus-interval (ISI), two distinct motion percepts are usually observed: 'element motion' (outer dots flip) and 'group motion' (all dots shift together). Short ISIs usually give rise to the percept of 'element motion', while long ISIs give rise to the perception 'group motion'. A great interactive demo can be found in Michael Bach's visual illusion website: Pikler-Ternus Display demonstration.
Based on classical Ternus apparent motion paradigm, recently we (Shi, Chen & Müller, 2010) examined crossmodal temporal interactions on the visual apparent motion, and found that spatial uninformative auditory stimuli can modulate the visual motion percepts. In particular, auditory interval defined by beeps influenced the perceived visual interval (defined by visual Ternus frame). As a result, two motion percepts were changed accordingly. By measuring variabilities of auditory and visual interval, we demonstrated that the perceived audiovisual interval could be predicted by Maximum Likelihood Estimation (MLE).
Perceptual grouping and crossmodal temporal interaction
In this project, we investigate influences of relative timing of multimodal events and the event structure on uni- and crossmodal perceptual grouping using tactile two-tap apparent-motion streams. In our recent study (Chen, Shi, Müller, 2011), we showed that with full pairing audiotactile configuration, the bi-stable tactile apparent motion rivalry to be systematically resolved by audiotactile asynchrony. However, when the audiotactile asynchrony was increased such that the beeps occurred temporally proximal to the even numbered tactiel stimuli, a reversed effect on the direction of apparent motion was found. Moreover, when only half tactile events were paired with beeps, a consistently dominant direction of apparent motion was observed. The finding suggest that systematic influence of auditory timing on the motion percept in a full-pairing crossmodal condition, however, unequal odd- and even-numbered audiotactile pairs leads to an attentional modulation of crossmodal grouping.
Intra- and crossmodal grouping on visual-tactile Apparent motion
Abstract of Chen, Shi, Müller, 2010: Previous studies of dynamic crossmodal integration have revealed that the direction of apparent motion in a target modality can be influenced by a spatially incongruent motion stream in another, distractor modality. Yet, it remains to be examined whether non-motion intra- and crossmodal perceptual grouping can affect apparent motion in a given target modality. To address this question, we employed Ternus apparent-motion displays, which consist of three horizontal aligned visual (or tactile) stimuli that can alternately be seen as either ‘element motion’ or ‘group motion’. We manipulated intra- and crossmodal grouping by cueing the middle stimulus with different cue-target onset asynchronies (CTOAs). In unimodal conditions, we found Ternus apparent motion to be readily biased towards ‘element motion’ by precues with short or intermediate CTOAs in the visual modality and by precues with short CTOAs in the tactile modality. By contrast, crossmodal precues with short or intermediate CTOAs had no influence on Ternus apparent motion. However, crossmodal synchronous tactile cues led to dominant ‘group motion’ percepts. And for unimodal visual apparent motion, precues with long CTOAs shifted apparent motion towards ‘group motion’ in general. The results suggest that intra- and crossmodal interactions on visual and tactile apparent motion take place in different temporal ranges, but both are subject to attentional modulations at long CTOAs.
Chen, L., Shi, Z., Müller, H. J. (2011) Interaction of perceptual grouping and crossmodal temporal capture in tactile apparent-motion, PLoS One, 6(2): e17130. doi:10.1371/journal.pone.0017130
Shi, Z., Chen, L., Müller, H. J. (2010) Auditory temporal modulation of the visual Ternus effect: the influence of time interval, Experimental Brain Research, 203(4), 723-735.
Chen, L., Shi, Z., Müller, H. J., (2010) Intra- and cross-modal interactions in visual-tactile Ternus apparent motion, Brain Research, 1354, 152-162.