Improved emotion recognition in speech by autistics: design, validation, and implementation of acoustically modulated prosodies

Abstract

The prosodic structure of speech carries emotional information within uncertain acoustic environments where speaker, lexical, phonological, and affective dependencies for perception may exist. Socio-emotional disturbances are among the diagnostic criteria for the autism spectrum disorder, but autistics have been characterized with either impaired or typical emotional prosodies recognition, both at behavioral and neuronal scales. From a Bayesian framework we hypothesize that the oversampling of sensory evidence relative to the confidence ascribed towards internal representations would impair perception within highly variable environments. However, more reliable hierarchical structures of spectral and temporal cues would foster emotion recognition by autistics. Here, pitch, formants’ frequency, lexical stress, harmonics’ intensity, and speech rate have been edited from human voices to create stable acoustic environments. Electroencephalographic data have emphasized impaired emotion recognition while listening to human voices, that was improved when the acoustic volatility and variability of voices was reduced. Observations have been extended to the visual domain by behavioral measurements on a concomitant non-social task consistent with the over-precision ascribed to the environmental variability (sensory processing) that impaired the performance of autistics. To ensure the cultural validity of prosodies, a database adapted to the Mexican shaping of emotional expressions has been firstly developed. The recognition and induction of emotions by human and acoustically modulated voices have been validated by supervised learning and both neuronal and behavioral markers of human perception. This work offers insights into the definition of emotions by acoustic features, the neuronal processing of affective prosodies, its modulation by the acoustic variability of inputs, and the potential of certain environments to foster emotion recognition by autistics.