Evaluating the Influence of Vocal and Visual Realism on Subjective Experiences of Empathy for Virtual Characters

Abstract

With the expanding range of use cases for advancements in animation and voice synthesis, more opportunities arise for interactions with realistic virtual characters. As yet, it is unclear to what extent realism, either in appearance or voice, influences our reactions to emotional expressions in such characters.

This work describes the roles of vocal and visual realism in providing opportunities for empathy. These dimensions of realism are used to convey emotional expressions in faces and voices of virtual characters. In this work, physiological behaviour is observably influenced by these expressions in a manner that is consistent with the phenomenon of empathy. Moreover these dimensions of realism are seen to impact self-reports of empathy for these characters.

Over the last four years, our experiments have steadily narrowed in on aspects of virtual characters which influence perception, cognition and emotion in users. While our studies originally focused on the effect of appearance and the role of individual user traits, it gradually became clear that bimodal realism (i.e. vocal and visual realism) was the topic which required attention in this domain.

With this more holistic approach to realism in virtual characters, the question arose as to whether these variations could affect empathic responses. Since empathy, as measured in this thesis, was stimulated by the perception of emotions that varied in intensity between levels of realism, it became necessary to question whether visual or vocal realism may provide 'affordances' for emotion perception and empathy, as described in the language of ecological psychology. This question directed our research towards evaluations of of vocal-visual realism combinations that were ultimately observed to influence both physiological measurements of motor empathy, and self-reported perceptions of cognitive, affective and compassionate empathy.

What follows in this thesis is an attempt to embed these findings into recent trends in computer graphics and cognitive science research. Each research project that was conducted over the course of this doctorate is explained with reference to these fields, and their unifying domain of human-computer interaction, as well as the other fields that they invariably touch off. The findings ultimately indicate that realism and emotional expressions in virtual characters present affordance opportunities that blur the distinction between object-perception and social behaviour.