Investigating the use of recurrent motion modelling for speech gesture generation
Citation:
Ylva Ferstl, Rachel McDonnell, Investigating the use of recurrent motion modelling for speech gesture generation, 18th International Conference on Intelligent Virtual Agents, Sydney, NSW, Australia, November 05 - 08, 20, 2018, 93-98Download Item:
Abstract:
The growing use of virtual humans demands generating increasingly realistic behavior for them while minimizing cost and time. Gestures are a key ingredient for realistic and engaging virtual agents and consequently automatized gesture generation has been a popular area of research. So far, good gesture generation has relied on explicit formulation of if-then rules and probabilistic modelling of annotated features. Machine learning approaches have yielded only marginal success, indicating a high complexity of the speech-to-motion learning task. In this work, we explore the use of transfer learning using previous motion modelling research to improve learning outcomes for gesture generation from speech. We use a recurrent network with an encoder-decoder structure that takes in prosodic speech features and generates a short sequence of gesture motion. We pre-train the network with a motion modelling task. We recorded a large multimodal database of conversational speech for the purpose of this work.
Sponsor
Grant Number
Science Foundation Ireland (SFI)
13/RC/2016
Science Foundation Ireland (SFI)
Author's Homepage:
http://people.tcd.ie/ramcdonnDescription:
PUBLISHEDSydney, NSW, Australia
Author: Mc Donnell, Rachel
Sponsor:
Science Foundation Ireland (SFI)Science Foundation Ireland (SFI)
Other Titles:
18th International Conference on Intelligent Virtual AgentsType of material:
Conference PaperCollections
Availability:
Full text availableKeywords:
Machine learning, Character animation, Motion synthesis, Behaviour generation, Recurrent networks, Deep learningSubject (TCD):
Creative Technologies , Computer Graphics , Computer GraphicsDOI:
http://dx.doi.org/10.1145/3267851.3267898Metadata
Show full item recordLicences: