The Impact of Training Data Bias on Automatic Generation of Video Captions
Citation:
Alan Smeaton, Yvette Graham, Kevin McGuinness, Noel O'Connor, Se?n Quinn, Eric Arazo Sanchez, The Impact of Training Data Bias on Automatic Generation of Video Captions, Proceedings of the 25th International Conference on MultiMedia Modeling, 25th International Conference on MultiMedia Modeling, Thessaloniki, Greece, 2019, 178 - 190Abstract:
A major issue in machine learning is availability of training data. While this historically referred to the availability of a sufficient volume of training data, recently this has shifted to the availability of sufficient unbiased training data. In this paper we focus on the effect of training data bias on an emerging multimedia application, the automatic captioning of short video clips. We use subsets of the same training data to generate different models for video captioning using the same machine learning technique and we evaluate the performances of different training data subsets using a well-known video caption benchmark, TRECVid. We train using the MSR-VTT video-caption pairs and we prune this to reduce and make the set of captions describing a video more homogeneously similar, or more diverse, or we prune randomly. We then assess the effectiveness of caption-generating trained with these variations using automatic metrics as well as direct assessment by human assessors. Our findings are preliminary and show that randomly pruning captions from the training data yields the worst performance and that pruning to make the data more homogeneous, or diverse, does improve performance slightly when compared to random. Our work points to the need for more training data, both more video clips but, more importantly, more captions for those videos.
Sponsor
Grant Number
SFI stipend
12/RC/2289
Author's Homepage:
http://people.tcd.ie/ygrahamDescription:
PUBLISHEDThessaloniki, Greece
Author: Graham, Yvette
Sponsor:
SFI stipendOther Titles:
Proceedings of the 25th International Conference on MultiMedia Modeling25th International Conference on MultiMedia Modeling
Type of material:
Conference PaperCollections
Availability:
Full text availableSubject (TCD):
Creative Technologies , Natural Language ProcessingDOI:
http://dx.doi.org/10.1007%2F978-3-030-05710-7_15Metadata
Show full item recordLicences: