Analysis of Cinematic VR Directing and the Use of Computational Saliency to Assist Storytelling Ability

Abstract

This thesis is concerned with directing practices in VR Film and the application of saliency models to VR film to investigate whether they could inform creators if the viewer is likely to follow the story as they designed. Firstly the state of the art of current film making practices in Virtual Reality (VR), namely the construction and production of Ireland’s first live action VR film ’Faoladh’ is investigated. This allows us to form an insight into how film makers are approaching the field. The ’Director’s Cut Database’ is also introduced, which offers a range of VR films in a number of styles. The ’direc- tor’s cut’ is the viewport that the director of the VR films wished for the viewers to follow, and the metrics used to measure the viewers distance from this viewport are also detailed. Following this we investigate the history of narrative in order to investigate the reasons for storytelling and the aspirations of storytellers. The techniques and devices used by the directors of the films in the ’Director’s Cut Database’ are then investigated and the effects that these have on viewers, as in which devices were ef- fective in keeping the attention of the viewers to the areas that the directors wished for them to observe. In order to investigate whether computational saliency models could assist in providing informa- tion to the directors as to areas of competing visual interest we use two models, SalNet360 and AVS360, on the VR films in the database against the ground truth of where the viewers attention was. In the interest of being able to relate this information to directors we introduction a new metric VPSR (viewport saliency ratio), that hopes to provide an idea to the director as to how well their chosen viewport performs in the context of the entire omnidirectional film frame. Finally this thesis is concluded with how VR filmmaking is evolving and how new film making practices could have an effect on VR storytelling.