Delivery of omnidirectional video using saliency prediction and optimal bitrate allocation
Citation:
Ozcinar, C., İmamoğlu, N., Wang, W., Smolic, A., Delivery of omnidirectional video using saliency prediction and optimal bitrate allocation, Signal, Image and Video Processing (2020)Download Item:
Abstract:
In this work, we propose and investigate a user-centric framework for the delivery of omnidirectional video (ODV) on VR systems by taking advantage of visual attention (saliency) models for bitrate allocation module. For this purpose, we formulate a new bitrate allocation algorithm that takes saliency map and nonlinear sphere-to-plane mapping into account for each ODV and solve the formulated problem using linear integer programming. For visual attention models, we use both image- and video-based saliency prediction results; moreover, we explore two types of attention model approaches: (i) salient object detection with transfer learning using pre-trained networks, (ii) saliency prediction with supervised networks trained on eye-fixation dataset. Experimental evaluations on saliency integration of models are discussed with interesting findings on transfer learning and supervised saliency approaches.
Sponsor
Grant Number
Science Foundation Ireland (SFI)
15/RP/2776
Author's Homepage:
http://people.tcd.ie/smolica
Author: Smolic, Aljosa
Sponsor:
Science Foundation Ireland (SFI)Type of material:
Journal ArticleCollections
Series/Report no:
Springer Signal, Image and Video Processing;Availability:
Full text availableKeywords:
360°video streaming, Attention based bit- rate allocation, Saliency maps with transfer learning and supervisionSubject (TCD):
Creative Arts Practice , Creative Technologies , Digital Engagement , Image Processing , Information technology in education , Multimedia & CreativityDOI:
10.1007/s11760-020-01769-2Metadata
Show full item recordLicences: