Algorithms for Quality Optimization in Omnidirectional Video

Abstract

Omnidirectional video (ODV) is a recent imaging technology, which is currently getting increasingly popular thanks to its ability to create an immersive and interactive viewing experience. Nowadays, viewing ODV is becoming easy and affordable, thanks to relatively cheap head-mounted displays and the possibility to view ODV with widespread devices like smartphones and tablets. Moreover, ODV technology has reached a level that is mature enough to attract content producers. Even if the ODV technology has reached an adequate level of maturity, it is still developing. There are several challenges during ODV production that can introduce artifacts in ODV even with the current technology. Moreover, for the coding and transmission of ODV, the current solutions could be further developed and improved. Therefore, the objective of this thesis is to develop technologies that can help to improve the quality of ODV and be applied during ODV production and for the development of better coding and transmission methods. Three research areas characterize this thesis: artifact detection, artifact correction, and quality assessment. Regarding artifact detection, we propose a general framework that extends artifact detection methods for stereoscopic 3D (S3D) standard images to S3D omnidirectional images (ODI), which can be used by artists in the post-production workflow in order to optimize the quality. Moreover, methods for the detection of two common artifacts in S3D ODIs, namely color and sharpness mismatch, are also proposed. For the evaluation of the artifact detection framework, a new dataset of S3D ODIs with visual attention data was created. For the second research area, i.e., artifact correction, the thesis proposes three different solutions for the correction of color mismatch. One solution is based on traditional visual computing techniques, and the other two solutions are based on deep learning. The evaluation of these methods shows their effectiveness. Finally, for quality assessment, a framework that extends full-reference quality metrics for standard video to monoscopic ODV was developed. For the development and evaluation of the framework, a dataset of monoscopic ODVs with subjective quality scores and visual attention data was created. The evaluation of the framework shows that it has a better quality assessment performance than the commonly used quality metrics for monoscopic ODV.