Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Tile-based Adaptive Streaming for Virtual Reality Video

Tile-based Adaptive Streaming for Virtual Reality Video The increasing popularity of head-mounted devices and 360 video cameras allows content providers to provide virtual reality (VR) video streaming over the Internet, using a two-dimensional representation of the immersive content combined with traditional HTTP adaptive streaming (HAS) techniques. However, since only a limited part of the video (i.e., the viewport) is watched by the user, the available bandwidth is not optimally used. Recent studies have shown the benefits of adaptive tile-based video streaming; rather than sending the whole 360 video at once, the video is cut into temporal segments and spatial tiles, each of which can be requested at a different quality level. This allows prioritization of viewable video content and thus results in an increased bandwidth utilization. Given the early stages of research, there are still a number of open challenges to unlock the full potential of adaptive tile-based VR streaming. The aim of this work is to provide an answer to several of these open research questions. Among others, we propose two tile-based rate adaptation heuristics for equirectangular VR video, which use the great-circle distance between the viewport center and the center of each of the tiles to decide upon the most appropriate quality representation. We also introduce a feedback loop in the quality decision process, which allows the client to revise prior decisions based on more recent information on the viewport location. Furthermore, we investigate the benefits of parallel TCP connections and the use of HTTP/2 as an application layer optimization. Through an extensive evaluation, we show that the proposed optimizations result in a significant improvement in terms of video quality (more than twice the time spent on the highest quality layer), compared to non-tiled HAS solutions. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Multimedia Computing Communications and Applications (TOMCCAP) Association for Computing Machinery

Tile-based Adaptive Streaming for Virtual Reality Video

Download PDF
24 pages

Loading next page...
 
/lp/association-for-computing-machinery/tile-based-adaptive-streaming-for-virtual-reality-video-3dB8wGb5Ss
Publisher
Association for Computing Machinery
Copyright
Copyright © 2019 ACM
ISSN
1551-6857
eISSN
1551-6865
DOI
10.1145/3362101
Publisher site
See Article on Publisher Site

Abstract

The increasing popularity of head-mounted devices and 360 video cameras allows content providers to provide virtual reality (VR) video streaming over the Internet, using a two-dimensional representation of the immersive content combined with traditional HTTP adaptive streaming (HAS) techniques. However, since only a limited part of the video (i.e., the viewport) is watched by the user, the available bandwidth is not optimally used. Recent studies have shown the benefits of adaptive tile-based video streaming; rather than sending the whole 360 video at once, the video is cut into temporal segments and spatial tiles, each of which can be requested at a different quality level. This allows prioritization of viewable video content and thus results in an increased bandwidth utilization. Given the early stages of research, there are still a number of open challenges to unlock the full potential of adaptive tile-based VR streaming. The aim of this work is to provide an answer to several of these open research questions. Among others, we propose two tile-based rate adaptation heuristics for equirectangular VR video, which use the great-circle distance between the viewport center and the center of each of the tiles to decide upon the most appropriate quality representation. We also introduce a feedback loop in the quality decision process, which allows the client to revise prior decisions based on more recent information on the viewport location. Furthermore, we investigate the benefits of parallel TCP connections and the use of HTTP/2 as an application layer optimization. Through an extensive evaluation, we show that the proposed optimizations result in a significant improvement in terms of video quality (more than twice the time spent on the highest quality layer), compared to non-tiled HAS solutions.

Journal

ACM Transactions on Multimedia Computing Communications and Applications (TOMCCAP) – Association for Computing Machinery

Published: Dec 16, 2019

Keywords: HTTP adaptive streaming

References

  • A survey on quality of experience of HTTP adaptive streaming
    Seufert, M.; Egger, S.; Slanina, M.; Zinner, T.; Hoßfeld, T.; Tran-Gia, P.
  • Measuring the quality of experience of HTTP video streaming
    Mok, R.; Chan, E.; Chang, R.
  • Information Technology—Dynamic Adaptive Streaming over HTTP (DASH) - Part 1: Media Presentation Description and Segment Formats
    ISO/IEC,
  • State of Mobile Networks: USA
    OpenSignal,
  • State of Mobile Networks: Belgium
    OpenSignal,
  • An HTTP/2-based adaptive streaming framework for 360-degree virtual reality videos
    Petrangeli, S.; Swaminathan, V.; Hosseini, M.; De Turck, F.
  • Optimizing adaptive tile-based virtual reality video streaming
    van der Hooft, J.; Vega, M. Torres; Petrangeli, S.; Wauters, T.; De Turck, F.
  • A framework to evaluate omnidirectional video coding schemes
    Yu, M.; Lakshman, H.; Girod, B.
  • HEVC tile-based streaming to head-mounted displays
    Skupin, R.; Sanchez, Y.; Podborski, D.; Hellge, C.; Schierl, T.
  • 360 degrees video coding using region adaptive smoothing
    Budagavi, M.; Furton, J.; Jin, G.; Saxena, A.; Wilkinson, J.; Dickerson, A.
  • Adaptive 360 VR video streaming: Divide and conquer! In Proceedings of the IEEE International Symposium on Multimedia
    Hosseini, M.; Swaminathan, V.
  • An Intro to FOVAS: Field of View Adaptive Streaming for Virtual Reality
    Pixvana, Team
  • Next-Generation Video Encoding Techniques for 360 Video and VR
    Kuzyakov, E.; Pio, D.
  • Virtual reality content streaming: Viewport-dependent projection and tile-based techniques
    Zare, A.; Aminlou, A.; Hunnuksela, M. M.
  • Temporal MCTS Coding Constraints Implementation
    JCTVC-AC0,
  • Implementing motion-constrained tile and viewport extraction for VR streaming
    Son, J.; Jang, D.; Ryu, E.
  • Adaptive 360-degree video streaming using layered video coding
    Nasrabadi, A. Taghavi; Mahzari, A.; Beshay, J. D.; Prakash, R.
  • Optimizing 360 video delivery over cellular networks
    Qian, F.; Ji, L.; Han, B.; Gopalakrishnan, V.
  • Probabilistic viewport adaptive streaming for 360-degree videos
    Xu, Z.; Zhang, X.; Zhang, K.; Guo, Z.
  • Fixation prediction for 360-degree; video streaming in head-mounted virtual reality
    Fan, C.; Lee, J.; Lo, W.; Huang, C.; Chen, K.; Hsu, C.
  • Saliency in VR: How do people explore virtual environments? IEEE Trans
    Sitzmann, V.; Serrano, A.; Pavel, A.; Agrawala, M.; Gutierrez, D.; Masia, B.; Wetzstein, G.
  • Deep 360 pilot: Learning a deep agent for piloting through 360-degree sports video
    Hu, H.; Lin, Y.; Liu, M.; Cheng, H.; Chang, Y.; Sun, M.
  • A rate adaptation algorithm for tile-vased 360-degree video streaming
    Ghosh, A.; Aggarwal, V.; Qian, F.
  • HEVC-compliant tile-based streaming of panoramic video for virtual reality applications
    Zare, A.; Aminlou, A.; Hannuksela, M. M.; Gabbouj, M.
  • Tiled-based adaptive streaming using MPEG-DASH
    Feuvre, J. Le; Concolato, C.
  • Using MPEG DASH SRD for zoomable and navigable video
    D’Acunto, L.; van den Berg, J.; Thomas, E.; Niamut, O.
  • Hypertext Transfer Protocol (HTTP/1
    Fielding, R.; Reschke, J.
  • Hypertext Transfer Protocol Version 2 (HTTP/2)
    Belshe, M.; Peon, G.; Thomson, M.
  • Usage of HTTP/2 for Websites
    W3Techs,
  • Low latency live video streaming over HTTP 2
    Wei, S.; Swaminathan, V.
  • 360ProbDASH: Improving QoE of 360 video streaming using tile-based HTTP adaptive streaming
    Xie, L.; Xu, Z.; Ban, Y.; Zhang, X.; Guo, Z.
  • Adaptive multicast streaming of virtual reality content to mobile users
    Ahmadi, H.; Eltobgy, O.; Hefeeda, M.
  • Predicting the performance of virtual reality video streaming in mobile networks
    Filho, R. I. T. da Costa; Luizelli, M. C.; Vega, M. Torres; van der Hooft, J.; Petrangeli, S.; Wauters, T.; De Turck, F.; Gaspary, L. P.
  • Which saliency weighting for omni directional image quality assessment? In Proceedings of the International Conference on Quality of Multimedia Experience
    Rai, Y.; Callet, P. Le; Guillotel, P.
  • Quality assessment for adaptive virtual reality video streaming: A probabilistic approach on the user’s gaze
    van der Hooft, J.; Vega, M. Torres; Petrangeli, S.; Wauters, T.; De Turck, F.
  • SARA: Segment-aware rate adaptation algorithm for dynamic adaptive streaming over HTTP
    Juluri, P.; Tamarapalli, V.; Medhi, D.
  • A dataset for exploring user behaviors in VR spherical video streaming
    Wu, C.; Tan, Z.; Wang, Z.; Yang, S.
  • The Multiple-Choice Knapsack Problem
    Kellerer, H.; Pferschy, U.; Pisinger, D.
  • HTTP/2-based adaptive streaming of HEVC video over 4G/LTE networks
    van der Hooft, J.; Petrangeli, S.; Wauters, T.; Huysegems, R.; Alface, P. Rondao; Bostoen, T.; De Turck, F.