Overview:
This week, I read a paper ‘Resolution-Defined Projections for Virtual Reality Video Compression’, which is published on 2017 IEEE Virtual Reality (VR) by Charles Dunn and Brian Knott.
First of all, Existing video encoders cannot encode the spherical videos directly. Traditional compression algorithms have massive adoption and hardware support on computers and mobile devices. In this respect, the video should be mapped to a 2D plane before encoding. However, popular projections used for VR video benefit from mathematical or geometric simplicity, but result in suboptimal resolution distributions.
·
So, the author propose a method for generating a projection to match a desired resolution function. This method allows for customized projections with smooth, continuous and optimal resolution functions. Compared to commonly used projections, this resolution-defined projections drastically improve compression ratios for any given quality.
Problem
1. equirectangular projection (mathematical simplicity, inaccurately magnifying polar regions)
2. cubic mapping projects (avoids the singularities, sacrifices adjacency, suboptimal resolution distribution)
3. pyramid projection (greatly improve resolution in the viewing direction, resolution undesirably)
Method
1. Mapping from the Cartesian unit sphere in (x;y;z) space to a rectangle in (u;v;w) space.
2. Using an existing rectangular projection as a seed for deriving the new projection, guaranteeing that the final projection maps the unit sphere to a rectangle (requires that the Jacobian of the seed projection J˜ and the desired resolution function be separable in (a;b;c) space).
Result
1. Total resolution.
2. Total uniformity.
3. Three weighting functions (a. impulse function, b. foveated resolution function, c. approximates the empirical viewing patterns of humans while viewing VR video content).
Advantage
This method derives a nearly optimal projection (this novel projection’s resolution function peaks in the forward viewing direction, with resolution smoothly decreasing in all directions. Again, the projection suffers from some distortion near the poles, but the author are able to deal with this distortion outside the scope of this paper to improve uniformity)
Overall, outperform existing industry standards, producing content with increased resolution and decreased distortion in important areas
I think the problem as description in this paper do exist for equirectangular, cubic mapping, pyramid projection. It is a good choice to develop new better projection method (but should consider the complexity and applicability).