Dynamic point clouds and meshes are used in a wide variety of applications such as gaming, visualization, medicine, and more recently AR/VR/MR. This paper presents two extensions of MPEG-I Video-based Point Cloud Compression (V-PCC) standard to support mesh coding. The extensions are based on Edgebreaker and TFAN mesh connectivity coding algorithms implemented in the Google Draco software and the MPEG SC3DMC software for mesh coding, respectively. Lossless results for the proposed frameworks on top of version 8.0 of the MPEG-I V-PCC test model (TMC2) are presented and compared with Draco for dense meshes.

The inordinate power consumption of contemporary display panels accelerates battery depletion in mobile devices. Existing display-power reduction approaches are suboptimal because they incur additional computations and latency on the mobile device or because they assume a server-client model in which a single entity controls the server and the client. To overcome these limitations, we extract standardized metadata at a server and transmit it to a client along with the video bitstream. This metadata includes quality indicators, associated power controls, and contrast-enhancement information. The metadata guarantees that specified quality levels are achieved while avoiding flicker and minimizing power consumption. Furthermore, the metadata has minimal computational, storage & power overheads and, because it is standardized, the metadata benefits servers and clients in open ecosystems wherein all entities function independently. Our research has been implemented on mobile devices and provides, on average, 32.5 percent power reduction at high quality and up to 84 percent power reduction at acceptable quality. The ISO/IEC Moving Pictures Expert Group has recently standardized our proposed metadata in the Green Metadata Standard.

This paper introduces a view-dependent method for rendering high-dynamic-range (HDR) video in virtual reality (VR) on VR displays such as head mounted displays (HMD), mobile phones, TVs, and computer monitors. The user's view direction is taken into account to design a tone-mapping operator which appropriately displays the HDR content on the display device. The proposed method can be utilized if HDR capturing and playback (i.e. HDR camera and 10-bit video codec) are available. However, it can also be used on the 8-bit pipeline (i.e. 8-bit camera and 8-bit video codec). A VR+HDR prototype was implemented using Samsung Gear360 camera and SamsungVR Android app on Samsung Galaxy Note 5 smartphone. The subjective comparison between the proposed method and rendering of 360-degree VR video through global tone mapping was performed using this prototype and showed that for all test sequences, the proposed technique significantly improves the video quality in VR rendering.

Super resolution (SR) for real-life video sequences is a challenging problem due to complex nature of the motion fields. In this paper, a novel blind SR method is proposed to improve the spatial resolution of video sequences, while the overall point spread function of the imaging system, motion fields, and noise statistics are unknown. To estimate the blur(s), first, a nonuniform interpolation SR method is utilized to upsample the frames, and then, the blur(s) is(are) estimated through a multi-scale process. The blur estimation process is initially performed on a few emphasized edges and gradually on more edges as the iterations continue. Also for faster convergence, the blur is estimated in the filter domain rather than the pixel domain. The high-resolution frames are estimated using a cost function that has the fidelity and regularization terms of type Huber-Markov random field to preserve edges and fine details. The fidelity term is adaptively weighted at each iteration using a masking operation to suppress artifacts due to inaccurate motions. Very promising results are obtained for real-life videos containing detailed structures, complex motions, fast-moving objects, deformable regions, or severe brightness changes. The proposed method outperforms the state of the art in all performed experiments through both subjective and objective evaluations. The results are available online at http://lyle.smu.edu/~rajand/Video_SR/.

This paper presents, for the first time, a unified blind method for multi-image super-resolution (MISR or SR), single-image blur deconvolution (SIBD), and multi-image blur deconvolution (MIBD) of low-resolution (LR) images degraded by linear space-invariant (LSI) blur, aliasing, and additive white Gaussian noise (AWGN). The proposed approach is based on alternating minimization (AM) of a new cost function with respect to the unknown high-resolution (HR) image and blurs. The regularization term for the HR image is based upon the Huber-Markov random field (HMRF) model, which is a type of variational integral that exploits the piecewise smooth nature of the HR image. The blur estimation process is supported by an edge-emphasizing smoothing operation, which improves the quality of blur estimates by enhancing strong soft edges toward step edges, while filtering out weak structures. The parameters are updated gradually so that the number of salient edges used for blur estimation increases at each iteration. For better performance, the blur estimation is done in the filter domain rather than the pixel domain, i.e., using the gradients of the LR and HR images. The regularization term for the blur is Gaussian (L2 norm), which allows for fast noniterative optimization in the frequency domain. We accelerate the processing time of SR reconstruction by separating the upsampling and registration processes from the optimization procedure. Simulation results on both synthetic and real-life images (from a novel computational imager) confirm the robustness and effectiveness of the proposed method.

We present experimental results of digital super resolution (DSR) techniques on low resolution data collected using PANOPTES, a multi-aperture miniature folded imaging architecture. The flat form factor of PANOPTES architecture results in an optical system that is heavily blurred with space variant PSF which makes super resolution challenging. We also introduce a new DSR method called SRUM (Super-Resolution with Unsharpenning Mask) which can efficiently highlight edges by embedding an unsharpenning mask to the cost function. This has much better effect than just applying the mask after all iterations as a post-processing step.