As video data is increasingly consumed by machines rather than solely by humans, there is a growing demand for new compression methods that efficiently accommodate this shift. Since object information extracted from videos is crucial for machine consumption, analyzing the similarity between adjacent pictures based on tracked object information can help identify motion-based redundancy which can then be utilized for video compression. In this paper, we perform object tracking on input video and analyze the similarity between adjacent pictures based on the movement of tracked objects. After classifying the pictures based on their similarity, highly redundant pictures within each group are aggressively resampled in the temporal domain to improve compression efficiency while maintaining machine performance. We propose a novel picture grouping method to cluster similar adjacent pictures and describe the process of similarity assessment. We evaluated the compression efficiency of the proposed object tracking-based adaptive temporal resampling through performance evaluation experiments, achieving BD-mAP improvements in object detection of 1.29%, 0.47%, and 2.44% for Random Access (RA), Low-Delay (LD), and All Intra (AI) modes, respectively, and achieving BD-MOTA improvements in object tracking of 0.02%, 0.61%, and 6.86% for RA, LD, and AI modes, respectively.

Traditional multi-view stereo (MVS) methods rely heavily on photometric and geometric consistency constraints, but newer machine learning-based MVS methods check geometric consistency across multiple source views only as a post-processing step. In this paper, we present a novel approach that explicitly encourages geometric consistency of reference view depth maps across multiple source views at different scales during learning (see Fig. 1). We find that adding this geometric consistency loss significantly accelerates learning by explicitly penalizing geometrically inconsistent pixels, reducing the training iteration requirements to nearly half that of other MVS methods. Our extensive experiments show that our approach achieves a new state-of-the-art on the DTU and BlendedMVS datasets, and competitive results on the Tanks and Temples benchmark. To the best of our knowledge, GC-MVSNet is the first attempt to enforce multi-view, multi-scale geometric consistency during learning.

Estimating depth from a monocular image is an ill-posed problem: when the camera projects a 3D scene onto a 2D plane, depth information is inherently and permanently lost. Nevertheless, recent work has shown impressive results in estimating 3D structure from 2D images using deep learning. In this paper, we put on an introspective hat and analyze state-of-the-art monocular depth estimation models in indoor scenes to understand these models’ limitations and error patterns. To address errors in depth estimation, we introduce a novel Depth Error Detection Network (DEDN) that spatially identifies erroneous depth predictions in the monocular depth estimation models. By experimenting with multiple state-of-the-art monocular indoor depth estimation models on multiple datasets, we show that our proposed depth error detection network can identify a significant number of errors in the predicted depth maps. Our module is flexible and can be readily plugged into any monocular depth prediction network to help diagnose its results. Additionally, we propose a simple yet effective Depth Error Correction Network (DECN) that iteratively corrects errors based on our initial error diagnosis.

To provide very high quality and/or high resolution video content under limited bandwidth conditions for transmission or storage, the high efficiency video coding (HEVC) has been recently finalised by the joint collaborative team on video coding. In this study, the authors propose a fast prediction unit (PU) decision method to reduce the computational complexity of HEVC encoder. They use an early PU decision in each coding unit‐level based on spatio‐temporal analyses and depth correlations. They also consider a classification of motion activity. Experimental results show that the encoding complexity can be reduced by up to 38% on average in the random access main profile configuration with only a small bit‐rate increment and a peak signal to noise ratio (PSNR) decrement, compared to high efficiency video coding test model (HM) 7.0 reference software.

High Efficiency Video Coding (HEVC) has brought significant improvement to video compression in terms of coding performance and subjective quality. However, it is computationally more complex than H.264/Advanced Video Coding (AVC) and, hence, adaptation of HEVC to real-time video services has not been promising. In order to reduce the computational cost, we propose a fast mode decision method that is based on the adaptive ordering of modes. Our experimental results show that the proposed method reduces the total encoding time by 47.27%-60.73% and 47.65%-61.31%, with negligible degradation of coding performance, which is a Bjøntegaard delta rate increment of 0.35%-1.04% and 0.64%-1.80% under random-access configuration and low-delay configuration, respectively, compared with HEVC test model 12.1 with fast mode off.

In this paper, we propose a fast PU decision method to reduce encoder complexity. We use an early PU decision in each CU-level based on spatio-temporal analyses and depth correlations. We also consider a classification of motion activity. Experimental results show that the encoding complexity can be reduced by up to 37.98% on average in the random access Main profile configuration with only a small bit-rate increment and a PSNR decrement, compared to high efficiency video coding test model (HM) 7.0 reference software.