Developing an efficient quality control method for streaming a 3D space expressed by point clouds is mandatory. Based on this motivation, we develop a Level of Detail (LOD)-based quality optimization method for 3D space point cloud streaming. The proposed method adopts a 3D tile-based method and considers a distance between a user's viewpoint and a 3D tile (i.e., LOD) to maximize the user's perceptual quality. In the experiment, we capture an actual 3D space of our laboratory room using a LiDAR camera and conduct performance evaluations on the Unity platform. The results conclude that the proposed method achieves the highest perceptual quality.

3D human motion prediction is the task of forecasting future motions from past observations. Previous work predicts diverse motions to cover uncertain future possibilities over a longer time. However, since most of them do not consider the surrounding context and use only the target person’s motion as the past observation, they generate overly diverse predictions rather than actual possibilities. Therefore, we propose a model that narrows down the possibilities by considering the context, especially interactions with nearby objects. Our proposed model predicts motions based on latent codes diversely sampled from a context-aware latent space. The experimental results show that our model improves accuracy and generates diverse predictions by focusing on possibilities that are consistent with object locations.

We discuss the problem of 3D pose estimation for multi-view videos. With previous frame-by-frame multi-view methods, it has been difficult to achieve stable estimation under challenging settings such as low-resolution or with only a few views. Temporal approaches are effective ways of addressing such problems, but enforcing temporal consistency with neigh-boring frames sometimes damages the precision of the results. We propose a temporal approach with selective corrections based on the observation that errors in the frame-by-frame approach are concentrated under certain adverse conditions. Our method evaluates the confidence of the frame-by-frame results and compensates for the inaccurate keypoints with temporal information while retaining the accurate keypoints. In our experiments on the CMU Panoptic dataset customized for low-resolution and a few views, we reported 32.98 mm for MPJPE and 98.64% for 3D-PCK@150. Compared to the state-of-the-art method, our method improved MPJPE by 1.14 mm and corrected 16 % of incorrect keypoints.

We tackle the problem of refining and denoising a series of 3D human poses estimated from a low-resolution video. Low-resolution often causes the wrong pose estimation, e.g., left-right switching and the absence of keypoints. We propose to use the variational autoencoder (VAE) to remove these challenging noises. The VAE model utilizes time-series information and motion priors in denoising. From our experiments, the VAE model can reduce the pose estimation error (MPJPE) for poor-quality images by 24.37mm, from the original 105.53mm. This improves about 6.5 times over the traditional DCT approach. In addition, it removes jitters and generates smooth movements, which is helpful in recognition of human behaviors.