Recently, implicit function (IF)-based methods for clothed human reconstruction using a single image have received a lot of attention. Most existing methods rely on a 3D embedding branch using volume such as the skinned multi-person linear (SMPL) model, to compensate for the lack of information in a single image. Beyond the SMPL, which provides skinned parametric human 3D information, in this paper, we propose a new IF-based method, DIFu, that utilizes a projected depth prior containing textured and non-parametric human 3D information. In particular, DIFu consists of a generator, an occupancy prediction network, and a texture prediction network. The generator takes an RGB image of the human front-side as input, and hallucinates the human back-side image. After that, depth maps for front/back images are estimated and projected into 3D volume space. Finally, the occupancy prediction network extracts a pixel-aligned feature and a voxel-aligned feature through a 2D encoder and a 3D encoder, respectively, and estimates occupancy using these features. Note that voxel-aligned features are obtained from the projected depth maps, thus it can contain detailed 3D information such as hair and cloths. Also, colors of each query point are also estimated with the texture inference branch. The effectiveness of DIFu is demonstrated by comparing to recent IF-based models quantitatively and qualitatively.

This paper proposes a point cloud (PC) visual quality assessment (VQA) framework that reflects the human visual system (HVS). The proposed framework compares natural images acquired using a digital camera and PC images generated via 2D projection in terms of appropriate objective quality evaluation metrics. Humans primarily consume natural images; thus, human knowledge is typically formed from natural images. Thus, natural images can be more reliable reference data than PC data. The proposed framework performs an image alignment process based on feature matching and image warping to use the natural images as a reference which enhances the similarities of the acquired natural and corresponding PC images. The framework facilitates identifying which objective VQA metrics can be used to reflect the HVS effectively. We constructed a database of natural images and three PC image qualities, and objective and subjective VQAs were conducted. The experimental result demonstrates that the acceptable consistency among different PC qualities appears in the metrics that compare the global structural similarity of images. We found that the SSIM, MAD, and GMSD achieved remarkable Spearman rank-order correlation coefficient scores of 0.882, 0.871, and 0.930, respectively. Thus, the proposed framework can reflect the HVS by comparing the global structural similarity between PC and natural reference images.

Various studies have been conducted on instance segmentation and made great strides over the past few years. Most recently, instance-specific mask generation via dynamic kernel predictions has shown the significant performance improvement even without bounding boxes as well as anchors. However, this scheme still does not fully consider dynamic properties since the size of the receptive field is not enough to cover the spatially-meaningful range due to memory limitations. Furthermore, the single-fused feature often fails to grasp complicated boundaries for objects of different sizes. In this article, we propose the dynamic residual filtering method with the Laplacian pyramid, which separately restores the global layout and local boundaries of instance masks. Specifically, we firstly apply the Laplacian pyramid-based decomposition scheme to features encoded from the backbone and subsequently restore sub-band mask residuals from coarse to fine pyramid levels. To do this, we design spatially-aware convolution filters to progressively capture the residual form of mask features at each level of the Laplacian pyramid while holding deformable receptive fields with dynamic offset information. This is fairly desirable since global and local properties of mask features can be accurately restored with keeping the spatial flexibility through the invertible process of the Laplacian reconstruction. Experimental results on the COCO dataset demonstrate that our proposed method achieves the state-of-the-art performance, i.e., 42.7% AP. The code and model are publicly available at:https://github.com/tjqansthd/LapMask.

The paper reports a new view synthesis method for omnidirectional video with the ability to slightly displace a virtual viewpoint, i.e. the paper describes a novel synthesis method for 3DoF+ 360 video. This new method is noteworthy because of its high versatility and reliability: the method is appropriate for both perspective and omnidirectional input views, is able to render both perspective and omnidirectional views, and the produced synthetic views differ from the respective ground truth images less than with other view synthesis methods. These important features result from several innovations, e.g., prioritization of input views, efficient inpainting adapted to equirectangular projections, and efficient color correction. The experimental results demonstrate that the new method outperforms the state-of-the-art methods used hitherto.

This paper proposes an improved phase parameter analysis and synthesis method for parametric stereo audio coding. For stereo signal with low interchannel coherence (IC), conventional phase parameter analysis and synthesis methods show poor quality, since they overlook the effect of IC to the phase parameters defined. The proposed system defines new phase parameters reflecting IC information, so that we can alleviate the deterioration of audio quality due to the synthesis of excessive phase difference, especially when IC is low. Subjective listening test results are presented to show that the proposed method provides always better audio quality than the conventional methods.

An interactive audio service is a new conceptual audio service that provides the users with opportunities for a variety of experiences on the alternative and advanced audio services. In the interactive audio service, users can freely control various audio objects to make their own audio sounds. A spatial audio object coding (SAOC) is a useful technology that can support most parts of the interactive audio service with a relatively low bit-rate, but is very poor to perfect gain control of a certain audio object, i.e., the target audio object. In this paper, the SAOC with a two-step coding structure is proposed to efficiently handle the target audio object as well as the normal audio objects. A transform coded excitation (TCX) based residual coding scheme is presented in the context of the sound quality enhancement. From experimental results, it can be noted that the various audio objects can be successfully handled with respect to the bit-rate and the sound quality by using the proposed two-step coding structure SAOC.

In this paper, we propose a method of restoring principal to ambient energy ratio (PAR) at the decoder in the principal component analysis (PCA)-based parametric audio coding. The conventional approach applies the post-scaling at the decoder using the energy information extracted from the input signal at the encoder. However, this approach has a problem that the relative energy of principal source in the reconstructed signal is smaller than the original signal and also affected by the panning angle of principal source. To restore the PAR at the decoder, the proposed method estimates the post-scaling factors using parametric information extracted from PCA-based formulation. The objective and subjective results verify the performance improvement of proposed method.

Spatial audio coding scheme is advanced technology in the sense of saving bitrate without any scarifications of sound quality. In this paper, the method to utilizing the spatial cues is present. This utilization is for supporting a new useful functionality, i.e., flexible sound scene control. The function of sound scene control may result in providing user preferable audio service by interaction and be utilized in upcoming applications such as multi-view video service and virtual reality applications. It will be shown that even with a little addition of computational operations; this useful functionality will be achievable maintaining the corresponding sound quality.

Binaural cue coding (BCC) has been introduced for compact representation of multi-channel audio. It exploits binaural cue parameters for capturing the spatial image of multi-channel audio. Recently, it has been standardized within MPEG as the name of "MPEG Surround." In this paper, we propose a sound source location cue coding (SSLCC) system for compressing multi-channel audio to be suitable at the narrow bandwidth transmission environment. To improve the compression ability of the conventional BCC, the SSLCC system utilizes the virtual source location information (VSLI) as a spatial cue parameter instead of the inter-channel level difference (ICLD) of the BCC system. Also the SSLCC system adopts enhanced pre/post processing algorithms to improve perceptual sound quality. Objective and subjective assessment results show that the proposed SSLCC system reveals better performance than the conventional BCC system.

Future audio applications will need to handle a wide variety of end-devices and networks from low quality mobile services to very high quality home entertainment services. The MPEG-21 Digital Item Adaptation standard specifies a variety of tools that aim to enhance the user experience and maximize the perceived quality of service. This paper focuses on the audio-related tools, including audio usage environment descriptions and techniques for audio resource adaptation. Additionally, subjective and objective measures are discussed in the context of bit-rate adaptation. New quality measures, brightness, cleanness, and wideness are proposed for controlling the quality of the audio transmission.