RGB-D cameras are essential for real-time 3D teleconferencing applications, often requiring multiple devices to maximize visibility across different viewpoints. However, calibrating a multi-camera rig is challenging due to the lack of common features across views and the difficulty of accurately extracting depth features. In this paper, we propose a novel algorithm for precise calibration of multiple RGB-D sensors. Our approach has two key contributions. First, we introduce a composite object consisting of multiple planar patterns, allowing each RGB-D device to capture at least one plane, thus eliminating the need for all cameras to share the same features. Second, we develop a non-linear optimization technique that iteratively minimizes global reprojection error while refining depth feature extraction. This method reduces the need for capturing excessive calibration images and mitigates errors caused by poor depth feature extraction.

Most multi-camera applications such as 3D volumetric teleconferencing assumes that each camera has a common and global lighting response. This assumption is not true as each individual camera usually exhibit radically different color responses. This leads to lesser quality of experience in 3D perception. To address this problem, we propose a simple but an efficient method to calibrate and correct color responses of multi-view cameras. The proposed method does not require the use of standard color calibration target, and only requires well lighted object or human in the scene.

Volumetric video is an emerging form of media, which allows users to consume content unconstrained to a specific viewing angle or position. Visual Volumetric Video-based Coding (V3C) is a technology developed in the Moving Picture Experts Group (MPEG), which provides tools for compressing volumetric video. Prior implementations and demonstrations of V3C-based content delivery have been focusing on pre-encoded content, utilizing technologies such as DASH for on-demand media delivery. In contrast to prior work, this paper presents coding techniques and architectural choices, which enable end-to-end real-time V3C encoding, streaming, decoding, and rendering. The presented system implements the discussed methods, demonstrates conversational low latency, utilizes standardized technologies, and runs on commercially available hardware.

Easy and fast animation of 3D characters is attractive for both gaming and entertainment applications. 3D mesh must be properly rigged and skinned in order to create seamless animation. This process can be time consuming and requires deep knowledge of appropriate software based on kinematic animation. In this work, we present a fast and lightweight deep neural model to automate 3D human animation using skeletal representation from 2D image pose, i.e., joints in 2D space. We accomplish this using Transformed Mesh Decoder (TMD), which is a novel layer for convolutional neural networks. To train the network, we generate a large and diverse dataset using Skinned Multi-Person Linear (SMPL) model. Experiment shows that our method is effective when compared to both the ground truth and state-of-the art linear blend skinning that require manually painted skinning weights for accurate result. The animation process is fast and can achieve approximately 10-15fps in practice. The proposed method is simple which opens the possibility for future improvement in real-time application.

We present a demonstration of Network Based Media Processing (NBMP) standard-compliant cloud service for reconstructing and Augmented Reality (AR) display of fully textured 3D human model using 2-5 images captured with a smartphone.

Most conventional camera calibration algorithms assume that the imaging device has a Single Viewpoint (SVP). This is not necessarily true for special imaging device such as fisheye lenses. As a consequence, the intrinsic camera calibration result is not always reliable. In this paper, we propose a new formation model that tends to relax this assumption so that a Non-Single Viewpoint (NSVP) system is corrected to always maintain a SVP, by taking into account the variation of the Entrance Pupil (EP) using thin lens modeling. In addition, we present a calibration procedure for the image formation to estimate these EP parameters using non linear optimization procedure with bundle adjustment. From experiments, we are able to obtain slightly better re-projection error than traditional methods, and the camera parameters are better estimated. The proposed calibration procedure is simple and can easily be integrated to any other thin lens image formation model.

Depth from Small Motion (DfSM) is particularly interesting for smartphone devices because it makes it possible to get depth information with minimal user effort and cooperation. The state of art method requires about 30 images for the optimization to converge fast and produce accurate depth-map. As the use of high number of frames contribute to long execution time and huge memory allocation, we propose a frame selection strategy using Inertial Measurement Unit (IMU) and image based analysis. As a result, only 5 frames with appropriate viewpoint from the reference one are used for the depth-map generation. Full experiment is done on an Android platform using optimized version of the proposed method with CPU-GPU co-processing under OpenCL. We are able to provide accurate camera parameters and depth-map estimates using only these selected frames.

Multiple cameras are increasingly prevalent for autonomous driving, and the increase need to have 360 degree perception of world space has led to the combination of various cameras in the varieties of narrow-angle and wide-angle field of view. This has led to issues regarding the quality of these optics as a result of bad and cheap designs. Intrinsic calibration is indispensable for accurate perception of the environment in order for it to make accurate decisions such as camera pose estimation and 3-D reconstruction. In this work, we propose a lens distortion model that has been motivated by unintentional tilt in the optical lens system. The proposed distortion model has been added to current state of art generic method of Kannala to form an extended model. To our knowledge, this is the first time that the idea of tilt distortion has been introduced for wide-angle view and fish-eye cameras. We show improved result of 4 to 13% from experiments.

The increase need to have 360 degree perception of world environment has led to the combination of various cameras in the varieties of narrow and wide-angle field of view. Current state of art methods do not provide an automatic means for fish-eye calibration, which is indispensable in an industrial environment where many lenses are to be calibrated in a relative short time. As automatic feature extraction is the key issue for fully automatic calibration framework, we address this issue to remove any form of human intervention during the calibration process. Unlike state-of-the-art methods, the proposed framework is completely automatic and not prone to detection errors. We evaluate the proposed feature extraction using state-of-the-art generic calibration model with both real and synthetic images.

Dimensional control of metallic objects is challenging since their surfaces present strong specular properties. The 3D measurement systems based on laser-camera triangulation project the laser on to the surface of the object and do not always provide satisfying results for metallic object. In this paper, we analyze images of metallic objects that are corrupted by speckle noise as a result of their surface aspect problems. The performance reached by those systems depends on the accuracy of extraction of the coordinates of pixels that correspond to the center of the laser line projected on the object. We have developed a column-wise laser stripe extraction method that can be used for industrial application in real-time. To qualify this system, we have made various experiments on synthetic images of these objects, and compare the accuracy of the proposed stripe extraction methods.