Amodal Instance Segmentation (AIS) endeavors to accurately deduce complete object shapes that are partially or fully occluded. However, the inherent ill-posed nature of single-view datasets poses challenges in determining occluded shapes. A multi-view framework may help alleviate this problem, as humans often adjust their perspective when encountering occluded objects. At present, this approach has not yet been explored by existing methods and datasets. To bridge this gap, we propose a new task called Multi-view Amodal Instance Segmentation (MAIS) and introduce the MUVA dataset, the first MUlti-View AIS dataset that takes the shopping scenario as instantiation. MUVA provides comprehensive annotations, including multi-view amodal/visible segmentation masks, 3D models, and depth maps, making it the largest image-level AIS dataset in terms of both the number of images and instances. Additionally, we propose a new method for aggregating representative features across different instances and views, which demonstrates promising results in accurately predicting occluded objects from one viewpoint by leveraging information from other viewpoints. Besides, we also demonstrate that MUVA can benefit the AIS task in real-world scenarios.1

This paper introduces KinZ, an Azure Kinect toolkit for Python and Matlab. This toolkit encapsulates the Azure Kinect's main functionality in an easy-to-use class with high-level methods. This software is written mostly in C/C++ with a pybind interface for Python and Mex functions for Matlab. It provides access to color, depth, infrared cameras, a mapping between cameras, colored point clouds, IMU sensor data, body tracking with 32 body joints plus 32 hand keypoints. We showed that KinZ applications could run in real-time at 1080p resolution, with the benefit of a high-level language reducing the learning curve and allowing a significant decrease in development time for prototyping and research.

Owing to their large field of view, a catadioptric imaging system is widely used in several research areas, including robot navigation, surveillance, and three‐dimensional (3D) reconstruction. The modelling and calibration of these systems allow the accurate measurement of objects. This paper presents a calibration method for a panoramic 3D reconstruction system based on pattern projection composed of two modules. The first module, which contains a charged coupled device camera and a parabolic mirror allows the acquisition of catadioptric images. The second module is a catadioptric projection system made of a light projector and a parabolic mirror that generates a projected pattern over the target for reconstruction purposes. Both the modules are calibrated independently. The method achieves a reprojection error of 0.6 pixels in the camera module and 1.2 pixels in the catadioptric projection system. To evaluate the performance of the proposed method, the results show 3D reconstructions of multiple objects with tolerances around 1.3 mm.

Nonverbal communication is an intrinsic part in daily face-to-face meetings. A frequently observed behavior during social interactions is mirroring, in which one person tends to mimic the attitude of the counterpart. This paper shows that a computer vision system could be used to predict the perception of competence in dyadic interactions through the automatic detection of mirroring events. To prove our hypothesis, we developed: (1) A social assistant for mirroring detection, using a wearable device which includes a video camera and (2) an automatic classifier for the perception of competence, using the number of nodding gestures and mirroring events as predictors. For our study, we used a mixed-method approach in an experimental design where 48 participants acting as customers interacted with a confederated psychologist. We found that the number of nods or mirroring events has a significant influence on the perception of competence. Our results suggest that: (1) Customer mirroring is a better predictor than psychologist mirroring; (2) the number of psychologist's nods is a better predictor than the number of customer's nods; (3) except for the psychologist mirroring, the computer vision algorithm we used worked about equally well whether it was acquiring images from wearable smartglasses or fixed cameras.