Abstract In recent years, panoptic segmentation has drawn increasing amounts of attention, leading to the rapid emergence of numerous related algorithms. A variety of deep neural networks have been used more frequently for panoptic segmentation, which is motivated by the significant success of deep learning methods in other tasks. This article presents a comprehensive exploration of panoptic segmentation, focusing on the analysis and understanding of RGB image data. Initially, the authors introduce the background of panoptic segmentation, including deep learning models and image segmentation. Then, the authors thoroughly cover a variety of panoptic segmentation‐related topics, such as datasets connected to the field, evaluation metrics, panoptic segmentation models, and derived subfields based on panoptic segmentation. Finally, the authors examine the difficulties and possibilities in this area and identify its future paths.

Generally, multimodality data contain different potential information available and are capable of providing an enhanced analytical result compared to monosource data. The way to combine the data plays a crucial role in multimodality data analysis which is worth investigating. Multimodality clustering, which seeks a partition of the data in multiple views, has attracted considerable attention, for example, robust multiview spectral clustering (RMSC) explicitly handles the possible noise in the transition probability matrices associated with different views. Spectral clustering algorithm embeds the input data into a low-dimensional representation by dividing the clustering problem into k subproblems, and the corresponding eigenvalue reflects the loss of each subproblem. So, the eigenvalues of the Laplacian matrix should be treated differently, while RMSC regularizes each singular value equally when recovering the low-rank matrix. In this paper, we propose a multimodality clustering algorithm which recovers the low-rank matrix by weighted nuclear norm minimization. We also propose a method to evaluate the weight vector by learning a shared low-rank matrix. In our experiments, we use several real-world datasets to test our method, and experimental results show that the proposed method has a better performance than other baselines.

We study the question of hand posture recognition by developing a new class of Bag-of-Features (BoF), namely Hierarchical BoF. The Hierarchical BoF captures spatial information by dividing whole hand area into several sub-regions and projecting local features onto horizontal and vertical directions. A similarity measurement based on Histogram Intersection Kernel (HIK) is proposed to classify hand postures. According to experimental result in Section 4, Hierarchical BoF works well in hand posture recognition task owing to the following properties: 1) the model captures spatial information of each component of hand postures; 2) the model has good adaptability for various complex background conditions.

Hand posture is a natural and effective interaction between human and robot. In this paper, we use monocular camera as input device, and an improved Bag-of-Words (BoW) method is proposed to detect and recognize hand posture based on a new descriptor ARPD (Appearance and Relative Position Descriptor) and spectral embedding clustering algorithm. To track hand motion rapidly and accurately, we have designed a new framework based on improved BoW and CAMSHIFT algorithm. The thorough evaluation of our algorithm is presented to show its usefulness.