Three-Dimensional Visual Grounding (3DVG) aims to locate objects in 3D scenes based on natural language queries. Existing methods typically rely on absolute position encoding of global objects to model spatial relationships for target localization. However, this often results in inadequate spatial understanding and redundant or invalid encoding. To address these limitations, we propose the Relative K-object Fusion Perception 3D Visual Grounding (3DRelKG) framework. By dynamically sampling a limited number of scene reference points, our approach enhances the modeling of spatial relationships for localization targets by learning only the positional and relative spherical geometric features of these reference points with respect to scene objects. Additionally, we introduce a heterogeneous feature fusion module, whose core is an information interaction mechanism based on the similarity matrix of heterogeneous features. This approach naturally avoids the problem of unreliable attention interaction weights that arise from directly calculating the similarity between heterogeneous features. Experiments on ScanRefer, SR3D, and NR3D demonstrate that our method outperforms state-of-the-art models, improving accuracy by 2.3%, 4.3%, and 2.1%, respectively, and increasing inference speed by 43.8%.

Deep-learning-based salient object detection (SOD) has achieved significant success in recent years. The SOD focuses on the context modeling of the scene information, and how to effectively model the context relationship in the scene is the key. However, it is difficult to build an effective context structure and model it. In this article, we propose a novel SOD method called dynamic and adaptive graph convolutional network (DAGCN) that is composed of two parts, adaptive neighborhood-wise graph convolutional network (AnwGCN) and spatially restricted K-nearest neighbors (SRKNN). The AnwGCN is novel adaptive neighborhood-wise graph convolution, which is used to model and analyze the saliency context. The SRKNN constructs the topological relationship of the saliency context by measuring the non-Euclidean spatial distance within a limited range. The proposed method constructs the context relationship as a topological graph by measuring the distance of the features in the non-Euclidean space, and conducts comparative modeling of context information through AnwGCN. The model has the ability to learn the metrics from features and can adapt to the hidden space distribution of the data. The description of the feature relationship is more accurate. Through the convolutional kernel adapted to the neighborhood, the model obtains the structure learning ability. Therefore, the graph convolution process can adapt to different graph data. Experimental results demonstrate that our solution achieves satisfactory performance on six widely used datasets and can also effectively detect camouflaged objects. Our code will be available at: https://github.com/CSIM-LUT/DAGCN.git.

Abstract One of the fundamental problems when building perception systems for robot is to be able to provide semantic information as well as positioning in three‐dimensional (3D) space. However, two‐dimensional (2D) object detectors only can provide the semantic information and pixel coordinate in 2D space. While, the depth image can reflect the relative distance, and the semantic description of the object is poor. In this article, a novel object perception and positioning method via deep perception learning object detection is proposed. First, the RGB image and depth image are collected through the Kinect, and the depth image is processed to ensure the robustness of the model. Then, the RGB image can obtain the object semantic and pixel location information through an object detector based on deep learning. Finally, the object size measurement and 3D positioning are realized by combining the pixel location and the depth information. As a result, the advantages of very accurate 2D detector and the accurate depth information can be effectively captured in our model. Experimental results demonstrate that our method achieves a high accuracy of size measurement and spatial positioning.

Recently, many excellent weakly supervised semantic segmentation (WSSS) works are proposed based on class activation mapping (CAM). However, there are few works that consider the characteristics of medical images. In this paper, we find that there are mainly two challenges of medical images in WSSS: i) the boundary of object foreground and background is not clear; ii) the co-occurrence phenomenon is very severe in training stage. We thus propose a Causal CAM (C-CAM) method to overcome the above challenges. Our method is motivated by two cause-effect chains including category-causality chain and anatomy-causality chain. The category-causality chain represents the image content (cause) affects the category (effect). The anatomy-causality chain represents the anatomical structure (cause) affects the organ segmentation (effect). Extensive experiments were conducted on three public medical image data sets. Our C-CAM generates the best pseudo masks with the DSC of 77.26%, 80.34% and 78.15% on ProMRI, ACDC and CHAOS compared with other CAM-like methods. The pseudo masks of C-CAM are further used to improve the segmentation performance for organ segmentation tasks. Our C-CAM achieves DSC of 83.83% on ProMRI and DSC of 87.54% on ACDC, which outperforms state-of-the-art WSSS methods. Our code is available at https://github.com/Tian-lab/C-CAM.

We propose a robust algorithm for aligning rigid, noisy, and partially overlapping red green blue-depth (RGB-D) point clouds. To address the problems of data degradation and uneven distribution, we offer three strategies to increase the robustness of the iterative closest point (ICP) algorithm. First, we introduce a salient object detection (SOD) method to extract a set of points with significant structural variation in the foreground, which can avoid the unbalanced proportion of foreground and background point sets leading to the local registration. Second, registration algorithms that rely only on structural information for alignment cannot establish the correct correspondences when faced with the point set with no significant change in structure. Therefore, a bidirectional color distance (BCD) is designed to build precise correspondence with bidirectional search and color guidance. Third, the maximum correntropy criterion (MCC) and trimmed strategy are introduced into our algorithm to handle with noise and outliers. We experimentally validate that our algorithm is more robust than previous algorithms on simulated and real-world scene data in most scenarios and achieve a satisfying 3-D reconstruction of indoor scenes.

In this study, the authors investigate the problem of enhanced image no‐reference (NR) quality assessment. For resolving the problem of the enhanced images, it is difficult to obtain reference images, this study proposes an NR image quality assessment (IQA) model based on colour space distribution. Given an enhanced image, our method first uses a gist to select a clear target image in which the scene, colour and quality are similar to the hypothetical reference images. And then, the colour transfer is used between the input images and target images to construct the reference image. Next, the appropriate IQA method is used to assess enhanced image quality. The absolute colour difference and feature similarity (FSIM) are used to measure the colour and grey‐scale image quality, respectively. Extensive experiments demonstrate that the proposed method is good at evaluating enhanced image quality for X‐ray, dust, underwater and low‐light images. The experimental results are consistent with human subjective evaluation and achieve good assessment effects.

Security surveillance is critical to social harmony and people's peaceful life. It has a great impact on strengthening social stability and life safeguarding. Detecting anomaly timely, effectively and efficiently in video surveillance remains challenging. This paper proposes a new approach, called S2-VAE, for anomaly detection from video data. The S2-VAE consists of two proposed neural networks: a Stacked Fully Connected Variational AutoEncoder (SF-VAE) and a Skip Convolutional VAE (SC-VAE). The SF-VAE is a shallow generative network to obtain a model like Gaussian mixture to fit the distribution of the actual data. The SC-VAE, as a key component of S2-VAE, is a deep generative network to take advantages of CNN, VAE and skip connections. Both SF-VAE and SC-VAE are efficient and effective generative networks and they can achieve better performance for detecting both local abnormal events and global abnormal events. The proposed S2-VAE is evaluated using four public datasets. The experimental results show that the S2-VAE outperforms the state-of-the-art algorithms. The code is available publicly at https://github.com/tianwangbuaa/.

In this study, the authors present a method to extract moving objects in image sequences. The proposed approach is based on a graph cuts algorithm defined on a spatiotemporal superpixel neighbourhood. Presegmented superpixels are partitioned into foreground and background while preserving temporal and spatial coherence. It achieves this goal by three steps. First, instead of operating at pixel level, the superpixels are advocated as basic units of the authors segmentation scheme. Second, within the graph cuts framework, two superpixel‐based data terms and two superpixel‐based smoothness terms are proposed to solve segmentation problem. Finally, the proposed method yields the segmentation of all the superpixels within video volume by the graph cuts algorithm. To illustrate the advantages of this approach, the quantitative and qualitative results are compared with other state‐of‐the‐art methods. The experimental results show that the proposed method gives better performance of segmentation with respect to these methods.

The iterative closest point (ICP) algorithm is an accurate approach for the registration between two point sets on the same scale. However, it can not handle the case with different scales. This paper proposes a fast and robust ICP algorithm for isotropic scaling point sets registration (FRISICP). In order to accurately and directly estimate the scale factor without any constraints, we introduce a bidirection distance measurement method into the least square (LS) problem. Then to keep computational efficiency when the number of points in the set increasing, we further introduce a sparse-to-dense hierarchical model in ICP algorithm to speed up the isotropic scaling point set matching process. Experimental results demonstrate that the proposed FRISICP method outperforms other algorithms on both 2D and 3D point sets.

In this paper, we present a method to extract moving objects in monocular image sequences. The proposed method is based on graph cuts defined on a spatio-temporal region adjacency graph (RAG). First, we initially over-segment each frame in the video, and take the over-segmented regions as the vertices in the 3D spatio-temporal graph. Second, multiple cues are fused together to extract objects accurately. Finally, accurate foreground/background segmentation are efficiently achieved by binary graph cut. The experimental results showed that the proposed method improved the performance of segmentation with respect to the popular methods.

We propose a new approach to motion segmentation that is based on auto-generated strokes. The novelty of the approach is twofold. First, inspired by recent work of other researchers we formulate the problem as that of interactive segmentation. Instead of inputting the strokes by the user, the strokes in our approach are auto-generated. The second novelty of the paper is formulation in which, unlike in many other motion segmentation algorithms, we do not use complex algorithm which fuses the output of multiple cues to segment foreground objects, a simple and effective maximum hybrid similarity method is presented. The maximum hybrid similarity does not need to set the threshold in advance. Experimental results have shown the superiority of the proposed method in extracting moving objects.

Saliency detection is an useful technique for image semantic analysis such as auto image segmentation, image retargeting, advertising design and image compression. Inspired by two existing saliency detection algorithms, named spectral residual (SR) and phase spectrum of quaternion Fourier transform (PQFT), we propose a new bottom-up saliency detection method which is featured with the introduction of hypercomplex spectral contrast (HSC) in saliency detection. The proposed HSC algorithm introduces the HSV color image vector space in hypercomplex number, and is better comprehensive to consider amplitude spectral contrast into saliency model as well as phase spectral contrast. Meanwhile, we also incorporate the human vision nonuniform sampling into our model, which is a common phenomenon that directs visual attention to the logarithmic center of image in natural scenes. Experimental results on two public saliency detection datasets show that our approach performs better than four state-of-the art approaches remarkably.

In this paper, we present a system for object-based video summarization facilitated by an efficient video object segmentation system. We eliminate the redundancy not only from spatial and temporal domain, but also from content domain. First, we detect shot boundaries and extract video objects by a 3D graph-based algorithm. Once the objects are obtained, the shape of the objects need to be represented. The key objects are extracted in a global manner by K-means clustering of shapes. Experimental results on the proposed object-based scheme combined with efficient video object segmentation show desirable summarization.

The JPEG2000 system provides scalability with respect to quality, resolution and color component in the transfer of images. However, scalability with respect to semantic content is still lacking. We propose a biologically plausible salient region based bit allocation mechanism within the JPEG2000 codec for the purpose of augmenting scalability with respect to semantic content. First, an input image is segmented into several salient proto-objects (a region that possibly contains a semantically meaningful physical object) and background regions (a region that contains no object of interest) by modeling visual focus of attention on salient proto-objects. Then, a novel rate control scheme distributes a target bit rate to each individual region according to its saliency, and constructs quality layers of proto-objects for the purpose of more precise truncation comparable to original quality layers in the standard. Empirical results show that the suggested approach adds to the JPEG2000 system scalability with respect to content as well as the functionality of selectively encoding, decoding, and manipulation of each individual proto-object in the image, with only some slightly trivial modifications to the JPEG2000 standard. Furthermore, the proposed rate control approach efficiently reduces the computational complexity and memory usage, as well as maintains the high quality of the image to a level comparable to the conventional post-compression rate distortion (PCRD) optimum truncation algorithm for JPEG2000.

Joint network-source video coding (JNSC) is targeted to achieve the optimum delivery of a video source to a number of destinations over network with capacity constraints. In this paper, a practical scalable multiple description coding is proposed for JNSC, based on Lagrangian rate allocation and scalable video coding. After the spatiotemporal wavelet transformation of input video sequence and the bit plane coding and context-based adaptive binary arithmetic coding, jointing network-source coding is performed on the coding passes of the code blocks (CB) using Lagrangian rate allocation.