Vibration sensing is crucial for machinery health monitoring, but traditional contact sensors face deployment challenges. Recently, millimeter wave (mmWave) radar has emerged as a promising contact-free alternative. However, since radar is mainly sensitive to the vibrations perpendicular to its antennas, existing works can only achieve 1D/2D vibration sensing based on single radar, or 3D sensing by multiple ones. In this paper, we propose 3DVidar, a single-radar 3D vibration sensing system without external reference target. Considering the insufficient information provided by single radar, we introduce a multi-point multi-path multi-antenna signal enhancement strategy to compensate for the lack of 3D vibration information. Furthermore, we develop two dedicated mechanisms to selectively filter the most informative radar signals for subsequent processing. Based on the enhanced signals, we design 3D-VRNet, a deep learning framework that incorporates positional priors and fuses multi-view signals through multi-scale convolutions and an attention mechanism. We implement 3DVidar on a commercial mmWave radar, and the results on two type of vibration targets show that it can accurately reconstruct 3D trajectory across various conditions.

Dexterous grasping and manipulation with multifingered robotic hands presents a significant challenge due to their high degrees of freedom and the need for task-specific adaptations. Existing methods usually adopt single-task learning framework or focus on simple stable wrap grasping, limiting their efficiency and generalization ability when encountering new task or precise functional grasping pose. In this article, we introduce MetaGrasp, a novel approach that defines dexterous functional grasping as a multitask reinforcement learning (RL) problem based on hand grasp pose classification. Our method features a unique gradual skill curriculum learning (GSCL) framework, which structures the learning process into three stages: beginner, intermediate, and advanced curriculum learning according to the level of difficulty. MetaGrasp leverages this hierarchical learning structure to develop a versatile, adaptive grasping policy that can grasp objects based on hand grasp pose and object point cloud inputs. Taking five hand grasp types as research cases, the trained policy with our MetaGrasp can be easily adpated to grasp different object instances from different object categories according to functional grasp intentions specified by one expert demonstration without requiring extensive system interaction. We categorize the dexterous functional grasping tasks of a five-fingered robotic hand into multiple tasks based on hand poses for RL, and to combine meta imitation learning (IL) with curriculum learning. The experimental results show that the MetaGrasp has better one-shot generalization ability on new grasp tasks, and outperforms state-of-the-art single-task dexterous grasping methods.

Humans achieve contact-rich dexterous grasping through the synergy of visual and tactile information. However, the high-dimensional action space of high DoF multi-fingered hands poses significant challenges to this operation. In this study, we address this complexity by controlling the robotic hand at the reduced dimensional level of individual fingers instead of the entire hand, and develop a finger-based multi-agent deep reinforcement learning strategy by regarding the wrist, arm, and each finger of the hand as intelligent agents. We commence by applying a single-agent reinforcement learning algorithm to guide the whole hand to reach the feasible approaching direction and distance to the object. Then, we develop neuroscience-inspired visuo-tactile fusion networks to train multiple agents to control their assigned fingers by effectively leveraging visual and tactile feedback. This enables dynamic and collaborative adjustments of finger-object interactions, ultimately achieving precise contact with specific areas of the objects. The grasping results on 8 objects show that our approach can achieve stable and compliant grasps. To the best of our knowledge, this is the first work that employs a finger-based multi-agent reinforcement learning approach to control the dexterous grasping process under the guidance of both visual and tactile feedback.

An efficient 3D spatial spectrum prediction method is essential for UAV networks operating in highly heterogeneous spectrum environments, enabling UAVs to proactively navigate toward areas with abundant available spectrum and make decisions to access to idle ones in advance. This paper introduces a novel Multi-Scale Temporal model for 3D spatial spectrum prediction (MST-3DSSP) that comprehensively captures complex correlations across 3D spatial, frequency, and multi-scale temporal domains. Specifically, the proposed model incorporates a 3D Spatial-Frequency Fusion (3DS-FF) module to extract and fuse 3D spatial and frequency features, along with a MultiScale Temporal Extraction (MS-TE) module that combines BiLSTM and Transformer blocks to capture both small scale and large scale temporal dependencies. These two modules enable the model to understand the complex correlations across 3D spatial, frequency, and multi-scale temporal domains, thereby allowing for more accurate spectrum predictions. Extensive experiments on real-world spectrum datasets demonstrate that MST-3DSSP significantly outperforms existing spectrum prediction methods, achieving higher prediction accuracy and reduced errors, thus providing a robust solution for improving spectrum efficiency in UAV networks.

Voice control has become an indispensable interaction method in smart devices. Compared to traditional microphones, mm Wave radar offers a promising solution for speech sensing in noisy environments. However, most current research relies on single-view information, such as vocal cord vibrations or lip movements, to classify speech, which overlooks important details like timbre, speech rate, and intonation, limiting the application of speech sensing. To address these issues, we develop a high-quality speech sensing method based on mm Wave radar, named SpDiff. This method accurately localizes the vocalizing target and, based on the human vocal mechanism, extracts multi-view speech features according to the movement characteristics of the vocal cords, lips, and face. Additionally, to generate high-quality speech signals, we design a conditional latent diffusion model (CLDM), which uses multi-view radar information as conditional guidance, accurately capturing the complex mapping relationships between radar and speech signal distributions. To evaluate the SpDiff method, we build a mmWave system using IWR1443Boost and recruit 14 volunteers to construct a dataset. Experimental results show that SpDiff achieves high standards in speech sensing, with the generated speech directly input into existing recognition models, achieving an average character and word error rate (CER/WER) of only 2.33% and 3.05%.

Robot grasp dataset is the basis of designing the robot's grasp generation model. Compared with the building grasp dataset for Low-DOF grippers, it is harder for High-DOF dexterous robot hand. Most current datasets meet the needs of generating stable grasps, but they are not suitable for dexterous hands to complete human-like functional grasp, such as grasp the handle of a cup or pressing the button of a flashlight, so as to enable robots to complete subsequent functional manipulation action autonomously, and there is no dataset with functional grasp pose annotations at present. This paper develops a unique Cost-Effective Real-Simulation Annotation System by leveraging natural hand's actions. The system is able to capture a functional grasp of a dexterous hand in a simulated environment assisted by human demonstration in real world. By using this system, dexterous grasp data can be collected efficiently as well as cost-effective. Finally, we construct the first dexterous functional grasp dataset with rich pose annotations. A Functional Grasp Synthesis Model is also provided to validate the effectiveness of the proposed system and dataset. Our project page is: https://hjlllll.github.io/DFG/.

Intelligent robotic manipulation is a challenging study of machine intelligence. Although many dexterous robotic hands have been designed to assist or replace human hands in executing various tasks, how to teach them to perform dexterous operations like human hands is still a challenge. This motivates us to conduct an in-depth analysis of human behavior in manipulating objects and propose an object-hand manipulation representation. This representation provides an intuitive and clear semantic indication of how the dexterous hand should touch and manipulate an object based on the object's own functional areas. At the same time, we propose a functional grasp synthesis framework, which does not require real grasp label supervision, but relies on the guidance of our object-hand manipulation representation. In addition, in order to obtain better functional grasp synthesis results, we propose a network pre-training method that can make full use of easily obtained stable grasp data, and a network training strategy to coordinate the loss functions. We conduct object manipulation experiments on a real robot platform, and evaluate the performance and generalization of our object-hand manipulation representation and grasp synthesis framework.

Category-level 6D pose estimation can be better generalized to unseen objects in a category compared with instance-level 6D pose estimation. However, existing category-level 6D pose estimation methods usually require supervised training with a sufficient number of 6D pose annotations of objects which makes them difficult to be applied in real scenarios. To address this problem, we propose a self-supervised framework for category-level 6D pose estimation in this paper. We leverage DeepSDF as a 3D object representation and design several novel loss functions based on DeepSDF to help the self-supervised model predict unseen object poses without any 6D object pose labels and explicit 3D models in real scenarios. Experiments demonstrate that our method achieves comparable performance with the state-of-the-art fully supervised methods on the category-level NOCS benchmark.

Many remarkable works have been proposed to deal with distortions problems in image fusion to date. However, the spectral distortion and the spatial distortion cannot always be well addressed at the same time. To deal with this, we propose an Adaptive Feature Pyramid Network (AFPN) to efficiently embed an Adaptive Detail Injection (ADI) module at different scales. Feature-domain injection gains are proposed in the ADI module to adaptively modulate spatial information and guide a refined detail injection. Furthermore, we propose a texture loss function to further guide our model to learn detail perception in each band. Experiments on QuickBird and GaoFen-1 datasets show that our method achieves superior performance and produces visually pleasing fusion images. Our code is available at https://github.com/yisun98/AFPN.

Traditional methods for RGB hand mesh recovery usually need to train a separate model for each dataset with the corresponding ground truth and are hardly adapted to new scenarios without the ground truth for supervision. To address the problem, we propose a self-supervised framework for hand mesh estimation, where we pre-learn hand priors from existing hand datasets and transfer the priors to new scenarios without any landmark annotations. The proposed approach takes binocular images as input and mainly relies on left-right consistency constraints including appearance consensus and shape consistency to train the model to estimate the hand mesh in new scenarios. We conduct experiments on the widely used stereo hand dataset, and the experimental results verify that our model can get comparable performance compared with state-of-the-art methods even without the corresponding landmark annotations. To further evaluate our model, we collect a large real binocular dataset. The experimental results on the collected real dataset also verify the effectiveness of our model qualitatively.

In recent years, many dexterous robotic hands have been designed to assist or replace human hands in executing various tasks. But how to teach them to perform dexterous operations like human hands is still a challenging task. In this paper, we propose a grasp synthesis framework to make robots grasp and manipulate objects like human beings. We first build a dataset by accurately segmenting the functional areas of the object and annotating semantic touch code for each functional area to guide the dexterous hand to complete the functional grasp and post-grasp manipulation. This dataset contains 18 categories of 129 objects selected from four datasets, and 15 people participated in data annotation. Then we carefully design four loss functions to constrain the model, which successfully generates the functional grasp of dexterous hand under the guidance of semantic touch code. The thorough experiments in synthetic data show our model can robustly generate functional grasp, even for objects that the model has not see before.

3D object detection is an important scene understanding task in autonomous driving and virtual reality. Approaches based on LiDAR technology have high performance, but LiDAR is expensive. Considering more general scenes, where there is no LiDAR data in the 3D datasets, we propose a 3D object detection approach from stereo vision which does not rely on LiDAR data either as input or as supervision in training, but solely takes RGB images with corresponding annotated 3D bounding boxes as training data. As depth estimation of object is the key factor affecting the performance of 3D object detection, we introduce an Instance-DepthAware (IDA) module which accurately predicts the depth of the 3D bounding box’s center by instance-depth awareness, disparity adaptation and matching cost reweighting. Moreover, our model is an end-to-end learning framework which does not require multiple stages or postprocessing algorithm. We provide detailed experiments on KITTI benchmark and achieve impressive improvements compared with the existing image-based methods. Our code is available at https://github.com/swords123/IDA-3D.

This paper focuses on the topic of vision based hand pose estimation from single depth map using convolutional neural network (CNN). Our main contributions lie in designing a new pose regression network architecture named CrossInfoNet. The proposed CrossInfoNet decomposes hand pose estimation task into palm pose estimation sub-task and finger pose estimation sub-task, and adopts two-branch cross-connection structure to share the beneficial complementary information between the sub-tasks. Our work is inspired by multi-task information sharing mechanism, which has been few discussed in hand pose estimation using depth data in previous publications. In addition, we propose a heat-map guided feature extraction structure to get better feature maps, and train the complete network end-to-end. The effectiveness of the proposed CrossInfoNet is evaluated with extensively self-comparative experiments and in comparison with state-of-the-art methods on four public hand pose datasets. The code is available.

This paper focuses on the success probability (or, equivalently, the signal-to-interference-plus-noise ratio (SINR) distribution) at the typical receiver in millimeter wave (mm-wave) device-to-device (D2D) networks. Unlike earlier works, we consider a more general and realistic case where devices in the network are equipped with heterogeneous antenna arrays so that the concurrent transmission beams are varying in width. Specifically, we first establish a general and tractable framework for the target network with Nakagami fading and directional beamforming. Next, we investigate the interactions among beams with different widths and their sensitivities to the adopted model for the antenna pattern. In addition, to show the impact of heterogeneous antenna arrays on the link performance, we derive the success probability of the typical receiver as well as its bounds to get deep insights on the performance of the network.

Millimeter-wave (mm-wave) device-to-device (D2D) communication is considered one of the most promising enabling technologies to meet the demanding requirements of future networks. Previous works on mm-wave D2D network analysis mostly considered the case that all devices were equipped with exactly the same number of antennas, whereas real networks are more complicated due to the coexistence of diverse devices. In this paper, we present a comprehensive investigation on the interference characteristics and link performance in mm-wave D2D networks where the concurrent transmission beams are varying in width. First, we establish a general and tractable framework for the target network with Nakagami fading and directional beamforming. To fully characterize the interference, we derive the mean and variance of the interference and then provide an approximation of the interference distribution by a mixture of the inverse gamma and the log-normal distributions. More importantly, the coexistence of varied beamwidths renders their interactions and thus the interference quite complicated and sensitive to the antenna pattern, highlighting the significance of adopting an accurate model for the antenna pattern. Second, to show the impact of heterogeneous antenna arrays on the link performance, we derive the signal-to-interference-plus-noise ratio and rate distributions of the typical receiver as well as their asymptotics, bounds, and approximations to get deep insights on the performance of the network.

Acoustic communication is the most versatile and widely used technology for underwater wireless networks. However, the frequencies used by current acoustic modems are heavily overlapped with the cetacean communication frequencies, where the man-made noise of underwater acoustic communications may have harmful or even fatal impact on those lovely marine mammals, e.g., dolphins. To pursue the environmental friendly design for sustainable underwater monitoring and exploration, specifically, to avoid the man-made interference to dolphins, in this paper, we propose a cognitive acoustic transmission scheme, called dolphin-aware data transmission (DAD-Tx), in multi-hop underwater acoustic networks. Different from the collaborative sensing approach and the simplified modeling of dolphins' activities in existing literature, we employ a probabilistic method to capture the stochastic characteristics of dolphins' communications, and mathematically describe the dolphin-aware constraint. Under dolphin-awareness and wireless acoustic transmission constraints, we further formulate the DAD-Tx optimization problem aiming to maximize the end-to-end throughput. Since the formulated problem contains probabilistic constraint and is NP-hard, we leverage Bernstein approximation and develop a three-phase solution procedure with heuristic algorithms for feasible solutions. Simulation results show the effectiveness of the proposed scheme in terms of both network performance and dolphin awareness.

Auction-based secondary spectrum market provides a platform for spectrum holders to share their under-utilized licensed bands with secondary users (SUs) for economic benefits. However, it is challenging for SUs to directly participate due to their limited battery power and capability in computation and communications. To shift complexity away from users, in this paper, we propose a novel multi-round service-oriented combinatorial spectrum auction with two-tier framework support. In Tier I, we introduce several secondary service providers (SSPs) to provide end-users with services by using purchased licensed bands even if the end-users do not have cognitive radio capability. When an SU submits its service request with certain bidding allowance to its SSP, the SSP will help find out which bands within its area are available and bid for the desired ones from the market in Tier II. Specifically, we formulate the bidding process at the SSP as an optimization problem by considering interference management, spectrum uncertainty, flow routing, and budget allowance. In Tier II, considering two possible manners of the seller, we propose two social-welfare-maximizing auction mechanisms accordingly, including the winner determination based on weighted conflict graph and the Vickrey-Clarke-Groves-styled price charging mechanism. Extensive simulations have been conducted and the results have demonstrated the higher revenue of the proposed scheme compared with the traditional commodity-oriented single-round truthful schemes.

Spectrum trading benefits primary users (PUs) by monetary gains and secondary users (SUs) by spectrum accessing opportunities in cognitive radio networks (CRNs). Unfortunately, most existing spectrum trading designs only focus on the guarantee of economic properties, but forget the wireless transmission nature, especially for multi-hop cognitive radio (CR) communications. In this paper, we propose an economic-robust session based spectrum trading, which has a joint consideration of economic properties such as incentive compatibility, individual rationality, and budget balance, and the end-to-end performance for multi-hop communications. Considering two bidding manners, i.e., bidding for the whole session and unit rate bidding, we formulate the spectrum trading optimization problems under multiple economic and multi-hop CR transmission constraints, design two pricing mechanisms to charge the winning spectrum bidders, and further mathematically prove the economic- robustness of the proposed spectrum trading schemes. Through extensive simulations, we show the proposed schemes are economic-robust and effective in improving spectrum utilization.

Simultaneous wireless information and power transfer (SWIPT) and interference alignment (IA) are two emerging techniques in energy harvesting and interference management for the next generation wireless networks, respectively. Although many studies have focused on SWIPT and IA, the conjunction of these two techniques is largely ignored, which should be noted to reuse the interference as energy for harvesting. In this paper, we jointly study SWIPT and IA in the multiuser MIMO system to realize energy harvesting and interference management simultaneously and effectively. Specifically, antenna selection (AS) based SWIPT scheme is proposed and analyzed for IA networks. Furthermore, power allocation (PA) for multiple data streams is designed to further improve its performance, and the closed-form solution can be obtained by Lagrange duality method. In addition, given the constrained number of antennas, another scheme called power splitting (PS) based SWIPT is utilized, where PA is also considered and formulated as a joint optimization problem. Simulation results are presented to show the superiority of the proposed schemes.

Inter-cell interference (ICI) and energy efficiency are two important issues in future generation cellular networks. These two issues are studied separately in most of previous works. Since both ICI and energy efficiency have great impacts on user quality of service (QoS) and energy consumption, they should be jointly studied and optimized in a common framework. In addition, most existing centralized schemes solving the ICI and energy efficiency problems may suffer from signaling overhead, outdated dynamics information, and scalability issues. In this paper, we proposed a common framework to dynamically allocate spectral resource to mitigate ICI and to save energy consumption at the same time. Base station (BS) sleep mode and fairness among users are considered in this paper. We first formulate the ICI and energy efficiency issues as a centralized optimization problem, and then we derive a distributed algorithm. Simulation results are presented to show the effectiveness of the proposed scheme.

Interference alignment (IA) is a promising technique that can eliminate the interferences in wireless networks effectively, and has been applied to cognitive radio (CR). However, the quality of desired signal may be poor when the interferences are aligned in the direction similar to that of the desired signal. Thus, we propose a novel IA scheme based on antenna selection to improve the performance of CR networks. In the proposed scheme, multiple antennas are equipped at each secondary receiver, and we choose some of them that have the optimal channel coefficients according to a certain objective function. Furthermore, we also consider the condition of imperfect channel state information (CSI), and an efficient antenna selection IA algorithm based on discrete stochastic optimization is proposed. Simulation results show that the proposed schemes can improve the performance of IA-based CR networks significantly.

This letter presents a method of correcting misaligned seismograms scanned during the digitization process of analog seismograms. The proposed method uses the low-rank matrix recovery technique to seek a Euclidean transformation that can be used to implement the correction. As the rank of a matrix is a natural measure of regularity and symmetry of images, a misaligned seismogram is assumed to be corrected when the rank of the texture extracted from the seismogram itself reaches the minimum. Therefore, the misalignment correction problem can be considered as a matrix rank minimization problem. The augmented Lagrange multiplier is applied to solve this minimization problem because of its fast convergence. Compared with the traditional geometrical methods, our method works efficiently and conveniently as well as overcomes corruptions, such as notes, spots, or marks.

Although dynamic base station (BS) operation (switch off or sleep mode) schemes can improve energy efficiency in cellular networks, the quality of service (QoS) (e.g., service blocking and service latency) experienced by users may be significantly affected by these energy-efficient schemes. On the other hand, to guarantee user QoS, user equipment (UE) admission control has been studied extensively. However, these two important areas have traditionally been addressed separately. In this paper, we propose a common framework to jointly study dynamic BS operation and UE admission. Specifically, we introduce a novel concept of BS admission control, and propose an optimal joint BS-UE admission control scheme considering both dynamic BS operation and dynamic UE traffic. The proposed scheme can maximize energy efficiency while guaranteeing user QoS. Simulation results are presented to show the effectiveness of the proposed scheme.

Super-resolution reconstruction (SRR) proposes a fusion of several low-quality images into one higher quality result with better optical resolution. However, due to the vast amount of calculation of the SRR algorithm, its implementation is too slow. In this paper, we present a GPU-based parallel implementation on SRR algorithm. The compute unified device architecture (CUDA) is a programming approach for performing scientific calculations on a graphics processing unit (GPU) as a data-parallel computing device. The proposed GPU-based implementation using CUDA is up to approximately 200 times faster than the corresponding optimized CPU counterparts.