Synthetic images serve as a promising alternative to real images in 3D hand pose estimation, providing accurate annotations at a lower cost. However, the domain gap between real and synthetic images constrains the generalization ability of hand pose estimation trained on synthetic data. Previous methods rely on Generative Adversarial Networks (GANs) for domain translation; however, they fail to achieve realistic depth synthesis due to instability and limited image quality. Diffusion models provide high-quality synthesis due to their stability and controllability. However, existing methods often ignore the 3D structure awareness in hand image generation. In this paper, we propose a Dual-Branch 3D Spatial-Aware Latent Diffusion (DSW-LD) for realistic depth image generation. The Global Structure Module (GSM) and the Local Geometry Module (LGM) complement each other, with GSM capturing global spatial structure through coarse-grained 3D joint features and LGM focusing on local geometric details using fine-grained 3D mesh representations. To maintain the global structure consistency, we adopt a layer-aware injection mechanism that enables the model to adaptively learn the optimal representation from fused 2D latent representations and 3D joint features. To explicitly align 3D and 2D features of local regions and enhance the flexibility of feature matching, we design a dynamic depth-aware interpolation to project 3D mesh features into 2D image space. Both quantitative and qualitative experimental results demonstrate the superiority of our method over the state-of-the-arts for realistic depth synthesis. Compared to training only on real depth images, our method enables the hand pose estimator to achieve significantly better performance with our synthetic data and less real data (10%).

Federated learning (FL) has recently been applied to the parameter-efficient fine-tuning (PEFT) of large language models (LLMs). While promising, client resource heterogeneity has imposed the challenge of the "bucket effect" to FL, where model configuration must cater to the client with the fewest resources. To tackle this issue, heterogeneous low-rank adaptation (LoRA) has recently emerged in FL, which enables clients to do local fine-tuning with different LoRA ranks. However, existing works in this area typically adopt zero-padding, stacking, or singular value decomposition (SVD) for LoRA aggregation, which often incur precision loss or significant overhead, limiting their practicality. In this article, we propose ECLoRA, a novel method for federated fine-tuning with heterogeneous LoRA settings across clients. ECLoRA employs randomized SVD (RSVD) to dramatically reduce aggregation overhead while introducing an error compensation (EC) mechanism that incorporates the decomposition error from previous rounds to improve aggregation precision. Extensive experiments on four widely used foundation models across six public tasks demonstrate the effectiveness of ECLoRA. Specifically, ECLoRA is: (1) accurate, significantly improving the final model performance; (2) fast, accelerating convergence with an average speedup of $1.54\times $ to $3.01\times $ ; and (3) practical, reducing aggregation time by approximately $40\times $ compared to classical SVD.

Novel object captioning task aims at describing objects that are absent from training data. Due to the scarcity of novel objects, it’s challenging to find a way to utilize external data to improve model’s reasoning ability. While previously designed methods all follow a deep learning approach, we boost novel object captioning by incorporating reasoning with traditional deep learning framework. We design a manual from dictionaries that provides our model with sufficient and accurate external information on novel objects. We propose Manual-guided Context-aware Novel Object Captioning model (MC-NOC) that utilizes image and caption context to generate novel object captions. It contains a Manual-Guided Novel Object Reasoning module to reason about novel objects based on other objects of the given image and a Caption Reconstruction module to incorporate novel objects into generated captions according to caption context. We validate MC-NOC with state-of-the-art performance on the challenging Held-out COCO and Nocaps dataset, leading their leaderboard. In particular, we improved the CIDER metric by 6.4 points on the held-out coco dataset. Comprehensive experiments demonstrate our model’s reasoning capability and the quality of generated captions.

Transformer-based pretrained language models (PLMs) have achieved great success in modern NLP. An important advantage of PLMs is good out-of-distribution (OOD) robustness. Recently, diffusion models have attracted a lot of work to apply diffusion to PLMs. It remains under-explored how diffusion influences PLMs on OOD data. The core of diffusion models is a forward diffusion process which gradually applies Gaussian noise to inputs, and a reverse denoising process which removes noise. The noised input reconstruction is a fundamental ability of diffusion models. We directly analyze OOD robustness by measuring the reconstruction loss, including testing the abilities to reconstruct OOD data, and to detect OOD samples. Experiments are conducted by analyzing different training parameters and data statistical features on eight datasets. It shows that finetuning PLMs with diffusion degrades the reconstruction ability on OOD data. The comparison also shows that diffusion models can effectively detect OOD samples, achieving state-of-the-art performance in most of the datasets with an absolute accuracy improvement up to 18%. These results indicate that diffusion reduces OOD robustness of PLMs.

RGB and depth modalities provide complementary information, which can be effectively utilized to improve the performance of hand detection in the wild. Most existing fusion-based methods model the channel-wise or spatial-wise cross-modal correlation to exploit the complementary RGB-D information, in which the modeling operations are shared across all input samples. However, the input images show various modes due to the high diversity of scenes in the wild. This inter-sample variance cannot be effectively perceived by static modeling operations shared across all samples. To address this problem, we propose a Sample-Adapt Fusion Network (SAFNet) with Channel Dynamic Refinement Module (CDRM) and Spatial Dynamic Aggregation Module (SDAM) to adaptively model the channel-wise and spatial-wise cross-modal correlation. Specifically, we propose a Multi-kernel Attention Module (MAM) to individually generate attention maps for each input sample by applying learnable weighting operations to multiple convolutional kernels. Our method outperforms state-of-the-art methods on CUG Hand dataset.

Web-based AR technology has broadened human-computer interaction scenes from traditional mechanical devices and flat screens to the real world, resulting in unconstrained environmental challenges such as complex backgrounds, extreme illumination, depth range differences, and hand-object interaction. The previous hand detection and 3D hand pose estimation methods are usually based on single modality such as RGB or depth data, which are not available in some scenarios in unconstrained environments due to the differences between the two modalities. To address this problem, we propose a multimodal fusion approach, named Scene-Adapt Fusion (SA-Fusion), which can fully utilize the complementarity of RGB and depth modalities in web-based HCI tasks. SA-Fusion can be applied in existing hand detection and 3D hand pose estimation frameworks to boost their performance, and can be further integrated into the prototyping AR system to construct a web-based interactive AR application for unconstrained environments. To evaluate the proposed multimodal fusion method, we conduct two user studies on CUG Hand and DexYCB dataset, to demonstrate its effectiveness in terms of accurately detecting hand and estimating 3D hand pose in unconstrained environments and hand-object interaction.

Artistic portrait drawing (APDrawing) generation has seen progress in recent years. However, due to the naturally high scarcity and artistry, it is difficult to collect large‐scale labeled and paired data and generally divide drawing styles into several specific recognized categories. Existing works suffer from the limited labeled data and naive manual division of drawing styles according to the corresponding artists. They cannot adapt to the actual situations, for example, a single artist might have multiple drawing styles and APDrawings from different artists might share similar styles. In this paper, we propose to use unlabeled and unpaired data and perform the task in an unsupervised manner. Without manual division of drawing styles, we take each portrait drawing as a unique style and introduce self‐supervised feature learning to learn free styles for unlabeled portrait drawings. Besides, we devise a style bank and a decoupled cycle structure to take over two main considerations in the task: generation quality and style control. Extensive experiments show that our model is more adaptable to different style inputs than state‐of‐the‐art methods.

The primary concern of numerous online social media network (SMN) platforms is how to provide users with effective and personalized web services. To achieve this goal, SMN platforms typically begin by collecting user preferences based on user behaviors (e.g., browsing history, posts) or user profiles. However, the effective information about a specific user on a single SMN platform is limited and monotonous, preventing a comprehensive reflection of the user's preferences. Therefore, recognizing anonymous but identical users across two SMNs to integrate their information is crucial for enhancing web services. Clearly, cross-platform research has the potential to aid in the resolution of numerous problems in service computing theory and applications. Therefore, in this article, we present theCross-PlatformUserMatcher (CPUM) framework, which attempts to map users into a union vector space and then performs user matching based on distance metrics. In particular, we introduce a GNN-based encoderAdaptiveGraphAttention Network (AdaGAT) for modeling user attributes and topology jointly in the social networks to capture two typical alignment principles: topology consistency and attribute consistency. Moreover, we derive AdaGAT from the heuristic of the spectral network alignment technique FINAL, which theoretically guarantees AdaGAT's efficacy. To the best of our knowledge, AdaGAT is the first representation-based alignment model to integrate these two alignment principles synergistically. In addition, two position encoding schemes are introduced to prevent alignment confusion that commonly arises with GNN-based alignment models. Extensive experiments on real-world datasets validate the superiority of the proposed framework.

Network anomaly detection provides a reliable and stable service to detect faults and prevent security attacks effectively. However, existing detection methods still encounter many challenges. The supervised learning method is unsuitable because the anomaly samples are seriously sparse and hard to label. Unsupervised learning, as a promising method, is widely used while the discriminative features are ignored when reconstructing from the normal feature space. This paper proposes a novel probability correlation learning based on autoencoder called PCDetect, a semi-supervised learning method. Since we assumed the anomaly samples deviate from the distribution of normal samples, approximating the distribution of original data is proposed as an efficient preprocessing methodology to capture the discriminative features. Moreover, an encoder-decoder neural network associated with the proposed loss function is designed to learn the low-dimensional feature representation from raw data and constrain the latent representation to follow different referenced distributions based on a few anomaly labels. In this way, The correlation of the referenced distribution and the reconstruction of latent representation will be used to quantify the probability of anomaly. Extensive experiments are conducted on two public real-world datasets, NSL-KDD and UNSW-NB15. Results show the proposed PCDetect can efficiently cope with the imbalance and high-dimensional issues compared with several popular supervised learning methods, significantly improve the accuracy, and reduce the false rate as a whole compared with unsupervised learning.

With the fast developing of Health-IoT supported by massive Machine Type Communication (mMTC) of 5G, personal physiological data of patients endure various malicious attacks from intelligent eavesdropper during the transmission procedure. The leakage of patients' data will threaten their personal information security. In this paper, we propose a hybrid Relay Selection and Cooperative Jamming (RS-CJ) scheme to resist intelligent eavesdroppers in the sensor network tier in Healthcare-IoT systems. In the hybrid RS-CJ scheme, the relay is adaptively selected from bio-sensors before the physical data are transmitted, and the unselected bio-sensors act as friendly jammers. Moreover, we derived the security capacity of the legitimate and eavesdropping channels under the RS-CJ scheme. Then, we obtained the closed-form expressions of the Ergodic Achievable Security Rate (EASR). We also analyzed the negative influence on EASR when the intelligent eavesdropper holds different active jamming power. The simulation results verify the correctness of EASR, and our proposed RS-CJ scheme is feasible in general Healthcare-IoT systems. Compared with related solutions, EASR of the proposed hybrid RS-CJ scheme is 64% higher than the existing relay selection and interference scheme, as well as 3.38 times higher than that of the optional relay selection scheme proposed by other scholars.

As climate and energy issues are increasingly serious, energy efficiency in fifth generation (5G) communication networks has attracted tremendous attention. With the development of heterogeneous ultra-dense networks (H-UDNs), the growing number of base stations (BSs) will lead to more energy consumption. In this paper, we propose a BS sleeping strategy based on device-to-device (D2D) clustering communication. First, we obtain the optimal D2D cluster Head (CH) density by maximizing the total number of users which are served by the D2D CHs. Then, we consider the Received Signal Strength (RSS) and the Signal to Interference plus Noise Ratio (SINR) of the user to select the optimal D2D CHs. Furthermore, we adjust the load of the lightly loaded BSs and switch them into sleeping mode to save the energy of the network. The simulation results show that the strategy not only reduces energy consumption but also improves the user coverage probability of the network.

Multi-access Edge Computing (MEC) and ubiquitous smart devices help serve end-users efficiently and optimally through providing emerging edge-deployed services. Meanwhile, heavy and time-varying traffic loads are produced in the edge network, so that an efficient traffic forwarding mechanism is required. In this paper, we propose a parallel and distributed learning approach, DeepHop, to adapt to the volatile environments and realize hop-by-hop routing. The Multi-Agent Deep Reinforcement Learning (MADRL) is used to alleviate the edge network congestion and maximize the utilization of network resources. DeepHop determines the routing among edge network nodes for heterogeneous types of traffic according to the current workload and capability. By joining with an attention mechanism, DeepHop obtains the semantics from the elements of the network state to help the agents learn the importance of each element on routing. Experiment results show that DeepHop achieves the increase of successfully transmitted packets by 15% compared with the state-of-the-art algorithms. Besides, DeepHop with an attention mechanism reduces convergence time by nearly half compared with the common-used structures of neural networks.

The millimeter wave (mmWave) band has been considered as an effective way to meet the increasing traffic demand by sufficient bandwidth and high spatial reuse. The harsh propagation experienced at such high frequencies requires a dense base station deployment, which is not feasible to provide wired backhaul due to the unavailability of fiber drops. To address this issue, the integrated access and backhaul (IAB) architecture is proposed by the third generation partnership project (3GPP). In this paper, we investigate the spectrum and power allocation for IAB mmWave enabled cellular network to maximize the network capacity with considering the data rate requirements. A novel resource allocation scheme based on the sequential convex programming approach (RASCPA) is proposed. The simulation results show that our proposed scheme is superior to other schemes. The proposed scheme can increase the network capacity while satisfying the user data rate requirements.

As the deepening of 5G's support for the e-health industry, more and more wireless medical devices will suffer from various attacks and threats. Especially, the security of implantable medical devices (IMDs) which have limited computational capabilities and stringent power constraints becomes a critical issue. According to the channel state information, we exploit the special characteristics of the received signal strength (RSS) ratio between wearable proxy devices (WPDs) and IMDs in wireless body area networks (WBANs) to distinguish legitimate users and attackers. Moreover, based on the idea of proposed authentication request filtering (ARF), we design two corresponding light-weight security protocols to defend the forced authentication (FA) attacks and enhance the accessibility of IMD in emergency mode respectively. Simulation results show that the proposed ARF scheme to defend FA attacks achieves a high authentication response rate (ARR) with 99.2% for legitimate users and a low ARR with 2.4% for attackers at the maximum gap threshold point. Furthermore, when applied in emergency mode, the ARF scheme allows up to 96.3% emergency rescue devices to access the IMDs with only one attempt.

The ultra-dense deployment of small cells has been applied to the 5th-generation (5G) mobile networks. A large number of base stations (BSs) will lead to a dramatic increase in energy consumption, and network resources will be more difficult to fully utilize. In this paper, we propose the dynamic load adjustments (DLA) algorithm for small cells in heterogeneous ultra-dense networks. The proposed algorithm applies Q-learning to learn effective offloading policies which could combine the energy-saving function and the load balancing function. Based on the DLA algorithm, the heterogeneous ultra-dense networks could adjust the traffic load to turn off some redundant BSs or balance the load between heavily loaded BSs and lightly loaded BSs. The simulation results show that the algorithm not only improves the network energy efficiency when the average load of the networks is light, but also improves the network throughput when the average load of the networks is heavy.