The deployment of multimodal large language models (MLLMs) in edge-cloud networks faces critical challenges, including computational resource heterogeneity, memory bottlenecks, and bandwidth constraints. To address these issues, we propose EdgeSD, a novel framework that accelerates MLLM inference by integrating speculative decoding (SD) with edge-cloud collaboration. First, EdgeSD decouples the vision encoding and decoding processes of the draft MLLM across heterogeneous edge servers (ESs). This disaggregation architecture overcomes single-node memory constraints, enabling optimized resource utilization and high-resolution input processing. Second, to resolve the communication bottleneck and computational burden inherent in this distributed architecture, EdgeSD integrates a bandwidth-aware dynamic image token merging (ITM) method. Unlike general pruning techniques, this EdgeSD-specific ITM method focuses on minimizing inter-ES transmission latency for vision-decoding disaggregation while maintaining draft quality. Third, to optimize SD efficiency on consumer-grade ESs, EdgeSD employs an adaptive and scalable token tree structure solved using a parallel delta-stepping algorithm. This structure maximizes the number of accepted tokens under strict edge latency constraints. Extensive experiments on six multimodal datasets and five benchmarks with various MLLM pairs demonstrate that EdgeSD achieves substantial acceleration and throughput gains in edge-cloud collaboration scenarios using a lightweight draft MLLM, achieving 3.04-5.12x speedup compared to baseline methods.

Medical point cloud segmentation accuracy is often limited by feature confusion in boundary regions, which arises from point sparsity, shape complexity, and structural similarity. To address this, we introduce a boundary-aware perspective that categorizes boundaries into inner and outer types. We propose the Dynamic Boundary-Aware Network (DBA-Net), which employs a Boundary-aware Dual Stream (BDS) module to decouple semantic and boundary features via a Cross-Stream Attention Module (CSAM), alongside an Adaptive Boundary Pseudo-Label Calculation (ABP-LC) strategy for adaptive label generation. For outer boundaries, the Outer Boundary Adaptive Contextual Discrepancy-guided Graph Convolution (OACD-GC) module, incorporating a Soft Edge Connection (SEC) strategy and an Iterative Optimization Mechanism (IOM), enhances inter-class discrimination. For inner boundaries, the Inner Boundary Dy-namic Supervised Contrastive Enhancement (ID-SCE) module, utilizing a Multi-Positive Sample (MPS) strategy and a Dynamic Hard Negative Sample Update (DHNU) mechanism, improves intra-class aggregation and inter-class differentiation. Extensive experiments on the IntrA and 3DTeethSeg datasets demonstrate DBA-Net's superior performance in boundary recognition and overall segmentation accuracy.

Open-world specific emitter identification (SEI) is a practical but challenging task, because it requires accurate identification of both known and unknown emitters in open environments with channel variations. However, traditional closed-set SEI methods suffer from severe performance degradation in open-world environments since 1) closed-set SEI models would misclassify unknown specific emitters (SEs) as known ones, and 2) channel variations would lead to distribution shifts in the radio frequency (RF) fingerprint of known SEs. It is challenging to address both issues, as they coexist and interact with each other. In this paper, we propose a novel prototype matching with domain alignment framework for open-world SEI, which designed to simultaneously address the challenges of unknown SE identification and channel variation. The proposed framework first pretrains a feature extractor and then trains the pretrained feature extractor and an extended open-set classifier via the self-training paradigm. Towards the former, a partial domain alignment strategy is introduced to mitigate the impact of channel variations. Towards the latter, a prototype matching-based pseudo-labeling strategy is proposed to generate reliable pseudo-labels, facilitating subsequent fine-grained identification of unknown SEs. An open-set classification loss is introduced later to help the classifier avoid classifying unknown SEs as known ones. Experiments conducted on a public WiFi dataset demonstrate the superiority and robustness of the proposed method, illustrating its potential application prospects in open-world environments.

Virtual Reality (VR) creates a highly realistic and controllable simulation environment that can easily manipulate users' perception of space and time. However, while the sensation of “losing track of time” is often associated with enjoyable experiences, both the relationship between time perception and user experience in VR, and the underlying mechanisms of time perception itself, remain largely unexplored. In this study, we first investigated how different zeitgebers—such as light color, music tempo, and VR task—affect time perception. We then introduced the Relative Subjective Time Change (RSTC) method to explore the link between time perception and user experience quantitatively. Furthermore, to uncover the mechanisms underlying time perception in VR, we propose a computational model based on CNN and Transformer, named the Time Perception Modeling Network (TPM-Net), which leverages multimodal physiological data to infer users' time perception states in VR. In a between-subject experiment with 56 participants, our results indicate that the VR task factor significantly influences time perception, with red light and slow-tempo music contributing to an underestimation of time. The RSTC method effectively demonstrates that a relative underestimation of time in VR is strongly associated with enhanced user experience, presence, and engagement. Moreover, the TPM-Net shows great potential in modeling time perception, enabling further inference of relative changes in both time perception and user experience. Our study comprehensively elucidates the mechanisms of time perception in VR. It provides valuable insights and promising methodologies for exploring the relationship between time perception and user experience. Modeling time perception through physiological data marks a first step toward objectively assessing users' temporal perception states, offering a promising tool for VR-based therapy and training systems that require precise temporal awareness.

Spiking Neural Networks (SNNs) have garnered significant attention due to their biological plausibility and low power consumption. While spiking transformers enhance performance by combining SNNs with transformer architecture, most rely on rate coding, limiting energy efficiency. Temporal coding methods, such as Time-To-First-Spike (TTFS) coding, offer a more efficient alternative by encoding information based on the timing of a single spike. However, integrating TTFS with transformer architecture faces challenges due to incompatibility with batch normalization (BN) and residual connections (RC), which disrupt the precise spike firing times. In this paper, we propose temporal-coded BN (tBN) and temporal-coded RC (tRC) to address these issues. Building on tBN and tRC, we develop temporal-coded spiking attention (TSA) and temporal-coded spiking transformer (T-SpikeFormer), the first to combine TTFS coding with transformer architecture. Experimental results show our model achieves state-of-the-art performance for temporal-coded SNNs and comparable results to rate-coded SNNs while significantly reducing power consumption.

Facial expression recognition (FER) is a challenging task due to pervasive occlusion and dataset biases. Especially when facial information is partially occluded, existing FER models struggle to extract effective facial features, leading to inaccurate classifications. In response, we present ORSANet, which introduces the following three key contributions: First, we introduce auxiliary multi-modal semantic guidance to disambiguate facial occlusion and learn high-level semantic knowledge, which is two-fold: 1) we introduce semantic segmentation maps as dense semantics prior to generate semantics-enhanced facial representations; 2) we introduce facial landmarks as sparse geometric prior to mitigate intrinsic noises in FER, such as identity and gender biases. Second, to facilitate the effective incorporation of these two multi-modal priors, we customize a Multi-scale Cross-interaction Module (MCM) to adaptively fuse the landmark feature and semantics-enhanced representations within different scales. Third, we design a Dynamic Adversarial Repulsion Enhancement Loss (DARELoss) that dynamically adjusts the margins of ambiguous classes, further enhancing the model's ability to distinguish similar expressions. We further construct the first occlusion-oriented FER dataset to facilitate specialized robustness analysis on various real-world occlusion conditions, dubbed Occlu-FER. Extensive experiments on both public benchmarks and Occlu-FER demonstrate that our proposed ORSANet achieves SOTA recognition performance. Code is publicly available at https://github.com/Wenyuzhy/ORSANet-master.

The deployment of multimodal large language models (MLLMs) on resource-constrained mobile devices poses significant challenges due to their high computational demands. This paper introduces a novel two-timescale optimization framework for efficient MLLM inference in Edge-Cloud networks, addressing the problem of multi-timescale resource management by jointly optimizing slow-timescale MLLMs deployment decisions and fast-timescale batch scheduling, GPU resource allocation, and bandwidth allocation under dynamic network conditions and spatiotemporal request heterogeneity. Our key innovation is a hierarchical twin delayed deep deterministic policy gradient (HALTD3) algorithm that integrates attention mechanisms and long short-term memory networks to optimize slow-timescale MLLMs deployment and fast-timescale resource allocation, minimizing weighted system costs including deployment cost, end-to-end latency, and energy consumption, while meeting stringent quality-of-service requirements. Extensive experiments demonstrate that the HALTD3 algorithm substantially outperforms baseline methods in reducing system costs across diverse MLLM workloads and dynamic network scenarios, validating its effectiveness for practical edge-cloud collaborative inference.

Unsupervised domain adaptation (UDA) transfers knowledge from a labeled source domain to an unlabeled target domain on cross-domain object recognition by reducing a distribution discrepancy between the source and target domains (interdomain discrepancy). Prevailing methods on UDA were presented based on the premise that target data are collected in advance. However, in online scenarios, the target data often arrive in a streamed manner, such as visual image recognition in daily monitoring, which means that there is a distribution discrepancy between incoming target data and collected target data (intradomain discrepancy). Consequently, most existing methods need to re-adapt the incoming data and retrain a new model on online data. This paradigm is difficult to meet the real-time requirements of online tasks. In this study, we propose an online UDA framework via jointly reducing interdomain and intradomain discrepancies on cross-domain object recognition where target data arrive in a streamed manner. Specifically, the proposed framework comprises two phases: classifier training and online recognition phases. In the former, we propose training a classifier on a shared subspace where there is a lower interdomain discrepancy between the two domains. In the latter, a low-rank subspace alignment method is introduced to adapt incoming data to the shared subspace by reducing the intradomain discrepancy. Finally, online recognition results can be obtained by the trained classifier. Extensive experiments on DA benchmarks and real-world datasets are employed to evaluate the performance of the proposed framework in online scenarios. The experimental results show the superiority of the proposed framework in online recognition tasks.

Cross-subject mental fatigue detection via Electroencephalography (EEG) is challenging because EEG from different individuals varies greatly. Existing works have exploited domain adaption to alleviate the individual discrepancy due to personality, gender and so on. However, the distributions of data from new subjects and old ones are aligned by deceiving the domain discriminator. An inevitable issue of such a paradigm is that the samples near the decision boundary are easy to be misclassified. To address this issue, we propose a Separable Spatio-temporal Feature Aggregation (SSFA) that consists of a Spatio-temporal Feature Extractor (SFE) and a Separable Feature Aggregation mechanism (SFA). Specifically, SFE utilizes the spatio-temporal information in EEG and automatically tune the weights of temporal and spatial features, so as to update the model along the optimal direction and obtain more discriminative features. In addition, SFA employs two classifiers combined with sliced Wasserstein Discrepancy to aggregate each separate class together, facilitating the mapping of the new subjects to the support region of the old subjects. Leave-one-subject-out experiments conducted on a public fatigue dataset show that the proposed method performs better than state-of-the-art on many evaluation metrics especially with an accuracy of 85.91%.

Multimodal physiological-based emotion recognition is one of the most available but challenging studies due to complexity of emotions and individual differences in physiological signals. However, existing studies mainly combine multimodal data to fuse multimodal information in offline scenarios, ignoring data/modalities correlation among multimodal data and individual differences of non-stationary physiological signals in online scenarios. In this paper, we propose a novel Online Multimodal HyperGraph Learning (OMHGL) method to fuse multimodal information for emotion recognition based on time-series physiological signals. Our method consists of multimodal hypergraph fusion and online hypergraph learning. Specifically, the multimodal hypergraph fusion can fuse multimodal physiological signals to effectively obtain emotionally dependent information via leveraging multimodal information and higher-order correlations among multimodal data/modalities. The online hypergraph learning is designed to learn new information from online data by updating hypergraph projection. As a result, the proposed online emotion recognition model can be more effective for emotion recognition of target subjects when target data arrive in an online manner. Experimental results have demonstrated that the proposed method significantly outperforms the baselines and compared state-of-the-art methods in online emotion recognition tasks.

In generative adversarial network (GAN) based zero-shot learning (ZSL) approaches, the synthesized unseen visual features are inevitably prone to seen classes since the feature generator is merely trained on seen references, which causes the inconsistency between visual features and their corresponding semantic attributes. This visual-semantic inconsistency is primarily induced by the non-preserved semantic-relevant components and the non-rectified semantic-irrelevant low-level visual details. Existing generative models generally tackle the issue by aligning the distribution of the two modalities with an additional visual-to-semantic embedding, which tends to cause the hubness problem and ruin the diversity of visual modality. In this paper, we propose a novel generative model named learning modality-consistent latent representations GAN (LCR-GAN) to address the problem via embedding the visual features and their semantic attributes into a shared latent space. Specifically, to preserve the semantic-relevant components, the distributions of the two modalities are aligned by maximizing the mutual information between them. And to rectify the semantic-irrelevant visual details, the mutual information between original visual features and their latent representations is confined within an appropriate range. Meanwhile, the latent representations are decoded back to both modalities to further preserve the semantic-relevant components. Extensive evaluations on four public ZSL benchmarks validate the superiority of our method over other state-of-the-art methods.

Accurate and robust specific emitter identification (SEI) is very challenging since distribution shift of signals occurs in cross-session scenario. General domain adaptation (DA) is proposed to alleviate the shift by aligning different signal distributions. However, existing general-DA based SEI methods which focus on the shift in the same session cannot be directly applied to cross-session SEI, since the distribution of signals varies more drastically in different sessions due to the continuously changing hardware imperfections. In this paper, we propose a novel method named adversarial domain adaptation with wasserstein distance (ADAW) to tackle the cross-session SEI. Specifically, to alleviate the severer distribution shift of signals in different sessions, a generative model is applied to map the data of previous session to latter session regardless of the degree of radio frequency fingerprints (RFFs) variations. Then, a wasserstein distance guided adversarial unsupervised domain adaptation (UDA) strategy is introduced to learn common feature representations for signals of different sessions, such that the model trained on the signals of previous session can precisely identify the signals of latter session. Experiments on ADS-B signals of same emitters in three distinct time sessions validate the capability of ADAW for SEI under cross-session and noisy conditions.

Electrocardiogram (ECG) signal based cross-subject emotion recognition methods reduce the influence of individual differences using domain adaptation (DA) techniques. These methods generally assume that the entire unlabeled data of unknown target subjects are available in training phase. However, this assumption does not hold in some practical scenarios where the data of target subjects arrive one by one in an online manner instead of being acquired at a time. Thus, existing DA methods cannot be directly applied in this case since the unknown target data is inaccessible in training phase. To tackle the problem, we propose a novel online cross-subject ECG emotion recognition method leveraging hypergraph-based online transfer learning (HOTL). Specifically, the proposed hypergraph structure is capable of learning the high-order correlation among data, such that the recognition model trained on source subjects can be more effectively generalized to target subjects. Meanwhile, the structure can be easily updated by adding a hyperedge which connects a newly coming sample with the current hypergraph, resulting in further reduce the individual differences in online manner without re-training the model. Consequently, HOTL can effectively deal with the online cross-subject scenario where unknown target ECG data arrive one by one and varying overtime. Extensive experiments conducted on the Amigos dataset validate the superiority of the proposed method.

Recently, source-free domain adaptation (SFDA) has attracted extensive attention in medical image segmentation due to the ability of knowledge transfer without accessing source data. However, existing SFDA methods suffer from severe performance degradation since the style of the target data shifts from the source. Although traditional unsupervised domain adaptation (UDA) methods are capable of addressing the style shifts issue using both domain data, they fail to extract the source style due to a lack of source data in source-free scenarios. In this paper, we propose a novel style-insensitive source-free domain adaptation framework (SI-SFDA) for medical image segmentation to reduce the impacts of style shifts. The proposed framework first pretrains a generalized source model and then adapts the source model in a source data-free manner. Towards the former, a cross-patch style generalization (CPSG) mechanism is introduced to reduce the style sensitivity of the source model via a self-training paradigm with Transformer structure. Towards the latter, an adaptive confidence regularization (ACR) loss with dynamic scaling strategy is developed to further reduce the classification confusion caused by style shifts. The proposed ACR loss is model-independent so that it can be used with other methods to improve the segmentation performance. Extensive experiments are conducted on five public medical image benchmarks, the promising performance on organ and fundus segmentation tasks demonstrates the effectiveness of our framework.

In zero-shot learning (ZSL) tasks, especially in generalized zero-shot learning (GZSL), the model tends to classify unseen test samples into seen categories, which is well known as the domain shift problem, because the model is trained from seen samples without unseen samples. Recently, generative adversarial network (GAN) based methods have achieved good performance in GZSL, which replace real unseen features by synthesizing fake ones to mitigate the domain shift. However, the domain shift problem is still not well solved, due to the lacking of unseen samples in the training progress of the GAN generator. In this paper, we propose a generative model named discriminative learning GAN (DL-GAN) to alleviate the domain shift in GZSL. Specifically, the DL-GAN is designed with three novel components: a dual-stream embedding model that aligns features to the ground-truth attributes to extract discriminative latent attributes from features, an attribute-based generative model that generates high-quality unseen features from semantic attributes to guarantee inter-class discriminability and semantic consistency, and a seen/unseen classifier that leverages validation samples to distinguish seen samples from unseen ones. Experimental results on four widely used datasets verify that our proposed approach significantly outperforms the state-of-the-art methods under the GZSL protocol.

Heart rate (HR) monitoring using photoplethysmography (PPG) is a promising feature in modern wearable devices. PPG is easily contaminated by motion artifacts (MA), hindering estimation of HR. For quasi-periodic motions, previous works generally focused on a few specific motions, such as walking and fast running. However, they may not work well for many different quasi-periodic motions where MA are very complex. In this paper, a robust HR monitoring scheme for different quasi-periodic motions using wrist-type PPG is proposed, which consists of dictionary learning for signal characteristics learning, human motion recognition for the current motion recognition and dictionary selection, sparse representation-based MA elimination for denoising, and spectral peak tracking for HR-related spectral peak tracking. The proposed scheme is robust to MA caused by different motions and has high accuracy. Experiments on six common quasi-periodic motions showed that the average absolute error of heart rate estimation was 2.40 beat per minute, and also showed that the proposed method is more robust than some state-of-the-art approaches for different motions.

As a result of the limited available energy, radio frequency (RF)-powered devices must be capable of efficiently utilizing scarce energy by planning task execution according to the current harvested energy. However, the energy utilization efficiency is challenging to be improved in RF-powered devices, since sensing the harvested energy consumes a significant amount of energy that should be used for task execution. In this paper, we propose Sentinel, a novel low power method to sense the harvested energy. Sentinel is fully delegated to detect the energy for the device, while the device does not participate in the energy sensing. By this means, the computing overhead of the device is reduced. Sentinel works with low energy consumption, and functions as a trigger to activate the device when, and only when, the energy reaches an expected energy threshold. We also present a lightweight scheme to set the desired thresholds so that Sentinel achieves detecting any expected thresholds. We implement Sentinel by off-the-shelf components and conduct experiments to show that Sentinel consumes only 5.2% of energy overhead of the general energy sensing technique. With Sentinel, we show that the energy utilization efficiency can be improved up to 94.9%, outperforming the best existing works at 64.7% in the WISP platform.

Heart rate (HR) monitoring based on Photoplethys-mography (PPG) has drawn increasing attention in modern wearable devices due to its simple hardware implementation and low cost. In this work, we propose a variational mode decomposition(VMD)-based HR estimation method using wrist-type PPG signals during physical exercise. To remove motion artifacts (MA), VMD was first used and then a post-processing method after VMD was proposed to guarantee the robustness of MA removal. The performance of our proposed method was evaluated on two PPG datasets used in 2015 IEEE Signal Processing Cup. The method achieved the average absolute error of 1.45 beat per minute (BPM) on the 12 training sets and 3.19 BPM on the 10 testing sets, confirmed by the experimental results.

In this paper, we propose a flexible fully-multiplicative orthogonal-group based ICA (FlexibleOgICA) algorithm, which can instantaneously separate the mixture of sub-Gaussian and super-Gaussian source signals. It adopts a self-adaptive nonlinear function, which adjusts its parameter to achieve better performance based on the estimation of the kurtosis of super-Gaussian source signals. We also have successfully applied the algorithm to obtain the fetal electrocardiogram (FECG) signal, showing its fast convergence speed and high separation performance