Semi-supervised learning (SSL) has gained remarkable traction for its capacity to utilize extensive unlabeled datasets while relying on minimal labeled samples, thereby addressing the limitations of annotation scarcity. This study introduces an innovative framework, Feature-Enriched Mutual Distillation with Semi-Supervised Contrastive Learning (FMDSSCL), specifically designed for classifying Electroencephalogram (EEG) signals. The proposed approach integrates four foundational SSL strategies, i.e., pseudo-labeling, entropy minimization, generic regularization, and consistency regularization, into a cohesive framework augmented by feature-based mutual distillation and contrastive learning (CL), unified under a novel loss function. Feature-based mutual distillation enables cross-layer knowledge transfer, significantly improving feature extraction capabilities across model layers. Simultaneously, CL emphasizes identifying commonalities in representations across identical instances and supplementary data samples. To further enhance the exploration of EEG signal characteristics, the framework employs a fully convolutional network (FCN) comprising three convolutional blocks, effectively capturing both localized and global temporal patterns. The efficacy of FMDSSCL is demonstrated through comprehensive evaluations on four benchmark datasets, i.e., FingerMovements, MotorImagery, PenDigits, and SelfRegulationSCP, where it outperforms a variety of established semi-supervised and fully supervised approaches, solidifying its contribution to SSL research in EEG analysis.

Wearable human activity recognition (HAR) focuses on classifying human activities from multi-sensor data collected by wearable devices and has become increasingly important in pervasive computing. However, existing methods face several challenges: (1) aggregating heterogeneous local models while preserving user-specific data distributions, (2) achieving personalized adaptation of global models to diverse behavioral patterns, and (3) capturing both local and global temporal dependencies inherent in sensor time-series data. To address these challenges, we propose PFLMamba, a personalized federated learning framework integrating an attention-enhanced state-space model (ASSM) for hierarchical temporal feature extraction. PFLMamba employs a server-side personalized attention aggregation mechanism to tailor global models for individual clients, while ASSM captures both local and long-range temporal patterns on the client side. Extensive evaluations on the WISDM and PAMAP2 datasets demonstrate that PFLMamba achieves F 1 scores of 91.83 and 96.01, respectively, outperforming state-of-the-art federated learning baselines such as FCLFD, EFDLS, and FKD. PFLMamba’s effectiveness is further validated on multi-user and heterogeneous-sensor datasets, namely UCI-HAR and UNIMIB-SHAR, confirming its generalization across diverse populations and device types. Beyond predictive accuracy, PFLMamba exhibits a favorable trade-off between efficiency and performance, with lower client-side training overhead than teacher-student-based frameworks and competitive throughput relative to lightweight alternatives. Further experiments on resource-constrained edge devices (i.e., Raspberry Pi 4 and PYNQ-Z2) validate its practical feasibility, highlighting low-latency inference and moderate energy consumption. These results establish PFLMamba as a robust and efficient solution for personalized wearable HAR.

Multivariate time series classification (MTSC) requires a model capable of capturing both localized temporal patterns and long-range dependencies while effectively modeling complex inter-variable relationships. Existing convolutional neural network (CNN)-based capsule models suffer from limited receptive fields, constraining their ability to model long-range dependencies, while transformer-based capsule models rely solely on self-attention, which, despite its effectiveness in capturing global features, struggles with preserving local structures and efficiently processing long sequences. To overcome these limitations, we propose KACapMamba, a Knowledge-Aware State-Space Capsule Network, which integrates three attentive Mamba blocks with a routing layer to achieve hierarchical temporal modeling. Unlike conventional transformer-based methods that primarily depend on self-attention for global dependency modeling, each attentive Mamba block in KACapMamba fuses 1-dimensional CNNs, self-attention, state-space module (SSM), and mutual cross-attention, enabling a more structured and adaptive feature representation. Self-attention ensures effective long-range dependency modeling, while SSM provides a recurrent-state mechanism, inherently better suited for sequential processing compared to purely attention-based architectures, thereby enhancing temporal continuity and long-term pattern retention. Notably, mutual cross-attention addresses the limitations of traditional fusion strategies such as element-wise addition or multiplication, which lack the capacity to selectively enhance relevant features. By dynamically reweighting interactions between features, mutual cross-attention enables more expressive, context-aware representations, leading to improved feature disentanglement and inter-variable modeling. Additionally, the routing layer further enhances hierarchical feature disentanglement by refining capsule activations, reinforcing structural coherence and feature selectivity. Experiments conducted across the UEA benchmark archive demonstrate that KACapMamba consistently achieves the highest ‘win’/‘tie’/‘lose’/‘best’ ratios when evaluated against 10 leading transformer and Mamba architectures under both$Accuracy$and$F_{1}$metrics. Moreover, in comparison with 22 state-of-the-art MTSC models, it again secures the most favorable performance profile, demonstrating a clear and statistically supported advantage across both evaluation measures.

Numerous existing deep learning models for time-series classification (TSC) tend to overlook the intricate interplay between higher-and lower-level semantic information. While the focus is often on extracting higher-level semantics from lower-level sources, the reciprocal influence of lower-level information on higher levels is undervalued. To address this, we propose an ensemble transitive bidirectional decoupled self-distillation (ETBiDecSD) method for TSC. ETBiDecSD enhances the robustness of higher-level semantic information using an average feature ensemble (AFE) method to amalgamate the output from each level. Simultaneously, the integrated features are transmitted to each lower level through a directional decoupled distillation (DD) structure. Additionally, to promote deep interaction between higher-and lower-level semantic information, ETBiDecSD introduces a transitive bidirectional DD (TBDD) structure, facilitating the transfer of target-class and nontarget-class knowledge between higher and lower levels. Experimental results demonstrate that whether a fully convolutional network (FCN) with four convolutional blocks or InceptionTime with four Inception blocks is used as the baseline, ETBiDecSD outperforms a quantity of well-established self-distillation algorithms across 85 widely used UCR2018 datasets, as evidenced by the metrics “win”/“tie”/“lose” and avg. rank, which are derived from accuracy andF1-scores. Notably, when compared to a nonself-distillation FCN, ETBiDecSD achieves “win”/“tie”/“lose” results of 64/4/17 in terms of accuracy and 65/4/16 in terms ofF1-score. Similarly, in comparison to a nonself-distillation InceptionTime, ETBiDecSD attains “win”/“tie”/“lose” results of 60/12/13 for accuracy and 57/12/16 forF1-score.

Federated learning (FL) enables training machine learning models across distributed devices while preserving data privacy. However, the growing demand for large-scale model training raised by data-driven intelligent applications faces critical bottlenecks in traditional FL frameworks, including communication overhead and latency, edge heterogeneity, and inefficient resource utilization. To address these challenges, we propose a cloud-edge-end collaborative federated learning (CECFL) algorithm to empower the larger model training. CECFL introduces a dual-phase hierarchical aggregation mechanism, including synchronous end-edge aggregation to harmonize localized model updates from massive resource-constrained devices, and semi-asynchronous edge-cloud aggregation to mitigate stragglers and communication delays across heterogeneous tiers. Furthermore, we design a deep reinforcement learning based adaptive orchestration framework that dynamically optimizes end-edge associations and edge participation rates, ensuring efficient resource allocation. Extensive experiments are carried out to demonstrate the high effectiveness of the CECFL in terms of improving the model accuracy and the system efficiency.

Multi-modal semantic communication (MSC) has gained great attention due to its multi-modal processing ability. However, the existing MSC systems are mainly built on multi-modal large models that lead to inefficient computation on non-essential tokens, potentially restricting MSC from achieving more advanced levels of intelligence. To address this challenge, we propose a large model-empowered MSC system with a cross-modal attention-based token selection mechanism, denoted as LMECM-SC, which effectively utilizes the attention score across multi-modal tokens to filter out noisy or unuseful tokens, selectively learning the tokens that best benefit downstream applications. Meanwhile, we introduce the multi-modal adaptive semantic encoder and decoder that dynamically assign weights to encode multi-modal semantic information extracted from the selected tokens based on their modality and integrate semantic information with cross-modal attention scores at the receiver, optimizing the performance on downstream tasks. Experiment results indicate that LMECM-SC effectively reduces the number of tokens used for training, outperforming four baseline methods in terms of bilingual evaluation understudy score for text, learned perceptual image patch similarity for image, and perceptual evaluation of speech quality score for speech.

Over the years, various sophisticated deep learning algorithms have surfaced for multivariate time series classification (MTSC), notably the dual-network-based model. This model comprises two parallel networks tailored to time series data: one for local feature extraction and the other for global relation extraction. However, effectively integrating these dual networks poses a significant challenge. To address this, we propose a knowledge aggregation transformer network (KATN) for MTSC. KATN, composed of four aggregation transformer blocks, extracts abundant regularizations and connections hidden within the data. Each block incorporates a modified residual network (MResNet) for local feature extraction and a multi-head attention network for global relation extraction. Initially, the block merges MResNet's output feature with that of the multi-head attention network through an additive operation. Subsequently, it aligns features with a fully connected (i.e., dense) layer and activates neural units using the Gaussian error linear unit function. This strategic feature aggregation allows for capturing long-range dependencies among multiple variables in multivariate time series data. Experimental results demonstrate that KATN significantly outperforms 6 state-of-the-art transformer variants, achieving a ‘win’/‘tie’/‘lose’ record of 9/6/15 and securing the lowest AVG_rank score. Furthermore, when evaluated against 18 existing MTSC algorithms across 13 UEA datasets, KATN consistently delivers superior performance, attaining the lowest AVG_rank score among all compared methods.

This article proposes a federated contrastive learning with feature-based distillation (FCLFD) framework tailored for human activity recognition (HAR). The FCLFD system integrates a central server with multiple mobile users to address a diverse range of HAR challenges. The framework encompasses two pivotal elements: a contrastive student--teacher (CST) architecture with feature-based distillation and an average weight scheme (AWS). The CST framework facilitates the transfer of comprehensive knowledge from a teacher model to a student model through feature-based distillation and contrastive learning, with both models sharing an identical architecture. Each participating user periodically uploads the weights of its student model to the central server, where the AWS deployed on the server calculates the average weights based on contributions from all connected users. The aggregated weights are then redistributed to each user, who updates their teacher model accordingly. Experimental evaluations demonstrate that when 50 users are connected, the proposed FCLFD scheme obtains the highestF1values of 89.01 and 94.19, outperforming several state-of-the-art federated learning algorithms on the wireless sensor data mining (WISDM) and PAMAP2 datasets.

The unlawful use of noncooperative drones, or unmanned aerial vehicles (UAVs), poses serious threats to public safety and societal security, necessitating robust monitoring solutions. However, the detection of drones, particularly those flying remotely, is often hindered by the limited accuracy in identifying small targets. Additionally, challenges related to insufficient lightweight design and complexities in practical deployment further exacerbate the issue. To address these challenges, we propose a quad-head network, QHNet, designed to provide scalable, adaptable, and efficient drone detection across diverse scenarios. QHNet is available in five scalable model sizes—Nano (N), Small (S), Medium (M), Large (L), and Extra Large (X)—ensuring real-time detection tailored to varying resource constraints and operational requirements. The core of QHNet lies in its innovative quad detection head (QDH), which introduces an additional detection layer to perform secondary feature extraction for small targets, enhancing detection precision. Furthermore, the architecture integrates adaptive spatial feature fusion to improve scale invariance and overall detection accuracy. To further optimize performance, QHNet incorporates a four-scale feature fusion network (FSF) and a specialized small target IoU loss function (STIoU) to refine detection precision. Simultaneously, the lightweight coarse-to-fine processing unit (LCF) and SCDown downsampling (SCD) form a comprehensive lightweight scheme, significantly reducing computational overhead while maintaining high detection efficacy. Extensive experiments were conducted on DUT-Plus, an augmented version of the DUT Anti-UAV dataset enhanced through data augmentation. The experimental results confirm that QHNet attains outstanding performance, providing an advanced balance between detection accuracy and computational efficiency across all model sizes.

This article proposes a semi-supervised contrastive capsule transformer method with feature-based knowledge distillation (KD) that simplifies the existing semisupervised learning (SSL) techniques for wearable human activity recognition (HAR), called CapMatch. CapMatch gracefully hybridizes supervised learning and unsupervised learning to extract rich representations from input data. In unsupervised learning, CapMatch leverages the pseudolabeling, contrastive learning (CL), and feature-based KD techniques to construct similarity learning on lower and higher level semantic information extracted from two augmentation versions of the data, "weak" and "timecut," to recognize the relationships among the obtained features of classes in the unlabeled data. CapMatch combines the outputs of the weak- and timecut-augmented models to form pseudolabeling and thus CL. Meanwhile, CapMatch uses the feature-based KD to transfer knowledge from the intermediate layers of the weak-augmented model to those of the timecut-augmented model. To effectively capture both local and global patterns of HAR data, we design a capsule transformer network consisting of four capsule-based transformer blocks and one routing layer. Experimental results show that compared with a number of state-of-the-art semi-supervised and supervised algorithms, the proposed CapMatch achieves decent performance on three commonly used HAR datasets, namely, HAPT, WISDM, and UCI_HAR. With only 10% of data labeled, CapMatch achieves values of higher than 85.00% on these datasets, outperforming 14 semi-supervised algorithms. When the proportion of labeled data reaches 30%, CapMatch obtains values of no lower than 88.00% on the datasets above, which is better than several classical supervised algorithms, e.g., decision tree and -nearest neighbor (KNN).

Recently, numerous deep learning algorithms have addressed wearable human activity recognition (HAR), but they often struggle with efficient knowledge transfer to lightweight models for mobile devices. Knowledge distillation (KD) is a popular technique for model compression, transferring knowledge from a complex teacher to a compact student. Most existing KD algorithms consider homogeneous architectures, hindering performance in heterogeneous setups. This is an under-explored area in wearable HAR. To bridge this gap, we propose a heterogeneous mutual KD (HMKD) framework for wearable HAR. HMKD establishes mutual learning within the intermediate and output layers of both teacher and student models. To accommodate substantial structural differences between teacher and student, we employ a weighted ensemble feature approach to merge the features from their intermediate layers, enhancing knowledge exchange within them. Experimental results on the HAPT, WISDM, and UCI_HAR datasets show HMKD outperforms ten state-of-the-art KD algorithms in terms of classification accuracy. Notably, with ResNetLSTMaN as the teacher and MLP as the student, HMKD increases by 9.19% in MLP's $F_{1}$ score on the HAPT dataset.

This paper studies a distributed semantic communication (SC) problem for multivariate time series forecasting tasks in edge environments, with heterogeneous clients considered. At the client side, a semantic encoder is composed of a number of federated blocks and this number is subject to local resource availability. Each federated block consists of a patch-wise attention module (PAM) and a federated adapter, extracting semantic information for efficient transmission across wireless channels. Based on the federated adapters, this paper proposes an SC-oriented heterogeneous federated learning architecture, named SC-FedAda. SC-FedAda adopts self-distillation to facilitate cross-client and cross-layer knowledge sharing, enabling efficient collaborative inference. At the edge server, semantic signals are fed into a channel decoder and then a semantic decoder. The semantic decoder consists of a PAM and a fully connected network for forecasting tasks. Simulation results demonstrate that SC-FedAda outperforms four state-of-the-art federated learning-based structures under three types of wireless channels, i.e. SC-FedAda achieves much lower forecasting loss on three widely-used time series datasets, particularly in low signal-to-noise ratio scenarios.

Stream processing jobs often require elastic scaling in response to dynamically changing loads. In recent years, the development of serverless technology has provided new potential for elastic scaling of stream processing jobs. However, directly implementing stream processing jobs to serverless platforms faces many challenges such as program complexity, high latency and load imbalance. In this paper, we propose an efficient stream processing system under serverless environment. Our system provides a high-level programming interface and can automatically determine the parallelism under the current load through a non-linear model. Additionally, it reduces the latency and increases the throughput of stream processing jobs through decentralized orchestration and soft affinity scheduling policies. We conducted a series of experiments to evaluate the performance of our system. The experimental results show that our system effectively improves the throughput and reduces the latency of stream processing jobs under serverless environment compared with the widely-used scheduling and orchestration methods used in serverless environment.

Network function virtualization (NFV) enables the provision of different quality of service (QoS) levels through service function chains (SFCs), where NFV outsources big data tasks of end users to nearby edge servers. In multi-access edge computing (MEC), its dynamic and uncertainty nature poses great challenges to the SFC placement problem, which requires optimizing multiple potentially-conflicting objectives, such as network latency and load balancing. Moreover, user preferences may vary along with time, adding another layer of complexity to the problem. To address the problem above, we propose a novel distributed deep reinforcement learning (DRL) architecture based on a spatio-temporal encoder (STE), denoted as DDRL-STE. DDRL-STE is featured with equal-weight pre-training and transformer-based STE. Experimental results show that DDRL-STE outperforms three state-of-the-art DRL algorithms regarding latency and load balancing under three well-known network topologies, exhibiting its excellent potential in exploration and generalization.

System logs contain valuable information and they have emerged as one of the most crucial data sources for system monitoring aimed at enhancing service quality. IT support teams and system administrators are in dire need of an intelligent log-based QA system to help them quickly identify, diagnose, and resolve issues. In this paper, we propose a novel method for constructing log-based question-answering (QA) data using large language models, addressing challenges associated with limited dataset size and diversity in existing log-based QA systems. Our pipeline consists of three steps: generating questions, answering log questions, and refining question-answer pairs. The purpose of the generating questions is to create a diverse set of log-related queries that cover a wide range of potential issues. The second step, answering log questions, aims to extract relevant information from the logs to address the generated questions. This step ensures accurate and context-aware responses. Refining question-answer pairs is intended to improve the overall quality and consistency of the generated log-based QA data. We present a case study using ChatGPT to generate a new dataset, LogQuAD, containing over 28,000 question-answer pairs derived from more than 31,000 raw logs, representing a significant increase compared to existing datasets like LogQA. In our experimental setting, we sample half of the data as the training set and use memory-effect fine-tuning to fine-tune the model, named Gloss. Experimental results show that our method can generate high-quality log-based QA data, leading to improved performance of log-based QA models. Notably, our fine-tuned 7B model outperforms the LLaMA-65B model. This approach can potentially save valuable time for IT support teams and system administrators, enabling proactive problem resolution and optimal system performance.

Multi-user semantic communication (MUSC) has emerged as a promising paradigm for future 6G networks and applications, where massive clients (e.g., mobile devices) collaboratively construct a global semantic decoder without sharing their local data. However, due to the lack of direct access to clients’ data, MUSC is vulnerable to data poisoning attacks (DPAs), wherein malicious participants send updates derived from poisoned training samples. Current defense techniques against DPAs are designed for traditional networks and are not directly applicable to MUSC. In this paper, we propose an effective attack-defense game framework, denoted as DPAD-MUSC, tailored to defend against DPAs during image transmission for MUSC. First, we determine each attack-type's optimal attack policy based on reinforcement learning, with the aim of strengthening the attack while avoiding detection. To generate adversarial samples accordingly, we devise an adversarial samples generator (ADV-Generator) based on conditional generative adversarial network (CGAN). Then, we introduce an attack defender (DPA-Defender) to detect data poisoning attacks and exclude poisoned samples from the target model's learning process, with the adversarial samples generated under the guidance of the optimal attack policy to enhance the detector's robustness. Simulation results demonstrate that the DPAD-MUSC can find optimal attack policies that cause a greater accuracy drop in the target model while maintaining a higher evasion rate. The ADV-Generator can generate effective adversarial samples and the DPA-Defender outperforms five state-of-the-art methods on three widely used image datasets under additive white Gaussian noise (AWGN) channel in terms of Top-1 accuracy.

This paper investigates the problem of energy-efficient trajectory optimization with wireless charging (ETWC) in an unmanned aerial vehicle (UAV)-assisted mobile edge computing system. A UAV is dispatched to collect computation tasks from specific ground smart devices (GSDs) within its coverage while transmitting energy to the other GSDs. In addition, a high-altitude platform with a laser beam is deployed in the stratosphere to charge the UAV, so as to maintain its flight mission. The ETWC problem is characterized by multi-objective optimization, aiming to maximize both the energy efficiency of the UAV and the number of tasks collected via optimizing the UAV's flight trajectories. The conflict between the two objectives in the problem makes it quite challenging. Recently, some single-objective reinforcement learning (SORL) algorithms have been introduced to address the aforementioned problem. Nevertheless, these SORLs adopt linear scalarization to define the user utility, thus ignoring the conflict between objectives. Furthermore, in dynamic MEC scenarios, the relative importance assigned to each objective may vary over time, posing significant challenges for conventional SORLs. To solve the challenge, we first build a multi-objective Markov decision process that has a vectorial reward mechanism. There is a corresponding relationship between each component of the reward and one of the two objectives. Then, we propose a new trace-based experience replay scheme to modify sample efficiency and reduce replay buffer bias, resulting in a modified multi-objective reinforcement learning algorithm. The experiment results validate that the proposed algorithm can obtain better adaptability to dynamic preferences and a more favorable balance between objectives compared with several algorithms.

Time series data widely exist in public services, industrial environments, and military applications. Traditionally, the transmission of a huge volume of data for analytic tasks poses challenges, particularly in mobile environments with limited computing and communication resources. Semantic communication emerges as a solution for intelligently extracting various features from source data and efficiently transmitting task-related information to receivers, thereby reducing bandwidth consumption significantly. In this paper, we introduce a novel federated semantic communication system tailored for forecasting-oriented time series transmission tasks. The correlation of source data collected from terminal devices is mined and the corresponding semantic information is transmitted to an edge server for collaborative inference. To optimize the semantic analysis process, we devise a deep decomposition block at the transmitter side, decomposing time series into trend and multiple period components. This reduces noise interference from wireless channels, enhancing the overall transmission quality. For effective training and collaborative inference, we propose a Federated Mixture of period Routers (FedMoR) architecture. Within each channel encoder, period routers are divided into private and public ones. Private routers extract specialized features from individually collected data, mitigating accuracy degradation. Public routers share knowledge across all transmitters, enhancing temporal analysis robustness. Simulation results demonstrate that the proposed system outperforms two traditional technique-based and two semantic communication-based baselines under three common channels. The system achieves low mean square errors on five widely-used real-world time series forecasting datasets, particularly in the low signal-to-noise ratio regime.

Multivariate time series classification (MTSC) based on deep learning (DL) has attracted increasingly more research attention. The performance of a DL-based MTSC algorithm is heavily dependent on the quality of the learned representations providing semantic information for downstream tasks, e.g., classification. Hence, a model’s representation learning ability is critical for enhancing its performance. This article proposes a densely knowledge-aware network (DKN) for MTSC. The DKN’s feature extractor consists of a residual multihead convolutional network (ResMulti) and a transformer-based network (Trans), called ResMulti-Trans. ResMulti has five residual multihead blocks for capturing the local patterns of data while Trans has three transformer blocks for extracting the global patterns of data. Besides, to enable dense mutual supervision between lower-and higher-level semantic information, this article adapts densely dual self-distillation (DDSD) for mining rich regularizations and relationships hidden in the data. Experimental results show that compared with 5 state-of-the-art self-distillation variants, the proposed DDSD obtains 13/4/13 in terms of “win”/“tie”/“lose” and gains the lowest-AVG_rank score. In particular, compared with pure ResMulti-Trans, DKN results in 20/1/9 regarding win/tie/lose. Last but not least, DKN overweighs 18 existing MTSC algorithms on 10 UEA2018 datasets and achieves the lowest-AVG_rank score.

Molecular dynamics is a widely adopted simulation method for analyzing the movement of atoms and molecules. Traditional molecular dynamics simulation methods are computationally intensive and difficult to simulate a large number of atoms. In contrast, molecular dynamics based on deep potential models such as DeePMD can leverage deep learning techniques to improve simulation efficiency. Although DeePMD has incorporated mainstream deep learning frameworks, it still suffers from low performance and efficiency during its model training on heterogeneous systems such as CPU and GPU. Particularly, a large number of operators cannot be accelerated by GPU, resulting in low utilization of GPU computational resources. In this paper, we comprehensively analyze the computational bottlenecks and the corresponding root causes of DeePMD. We correspondingly propose several novel optimization strategies. Specifically, for preprocessing, we identify the computation redundancies and the GPU parallelization opportunities for performance optimization. For training, we propose optimization strategies such as operator fusion, redundancy elimination, and concurrent execution of multiple streams and threads in the computation process. Moreover, we apply systematical optimization of computational graphs and operators. The evaluation results show that DeePMD can achieve significant speedups in several cases after applying our proposed optimizations, resulting in a maximum overall speedup of 6.36× with acceptable accuracy.

Big data applications are widely adopted to mine valuable information from a tremendous amount of industry data, which is commonly represented as a series of map-reduce operations. Among various map-reduce frameworks, Hadoop is most commonly adopted for data processing at large scale. Although Hadoop eases the development of highly scalable distributed big data applications, inefficient implementation due to poor coding practice and deep software abstractions can cause severe performance issues such as unreasonable slowdown, high response latency, and waste of computing resources, which can lead to unsatisfactory serving delay or significant maintenance cost. In this paper, we first categorize three common types of redundant patterns in big data applications. Then we propose a tool-assisted optimization workflow to detect the redundant patterns automatically, which profiles the application by sampling hardware performance monitoring units. Moreover, we present a profiling visualization method that can help to pinpoint the redundant codes. Based on these approaches, we optimize several big data applications by eliminating redundancies, yielding up to 14.8% performance improvement.

This paper proposes a task-oriented semantic communication system for electrocardiogram (ECG) signal classification, called ECG-SC-DARTS. Based on deep learning, this system adopts the differentiable neural architecture search (DARTS) to automatically design the neural architecture of the semantic encoder under various channels. This paper improves the performance of the original DARTS by introducing a new recurrent neural network (RNN) cell with residual structure and a noise adding scheme for skip-connections. The RNN cell enhances the temporal semantic information extraction ability while the added noise reduces the risk of performance collapse caused by skip-connections. Experimental results demonstrate that ECGSC-DARTS generates appropriate neural architectures for the semantic encoder under AWGN, Rayleigh and Rician channels and these architectures outperform a number of baseline models, such as the original DARTS, fully convolutional network, multi-layer perception, and ResNet, regarding F1-score. Moreover, ECGSC-DARTS is more reliable than the traditional communication system in harsh channel environment.

We present a many-to-one distributed semantic communication system for multivariate time series classification. The system adopts a federated learning-based architecture to achieve low-redundancy collaborative inference, where an unsupervised auxiliary task is designed to coordinate the feature vectors at different dimensions between semantic encoders and the classifier. For each transmitter, we design a scale-adaptive semantic encoder by applying weighted sum to a number of predefined convolutional layers. The scale-adaptive semantic encoder can extract multi-scale features from time series following various distributions. A dynamic channel encoder is developed to adapt to the scale-adaptive semantic encoder, converting semantic features to complex symbols appropriate for wireless transmission. For the receiver, we apply the same scale-adaptive structure to the semantic decoder to extract multi-scale semantic features from all transmitters for accurate classification. Simulation results show that the proposed distributed semantic communication system outperforms two baseline systems under AWGN, Rician, and Rayleigh channels and achieves excellent Top-1 accuracy performance on three UEA2018 datasets, especially when the signal-to-noise ratio is low.

This paper studies the trajectory control and task offloading (TCTO) problem in an unmanned aerial vehicle (UAV)-assisted mobile edge computing system, where a UAV flies along a planned trajectory to collect computation tasks from smart devices (SDs). We consider a scenario that SDs are not directly connected by the base station (BS) and the UAV has two roles to play: MEC server or wireless relay. The UAV makes task offloading decisions online, in which the collected tasks can be executed locally on the UAV or offloaded to the BS for remote processing. The TCTO problem involves multi-objective optimization as its objectives are to minimize the task delay and the UAV's energy consumption, and maximize the number of tasks collected by the UAV, simultaneously. This problem is challenging because the three objectives conflict with each other. The existing reinforcement learning (RL) algorithms, either single-objective RLs or single-policy multi-objective RLs, cannot well address the problem since they cannot output multiple policies for various preferences (i.e. weights) across objectives in a single run. An evolutionary multi-objective RL (EMORL) algorithm is applied to address the TCTO problem. We improve the multi-task multi-objective proximal policy optimization of the original EMORL by retaining all new learning tasks in the offspring population, which can preserve promissing learning tasks. The simulation results demonstrate that the proposed algorithm can obtain more excellent non-dominated policies by striking a balance between the three objectives regarding policy quality, compared with two evolutionary algorithms, two multi-policy RL algorithms, and the original EMORL.

Over the years, several deep learning algorithms have been proposed for electroencephalography (EEG) signal classification. The performance of any learning method usually relies on the quality of the learned representation that provides semantic information for downstream tasks such as classification. Thus, it is crucial to improve the model’s representation learning capability. This paper proposes an Efficient Temporal Network with dual self-distillation for EEG signal classification, ETNEEG. It enhances the model’s representation learning by promoting mutual learning between higher-level and lower-level semantic information. The proposed ETNEEG consists of two main components: a parallel dual-network-based feature extractor called MLN-GRN and a dual self-distillation module. MLN-GRN includes a multi-scale local network (MLN) and a global relation network (GRN). MLN pays attention to local features of EEG data, and GRN is designed for learning global patterns of EEG data. Meanwhile, the dual self-distillation module extracts semantic information by mutual learning among the output layer and the low-level features. To evaluate the proposed method’s performance, seven widely used public EEG datasets, i.e., FaceDetection, FingerMovements, HandMovementDirection, MotorImagery, PenDigits, SelfRegulationSCP1, and SelfRegulationSCP2, are applied to a set of experiments. Experimental results demonstrate that the proposed ETNEEG achieves excellent performance on these datasets compared with fourteen existing algorithms.

Over the years, a number of semisupervised deep-learning algorithms have been proposed for time-series classification (TSC). In semisupervised deep learning, from the point of view of representation hierarchy, semantic information extracted from lower levels is the basis of that extracted from higher levels. The authors wonder if high-level semantic information extracted is also helpful for capturing low-level semantic information. This paper studies this problem and proposes a robust semisupervised model with self-distillation (SD) that simplifies existing semisupervised learning (SSL) techniques for TSC, called SelfMatch. SelfMatch hybridizes supervised learning, unsupervised learning, and SD. In unsupervised learning, SelfMatch applies pseudolabeling to feature extraction on labeled data. A weakly augmented sequence is used as a target to guide the prediction of a Timecut-augmented version of the same sequence. SD promotes the knowledge flow from higher to lower levels, guiding the extraction of low-level semantic information. This paper designs a feature extractor for TSC, called ResNet–LSTMaN, responsible for feature and relation extraction. The experimental results show that SelfMatch achieves excellent SSL performance on 35 widely adopted UCR2018 data sets, compared with a number of state-of-the-art semisupervised and supervised algorithms.

Building explainable model has become an important issue for credit risk assessment. Results can be presented as rule-based knowledge and are therefore considered interpretable because they indicate causation of classification. However, traditional knowledge extraction methods are not suited to finance because credit data involves discrete and continuous data, with missing values and imbalanced labels. In this study, a novel multi-label classification method is proposed, which summarizes high-dimensional structured data observations into a rule-based classifier in the form of a rule list. The solution is a novel hybrid evolutionary algorithm (hEA) which avoids preprocessing original data from complex credit datasets. The proposed method has significant advantages over established interpretable classification methods in terms of classification performance on complex credit data. The complete code of our proposed method are available at https://github.com/wenge963/The-RATP-method.

Over the years, a large number of deep learning algorithms have been developed for time series classification (TSC). These algorithms were usually invented by researchers with prior knowledge and experience. However, it is a critical challenge for beginners to design decent structures to address various TSC problems. To this end, we propose a robust neural temporal search (RNTS) framework for identifying the relationships and features in TSC data, which mainly contains a temporal search network and an attentional LSTM network. To be specific, inspired by the idea of neural architecture search (NAS), the temporal search network automatically transforms its structure for each dataset according to its characteristics, responsible for extracting basic features. The attentional LSTM network is used to explore the complex shapelets and relationships the former may ignore. Experimental results demonstrate that RNTS achieves the best overall performance on 24 standard datasets selected from the UCR 2018 archive, in terms of three measures based on the top-l accuracy, compared with a number of state-of-the-art approaches.

This article studies the joint optimization problem of computation offloading and resource allocation (JCORA) in mobile-edge computing (MEC). Deep reinforcement learning (DRL) is one of the ideal techniques for addressing the dynamic JCORA problem. However, it is still challenging to adapt traditional DRL methods for the problem since they usually lead to slow and unstable convergence in model training. To this end, we propose a temporal attentional deterministic policy gradient (TADPG) to tackle JCORA. Based on the deep deterministic policy gradient (DDPG), TADPG has two significant features. First, a temporal feature extraction network consisting of a 1-D convolution (Conv1D) residual block and an attentional long short-term memory (LSTM) network is designed, which is beneficial to high-quality state representation and function approximation. Second, a rank-based prioritized experience replay (rPER) method is devised to accelerate and stabilize the convergence of model training. Experimental results demonstrate that the decentralized TADPG-based mechanism can achieve more efficient JCORA performance than the centralized one, and the proposed TADPG outperforms a number of state-of-the-art DRL agents in terms of the task completion time and energy consumption.

Fraud detection technology is an important method to ensure financial security. It is necessary to develop explainable fraud detection methods to express significant causality for participants in the transaction. The main contribution of our work is to propose an explainable classification method in the framework of multiple instance learning (MIL), which incorporates the AP clustering method in the self-training LSTM model to obtain a clear explanation. Based on a real-world dataset and a simulated dataset, we conducted two comparative studies to evaluate the effectiveness of the proposed method. Experimental results show that our proposed method achieves the similar predictive performance as the state-of-art method, while our method can generate clear causal explanations for a few labeled time series data. The significance of the research work is that financial institutions can use this method to efficiently identify fraudulent behaviors and easily give reasons for rejecting transactions so as to reduce fraud losses and management costs.

Dynamic routing in software-defined networking (SDN) can be viewed as a centralized decision-making problem. Most of the existing deep reinforcement learning (DRL) agents can address it, thanks to the deep neural network (DNN) incorporated. However, fully-connected feed-forward neural network (FFNN) is usually adopted, where spatial correlation and temporal variation of traffic flows are ignored. This drawback usually leads to significantly high computational complexity due to large number of training parameters. To overcome this problem, we propose a novel model-free framework for dynamic routing in SDN, which is referred to as spatio-temporal deterministic policy gradient (STDPG) agent. Both the actor and critic networks are based on identical DNN structure, where a combination of convolutional neural network (CNN) and long short-term memory network (LSTM) with temporal attention mechanism, CNN-LSTM-TAM, is devised. By efficiently exploiting spatial and temporal features, CNN-LSTM-TAM helps the STDPG agent learn better from the experience transitions. Furthermore, we employ the prioritized experience replay (PER) method to accelerate the convergence of model training. The experimental results show that STDPG can automatically adapt for current network environment and achieve robust convergence. Compared with a number state-of the-art DRL agents, STDPG achieves better routing solutions in terms of the average end-to-end delay.

Together with the fast advancement of the Internet of Things (IoT), smart healthcare applications and systems are equipped with increasingly more wearable sensors and mobile devices. These sensors are used not only to collect data but also, and more importantly, to assist in daily activity tracking and analyzing of their users. Various human activity recognition (HAR) approaches are used to enhance such tracking. Most of the existing HAR methods depend on exploratory case-based shallow feature learning architectures, which struggle with correct activity recognition when put into real-life practice. To tackle this problem, we propose a novel approach that utilizes the convolutional neural networks (CNNs) and the attention mechanism for HAR. In the presented method, the activity recognition accuracy is improved by incorporating attention into multihead CNNs for better feature extraction and selection. Proof of concept experiments are conducted on a publicly available data set from wireless sensor data mining (WISDM) lab. The results demonstrate a higher accuracy of our proposed approach in comparison with the current methods.