In the scenario of class-incremental learning (CIL), deep neural networks have to adapt their model parameters to non-stationary data distributions, e.g., the emergence of new classes over time. To mitigate the catastrophic forgetting phenomenon, typical CIL methods either cumulatively store exemplars of old classes for retraining model parameters from scratch or progressively expand model size as new classes arrive, which, however, compromises their practical value due to little attention paid toparameter efficiency. In this paper, we contribute a novel solution, effective control of the parameters of a well-trained model, by the synergy between two complementary learning subnetworks. Specifically, we integrate one plastic feature extractor and one analytical feed-forward classifier into a unified framework amenable to streaming data. In each CIL session, it achieves non-overwritten parameter updates in a cost-effective manner, neither revisiting old task data nor extending previously learned networks; Instead, it accommodates new tasks by attaching a tiny set of declarative parameters to its backbone, in which only one matrix per task or one vector per class is kept for knowledge retention. Experimental results on a variety of task sequences demonstrate that our method achieves competitive results against state-of-the-art CIL approaches, especially in accuracy gain, knowledge transfer, training efficiency, and task-order robustness. Furthermore, a graceful forgetting implementation on previously learned trivial tasks is empirically investigated to make its non-growing backbone (i.e., a model with limited network capacity) suffice to train on more incoming tasks.

With the exponential growth of mobile applications, Android systems have become a significant source of big data which provides both vast opportunities and substantial privacy challenges. This makes it essential to adopt secure learning approaches like Federated Learning (FL). FL is a decentralized approach that trains models across distributed data without centralizing sensitive information. However, FL still faces security threats in the scope of big data, where the volume and variety of data increase the risks of sophisticated attacks such as the LOKI attacks. This attack exploits shared model updates in FL to infer and leak sensitive data, even in a decentralized setup. In this paper, we simulate the LOKI attacks within an FL environment using a real-world Android malware detection dataset characterized by dynamic analysis features. We propose a defense mechanism that combines differential privacy and anomaly detection to reduce the impact of LOKI attacks. While this mechanism is designed for mobile systems, where the large volume of data generated by numerous applications mirrors the complexities of big data environments, this approach is adaptable and can be applied to other big data contexts. Through extensive experiments, we demonstrate the effectiveness of the proposed mechanism in enhancing data privacy and securing FL for applications where big data privacy is foremost.

Real-world environments are inherently non-stationary, frequently introducing new classes over time. This is especially common in time series classification, such as the emergence of new disease classification in healthcare or the addition of new activities in human activity recognition. In such cases, a learning system is required to assimilate novel classes effectively while avoiding catastrophic forgetting of the old ones, which gives rise to the Class-incremental Learning (CIL) problem. However, despite the encouraging progress in the image and language domains, CIL for time series data remains relatively understudied. Existing studies suffer from inconsistent experimental designs, necessitating a comprehensive evaluation and benchmarking of methods across a wide range of datasets. To this end, we first present an overview of the Time Series Class-incremental Learning (TSCIL) problem, highlight its unique challenges, and cover the advanced methodologies. Further, based on standardized settings, we develop a unified experimental framework that supports the rapid development of new algorithms, easy integration of new datasets, and standardization of the evaluation process. Using this framework, we conduct a comprehensive evaluation of various generic and time-series-specific CIL methods in both standard and privacy-sensitive scenarios. Our extensive experiments not only provide a standard baseline to support future research but also shed light on the impact of various design factors such as normalization layers or memory budget thresholds. Codes are available at https://github.com/zqiao11/TSCIL.

Transformer architectures have been applied to graph-specific data such as protein structure and shopper lists, and they perform accurately on graph/node classification and prediction tasks. Researchers have proved that the attention matrix in Transformers has low-rank properties, and the self-attention plays a scoring role in the aggregation function of the Transformers. However, it can not solve the issues such as heterophily and over-smoothing. The low-rank properties and the limitations of Transformers inspire this work to propose a Global Representation (GR) based attention mechanism to alleviate the two heterophily and over-smoothing issues. First, this GR-based model integrates geometric information of the nodes of interest that conveys the structural properties of the graph. Unlike a typical Transformer where a node feature forms a Key, we propose to use GR to construct the Key, which discovers the relation between the nodes and the structural representation of the graph. Next, we present various compositions of GR emanating from nodes of interest and α-hop neighbors. Then, we explore this attention property with an extensive experimental test to assess the performance and the possible direction of improvements for future works. Additionally, we provide mathematical proof showing the efficient feature update in our proposed method. Finally, we verify and validate the performance of the model on eight benchmark datasets that show the effectiveness of the proposed method.

Electroencephalogram (EEG) has become increasingly popular in driver fatigue monitoring systems. Several decomposition methods have been attempted to analyze the EEG signals that are complex, nonlinear and non-stationary and improve the EEG decoding performance in different applications. However, it remains challenging to extract more distinguishable features from different decomposed components for driver fatigue recognition. In this work, we propose a novel decomposition-based hybrid ensemble convolutional neural network (CNN) framework to enhance the capability of decoding EEG signals. Four decomposition methods are employed to disassemble the EEG signals into components of different complexity. Instead of handcraft features, the CNNs in this framework directly learn from the decomposed components. In addition, a component-specific batch normalization layer is employed to reduce subject variability. Moreover, we employ two ensemble modes to integrate the outputs of all CNNs, comprehensively exploiting the diverse information of the decomposed components. Against the challenging cross-subject driver fatigue recognition task, the models under the framework all showed superior performance to the strong baselines. Specifically, the performance of different decomposition methods and ensemble modes was further compared. The results indicated that discrete wavelet transform-based ensemble CNN achieved the highest average classification accuracy of 83.48% among the compared methods. The proposed framework can be extended to any CNN architecture and be applied to any EEG-related tasks, opening the possibility of extracting more beneficial features from complex EEG data.

This article presents an approach to integrate a Pareto dominance concept into a comprehensive learning particle swarm optimizer (CLPSO) to handle multiple objective optimization problems. The multiobjective comprehensive learning particle swarm optimizer (MOCLPSO) also integrates an external archive technique. Simulation results (obtained using the codes made available on the Web at http://www.ntu.edu.sg/home/EPNSugan) on six test problems show that the proposed MOCLPSO, for most problems, is able to find a much better spread of solutions and faster convergence to the true Pareto-optimal front compared to two other multiobjective optimization evolutionary algorithms. © 2006 Wiley Periodicals, Inc. Int J Int Syst 21: 209–226, 2006.

The paper describes our investigation into the neural gas (NG) network algorithm and the hierarchical overlapped architecture (HONG) which we have built by retaining the essence of the original NG algorithm. By defining an implicit ranking scheme, the NG algorithm was made to run faster in its sequential implementation. Each HONG network generated multiple classifications for every sample data presented as confidence values. These confidence values were combined to obtain the final classification of the HONG architecture. Three HONG networks based on three different feature sets with global and structural features were also trained to obtain better classification on conflicting handwritten data. An excellent recognition rate for the NIST SD3 database was consequently obtained.