Class incremental learning (CIL) requires an agent to learn distinct tasks consecutively with knowledge retention against forgetting. Problems impeding the practice of CIL methods are twofold: (1) prompt update on non-i.i.d batch streams without boundary, namely the harsher online task-free CIL (OTCIL) scenario; (2) CIL methods suffer from heavy forgetting on learning long task streams, as shown in Fig. 1(a). To achieve efficient decision-making, the ensemble deep random vector functional link network (edRVFL) with forward regularization (-F) is proposed to replace the canonical Ridge (-R), reducing more regrets during OTCIL. Considering continuous distribution drifting on long stream, we further propose edRVFL-kF to adjust the intervention intensity of forward knowledge and derive incremental updates. edRVFL-kF can effectively avoid replay, retraining, and catastrophic forgetting while achieving lower regret over -R. Moreover, to improve robustness on non-i.i.d stream and eliminate intractable tuning of -kF, we rebuild with online Bayesian learning and propose the plug-and-play edRVFL-kF-Bayes, enabling all hard ks in multiple sub-learners to self-adapt to ever-changing distribution and optimization in OTCIL. Experiments were conducted on image datasets, including multiple evaluations, ablation tests, estimated forward, and compatibility studies, which distinctly validate the efficacy of edRVFL-kF-Bayes.

Online learning of deep neural networks faces challenges such as delayed non-incremental updating, increasing consumption, retrospective retraining, and catastrophic forgetting. To alleviate these drawbacks and achieve progressive immediate decision-making, we propose a novel Incremental Online Learning (IOL) framework of Randomized Neural Networks (Randomized NN), facilitating continuous improvements and analytics to Randomized NN performance in online scenarios. Within the framework, we further formulate IOL with ridge regularization (-R) and IOL with forward regularization (-F), both avoiding retrospective retraining and catastrophic forgetting. Moreover, the incremental algorithms for -R/-F on non-stationary batch stream are derived, featuring recursive weight updates and variable learning rates. Compared to -R, we recommend -F which improves learning performance using future unlabeled observations while further reducing online regrets to offline global experts. Additionally, we conduct a detailed analysis and theoretically derive relative cumulative regret bounds of the Randomized NN learners for -R/-F under adversarial assumptions via a novel methodology and present several corollaries, from which we observed the superiority in online learning acceleration and declined regret bounds of employing -F in IOL. Finally, our proposed methods were rigorously examined across diverse tasks, from simulation, regression, and classification tasks, to long-term time-series forecasting (LTSF) and continual learning (CL) fields, which distinctly validated the efficacy of the IOL frameworks and the advantages of forward regularization.

This paper proposes the use of the Linux kernel’s ftrace framework, particularly the function_graph tracer, to generate informative system-level data for machine learning (ML) applications. Experiments on a real-world encryption detection task demonstrate the efficacy of using the proposed features across several learning algorithms. The learner is subjected to the problem of detecting encryption activities across a large dataset of files, where function call traces and graph-based features are used. Empirical results highlight an outstanding accuracy of $99.28 \%$ on the task at hand, underscoring the efficacy of features derived from the function_graph tracer. The results were further validated using an additional experiment targeting a multi-label classification problem by identifying the running programs based on trace data. This work provides comprehensive methodologies for preprocessing raw trace data and extracting graph-based features, offering significant advancements in applying ML to system behavior analysis, program identification, and anomaly detection. By bridging the gap between system tracing and ML, this paper paves the way for innovative solutions in performance monitoring and security analytics.

Understanding surgical scenes can provide better healthcare quality for patients, especially with the vast amount of video data that is generated during MIS. Processing these videos generates valuable assets for training sophisticated models. In this paper, we introduce CLIP-RL, a novel contrastive languageimage pre-training model tailored for semantic segmentation for surgical scenes. CLIP-RL presents a new segmentation approach which involves reinforcement learning and curriculum learning, enabling continuous refinement of the segmentation masks during the full training pipeline. Our model has shown robust performance in different optical settings, such as occlusions, texture variations, and dynamic lighting, presenting significant challenges. CLIP model serves as a powerful feature extractor, capturing rich semantic context that enhances the distinction between instruments and tissues. The RL module plays a pivotal role in dynamically refining predictions through iterative actionspace adjustments. We evaluated CLIP-RL on the EndoVis 2018 and EndoVis 2017 datasets. CLIP-RL achieved a mean IoU of 81%, outperforming state-of-the-art models, and a mean IoU of 74.12% on EndoVis 2017. This superior performance was achieved due to the combination of contrastive learning with reinforcement learning and curriculum learning.

The growing popularity of robotic minimally invasive surgeries has made deep learning–based surgical training a key area of research. A thorough understanding of the surgical scene components is crucial, which semantic segmentation models can help achieve. However, most existing work focuses on surgical tools and overlooks anatomical objects. Additionally, current state-of-the-art (SOTA) models struggle to balance capturing high-level contextual features and low-level edge features. We propose a Feature-Adaptive Spatial Localization model (FASL-Seg), designed to capture features at multiple levels of detail through two distinct processing streams, namely a Low-Level Feature Projection (LLFP) and a High-Level Feature Projection (HLFP) stream, for varying feature resolutions - enabling precise segmentation of anatomy and surgical instruments. We evaluated FASL-Seg on surgical segmentation benchmark datasets EndoVis18 and EndoVis17 on three use cases. The FASL-Seg model achieves a mean Intersection over Union (mIoU) of 72.71% on parts and anatomy segmentation in EndoVis18, improving on SOTA by 5%. It further achieves a mIoU of 85.61% and 72.78% in EndoVis18 and EndoVis17 tool type segmentation, respectively, outperforming SOTA overall performance, with comparable per-class SOTA results in both datasets and consistent performance in various classes for anatomy and instruments, demonstrating the effectiveness of distinct processing streams for varying feature resolutions.

The Internet of Things (IoT) has become an integral part of our daily lives. While modern interactions have become more convenient due to the myriad of IoT devices, this diversity also makes IoT devices fertile targets for cyber attacks. However, due to the resource constraints of IoT deployment devices, intrusion detection schemes must be customized to meet the specific requirements of the IoT environment, especially in terms of power consumption and computing performance. In this paper, we benchmark multiple lightweight CNN-based models using public IoT network traffic datasets due to their wide popularity in network traffic classification. We evaluated also 1D and 2D variants of an optimized CNN model. Empirical results reveal that 1D models tend to perform better than 2D variants and other evaluated popular lightweight models. On the other hand, 2D-CNN offers less computation time and less memory footprint when compared with 1D-CNN indicating better efficiency. We further subject the competing methods to an early intrusion detection experiment. Results indicate that intrusions are successfully detected using as few as 6 initial packets of a session.

Non-technical loss and energy theft detection are crucial for improving the stability and reducing financial losses in smart grid and power grid utilities. Recently, the availability of massive datasets has improved detection capabilities using sophisticated techniques like deep neural networks. However, training models on extensive feature sets, such as multi-year data, can lead to confusion due to varied behavioral changes in electricity consumption. To address this, we propose a reformulation of the energy theft detection problem by segmenting the time series data and training individual models on each segment. These models’ anomaly scores are then aggregated to produce a final classification. Our framework has shown significant improvement, elevating the F1 score from 0.6 to 0.74, outperforming recent state-of-the-art techniques on the SGCC dataset, the only publicly available dataset labeled for energy theft.

Federated Learning (FL) for household-level Short-Term Load Forecasting (STLF) has emerged as a solution to privacy concerns in smart grids, enabling clients to collaboratively train models without sharing their consumption data with a central server. However, sharing model updates can still introduce privacy risks. A common solution to mitigate this risk is the use of differential privacy during the federation process. However, there is a lack of empirical privacy analysis of these techniques in smart grid scenarios. This paper proposes a property inference attack utilizing a single update per client to evaluate privacy risks in federated learning within smart grid contexts. We assess the privacy risk associated with the standard FedAvg algorithm and a differentially private noise-before-aggregation (NBA) FL scheme. Furthermore, we investigate the trade-off between privacy and utility in the NBA-FL scheme. Our empirical findings reveal significant information leakage with standard FedAvg scheme. An adversary with access to a single model update can identify global data properties of the FedAvg client local dataset with an Area Under the Curve (AUC) of 72%. This privacy leakage can be reduced using NBA-FL, which reduces the AUC to 60%. However, the addition of noise to the model updates results in a utility loss of up to 70% in the model’s predictive power. This significant degradation in model performance outweighs the advantages of the scheme.

Over the past years, notable progress has been made towards fighting misinformation spread over social media, encouraging the development of many fact-checking systems. However, systems that operate over Arabic content are scarce. In this work, we bridge this gap by proposing Tahaqqaq (Verify), an Arabic real-time system that helps users verify claims over Twitter with several functionalities, such as identifying check-worthy claims, estimating credibility of users in terms of spreading fake news, and finding authoritative accounts. Tahaqqaq has a friendly online Web interface that supports various real-time user scenarios. In the same breath, we enable public access to Tahaqqaq services through a handy RESTful API. Finally, in terms of performance, multiple components of Tahaqqaq outperform the state-of-the-art models on Arabic datasets.

The ransomware threat has loomed over our digital life since 1989. Criminals use this type of cyber attack to lock or encrypt victims' data, often coercing them to pay exorbitant amounts in ransom. The damage ransomware causes ranges from monetary losses paid for ransom at best to endangering human lives. Cryptographic ransomware, where attackers encrypt the victim's data, stands as the predominant ransomware variant. The primary characteristics of these attacks have remained the same since the first ransomware attack. For this reason, we consider this a key factor differentiating ransomware from other cyber attacks, making it vital in tackling the threat of cryptographic ransomware. This paper proposes a cyber kill chain that describes the modern crypto-ransomware attack. The survey focuses on the Encryption phase as described in our proposed cyber kill chain and its detection techniques. We identify three main methods used in detecting encryption-related activities by ransomware, namely API and System calls, I/O monitoring, and file system activities monitoring. Machine learning (ML) is a tool used in all three identified methodologies, and some of the issues within the ML domain related to this survey are also covered as part of their respective methodologies. The survey of selected proposals is conducted through the prism of those three methodologies, showcasing the importance of detecting ransomware during pre-encryption and encryption activities and the windows of opportunity to do so. We also examine commercial crypto-ransomware protection and detection offerings and show the gap between academic research and commercial applications.

Several fake claims are commonly repeated over time, especially on social media. To identify such previous claims, the verified claim retrieval task was studied, where, for a given input claim, the goal is to find previously-verified claims that are relevant to it. However, this view assumes that each claim was already verified, which may not be true for all claims in the real-world scenario. In this work, we introduce the Verified Claim Checking problem over Twitter, in which the relevant verified claims are retrieved only if the input claim was indeed previously-verified, thus saving computation time. We address the problem by proposing SpotVC, an end-to-end approach consisting of two stages, namely a filter and a reranker. The proposed filter achieved an average F1 of 0.81 while significantly reducing computation time. Moreover, the proposed reranker outperformed the state-of-the-art models on two public datasets and provided on-par performance on a third one. Overall, our proposed system exhibits an effective operational balance in the trade-off between efficiency and effectiveness for the real-world scenario.

Autism spectrum disorder is a neurodevelopmental disorder that is characterized by patterns of behaviours and difficulties with social communication and interaction. Children on the spectrum exhibit atypical, restricted, repetitive, and challenging behaviours. In this study, we investigate the feasibility of integrating wearable sensors and machine learning techniques to detect the occurrence of challenging behaviours in real-time. A session of a child with autism interacting with different stimuli groups that included social robots was annotated with observed challenging behaviors. The child wore a wearable device that captured different motion and physiological signals. Different features and machine learning configurations were investigated to identify the most effective combination. Our results showed that physiological signals in addition to typical kinetic measures led to more accurate predictions. The best features and learning model combination achieved an accuracy of 97%. The findings of this work motivate research toward methods of early detection of challenging behaviours, which may enable the timely intervention by caregivers and possibly by social robots.

In recent years, unmanned aerial vehicles (UAVs) have become increasingly accessible to the public due to their high availability with affordable prices while being equipped with better technology. However, this raises a great concern from both the cyber and physical security perspectives since UAVs can be utilized for malicious activities in order to exploit vulnerabilities by spying on private properties, critical areas or to carry dangerous objects such as explosives which makes them a great threat to the society. Drone identification is considered the first step in a multi-procedural process in securing physical infrastructure against this threat. In this paper, we present drone detection and identification methods using deep learning techniques such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) and Convolutional Recurrent Neural Network (CRNN). These algorithms will be utilized to exploit the unique acoustic fingerprints of the flying drones in order to detect and identify them. We propose a comparison between the performance of different neural networks based on our dataset which features audio recorded samples of drone activities. The major contribution of our work is to validate the usage of these methodologies of drone detection and identification in real life scenarios and to provide a robust comparison of the performance between different deep neural network algorithms for this application. In addition, we are releasing the dataset of drone audio clips for the research community for further analysis.

Over the past decade or so, several research groups have addressed the problem ofmulti-label classificationwhere each example can belong to more than one class at the same time. A common approach, calledBinary Relevance (BR), addresses this problem by inducing a separate classifier for each class. Research has shown that this framework can be improved if mutual class dependence is exploited: an example that belongs to class$X$is likely to belong also to class$Y$; conversely, belonging to$X$can make an example less likely to belong to$Z$. Several works sought to model this information by using the vector of class labels as additional example attributes. To fill the unknown values of these attributes during prediction, existing methods resort to using outputs of other classifiers, and this makes them prone to errors. This is where our paper wants to contribute. We identified two potential ways to prune unnecessary dependencies and to reduce error-propagation in our new classifier-stacking technique, which is namedPruDent. Experimental results indicate that the classification performance ofPruDentcompares favorably with that of other state-of-the-art approaches over a broad range of testbeds. Moreover, its computational costs grow only linearly in the number of classes.