This study tackles the challenge of neutralizing a malicious drone swarm targeting critical infrastructure. Given the impracticality of destroying all drones, we propose an uncertainty-aware threat assessment method, modeling payload size and positional uncertainties probabilistically. We studynode removalas the core decision problem to fragment communication and reduce attack power. The threat of each drone is quantified by its expected payload and distance-based probabilistic model. We also assess the probabilistic connectivity between drones to evaluate the swarm’s robustness. A novel evaluation function integrates these factors, and we introduce a probabilistic greedy search algorithm to minimize the swarm’s threat. We evaluate against centrality-based and weight-augmented dismantling heuristics, a recent DQN policy, and a small-n brute-force oracle. Our approach achieves a 70% reduction in attack power and near-optimal performance, deviating by less than 5% from the best possible outcomes under uncertainty.

In recent years, there has been growing interest in applying deep learning techniques for visual anomaly detection, particularly in the manufacturing sector. Various models have been developed to identify defects in manufacturing data, yet selecting and optimizing these models for anomaly detection in intelligent manufacturing environments remains a significant challenge. This research focuses on general-purpose visual anomaly detection, aiming to reduce dependence on domain-specific knowledge and create flexible, generic models. We propose a novel deep learning framework in which multiple models are trained for each image. The visual features and loss values from these models are computed and stored during training. During the testing phase, this stored information is used to select the most appropriate model for each new image using a k-Nearest Neighbors (kNN) approach. The proposed method, KGDL-VAD (Knowledge-Guided Deep Learning for Visual Anomaly Detection), was evaluated on the MVTec AD, and standard aerospace defect detection datasets, achieving an area under the curve (AUC) score of 0.96, outperforming baseline methods. In addition, KGDL-VAD surpasses ensemble learning approaches across multiple domain-independent datasets with varying numbers of trained classes.

This paper targets a new foundation for designing general-purpose learning systems, by establishing a consensus method that facilitates self-adaptation and flexibility to deal with different learning tasks and different data distribution. We present the Shapely Consensus Deep Learning (SCDL) as a consensus method for general-purpose solutions that do not require the help of domain experts. SCDL is two-level based learning process. In the first level, several deep learning models are trained and the Shapley Value is used to determine the contribution of each subset of models in the training. The models are pruned according to their contribution in the learning process. In the second level, the loss information of each data distribution is saved in the knowledge base. Both levels are explored to prune the models for each new observation. We present the evaluation of the generality of SCDL using different datasets with different shapes, and complexities. The results reveal the effectiveness of SCDL for weakly classification. Concretely, SCDL achieved 90% of AUC with less than 86% for the baseline solutions.

As the metaverse grows in popularity and complexity, securing its virtual environment is critical. Metaverse intrusion detection involves identifying and preventing unauthorized access, malicious activities, and potential threats. To address these challenges, we propose a novel Metaverse intrusion detection system (MIDS) that combines generative adversarial networks (GAN) and Transformer-based classifiers. The system operates in three stages: 1) generating diverse and realistic network traffic using GAN; 2) detecting intrusions with a Transformer-based classifier; and 3) ensuring data privacy through federated learning and a trusted authority mechanism. Unlike traditional methods, our approach employs dual aggregation, generating both global and local models tailored to users’ needs. Tested on public datasets, the method achieves state-of-the-art performance with an F1-score of 0.9984, demonstrating its effectiveness in generating realistic training data and improving MIDS performance. This approach can extend to other security domains requiring diverse data for training.

Abstract In an era defined by the relentless influx of data from diverse sources, the ability to harness and extract valuable insights from streaming data has become paramount. The rapidly evolving realm of online learning techniques is tailored specifically for the unique challenges posed by streaming data. As the digital world continues to generate vast torrents of real-time data, understanding and effectively utilizing online learning approaches are pivotal for staying ahead in various domains. One of the primary goals of online learning is to continuously update the model with the most recent data trends while maintaining and improving the accuracy of previous trends. Based on the various types of feedback, online learning tasks can be divided into three categories: learning with full feedback, learning with limited feedback, and learning without feedback. This survey aims to identify and analyze the key challenges associated with online learning with full feedback, including concept drift, catastrophic forgetting, skewed learning, and network adaptation, while the other existing reviews mainly focus on a single challenge or two without considering other scenarios. This article also discusses the application and ethical implications of online learning. The results of this survey provide valuable insights for researchers and instructional designers seeking to create effective online learning experiences that incorporate full feedback while addressing the associated challenges. In the end, some conclusions, remarks, and future directions for the research community are provided based on the findings of this review.

We present a novel approach for addressing computer vision tasks in intelligent transportation systems, with a strong focus on data security during training through federated learning. Our method leverages visual transformers, training multiple models for each image. By calculating and storing visual image features as well as loss values, we propose a novel Shapley value model based on model performance consistency to select the most appropriate models during testing. To enhance security, we introduce an intelligent federated learning strategy, where users are grouped into clusters based on constrastive clustering for creating a global model as well as customized local models. Users receive both global as well as local models, enabling tailored computer vision applications. We evaluated KGVT-ITS (Knowledge Guided Visual Transformers for Intelligent Transportation Systems) on various ITS challenges, including pedestrian detection, abnormal event detection, as well as near-crash detection. The results demonstrate the superiority of KGVT-ITS over baseline solutions, showcasing its effectiveness and robustness in intelligent transportation scenarios. More particularly, KGVT-ITS achieves significant improvements of about 8% against the existing ITS methods.

This paper introduces a novel concept for Human Activity Recognition (HAR) that allows robust analysis, classification, and understanding of human movements in various environments. It can be applied in various applications such as Health monitoring and analysis, fitness/dance training and performance analysis, interactive gaming, smart homes, and wearable devices. The novel method, coined as STL-HAR (SpatioTemporal Learning for HAR), learns from sensor data jointly represented in space and time to robustify the HAR process. In the new concept, we propose a hybrid model based on GNN (Graph Neural Network), and LSTM (Long Short-Term Memory). GNN first learns the spatial features from different sensor data locations. The learned features will then be injected to LSTM where the temporal information is captured by observing sensor status at different timestamps. We evaluate and analyze the performance of STL-HAR in real use case scenarios of HAR data compared with baseline HAR-based solutions. STL-HAR has achieved a recognition rate of 92% under different scenarios.

We introduce an innovative methodology for the identification of vehicular collisions within Internet of Vehicles (IoV) applications. This approach combines a knowledge base system with deep learning for model selection in an ensemble learning setting. It is designed to provide a general near-crash detection capability without relying on domain-specific knowledge, enabling the development of generic deep learning models. Our proposed methodology employs a novel deep learning approach, wherein multiple learning models are individually trained for each image. Subsequently, visual features are computed and stored for each trained image, along with the associated loss values from the training phase. This stored information is utilized to select the most suitable models for processing new image data during the testing phase. To facilitate efficient model selection, we employ a kNN (k Nearest Neighbors) strategy. To enhance both data and model security in IoV environments, we implement an intelligent federated learning (FL) strategy. Users are organized into clusters, and we employ two distinct aggregation methods, departing from conventional federated learning approaches. In the initial stage, we aggregate model data from all users to create a global model representing collective knowledge. In the subsequent stage, we aggregate models from each cluster to generate customized local models. Users are provided with both global and local models, allowing them to select the most suitable model for their specific crash detection needs. We test our approach, that we call Knowledge Guided Deep Learning for Near Crash Detection (KGDL-NCD), on well-known NCD benchmarks. The results demonstrate that KGDL-NCD surpasses baseline solutions, achieving an AUC (Area Under Curve) metric of 0.95.

We present a novel approach to detecting fake news in Internet of Things (IoT) applications. By investigating federated learning and trusted authority methods, we address the issue of data security during training. Simultaneously, by investigating convolution transformers and user clustering, we deal with multi-modality in fake news data. Firstly, we use dense embedding and the k-means algorithm to cluster users into groups that are similar to one another. We then develop a local model for each user using their local data. The server then receives the local models of the users along with the clustering information, and a trusted authority verifies their integrity there. We use two different types of aggregation in place of conventional federated learning systems. The initial step is to combine all the users' models to create a single global model. The second step entails compiling each user's model into a local model of comparable users. Both models are supplied to the users, who then select the most suitable model for identifying fake news. By conducting extensive experiments using Twitter data, we demonstrate that the proposed method outperforms various baselines, where it achieves an average accuracy of 0.85 in comparison to others that do not exceed 0.81.

Many Internet of Things (IoT) applications can benefit from Social Web of Things (S-WoT) methods that enable knowledge discovery and help solving interoperability problems. The semantic modeling of S-WoT is the main emphasis of this work where we suggest a novel solution, evolutionary clustering for ontology matching (ECOM), to explore correlations between S-WoT data using clustering and evolutionary computation methodologies. The ECOM approach uses a variety of clustering techniques to aggregate S-WoT data's strongly related ontologies into comparable categories. The principle is to match concepts of similar groups rather than full concepts of two ontologies, which necessitates splitting examples of each ontology into similar groups. We design two clustering algorithms for ontology matching using conventional methods, as well as sophisticated clustering techniques. Moreover, we develop an intelligent matching algorithm that uses evolutionary computation to quickly converge to (or ideally identify) optimal matches. Numerous simulations have been conducted using various ontology databases to demonstrate the application and precision of ECOM. Our findings clearly show that ECOM has better results when compared to cutting-edge ontology matching methods. The F-measure of ECOM exceeds 95% whereas it does not reach 90% for all baseline methods. The results also confirm that ECOM scales with big data in S-WoT environments.

The advent of transformative technologies such as the Internet of Things (IoT) has brought forth significant advancements in various sectors like smart cities, fintech, learning, and healthcare, as well as revolutionized online activities. The IoT has facilitated widespread connectivity by interconnecting numerous objects and services, but it has also made IoT and cloud infrastructures susceptible to cyberattacks, making cybersecurity a paramount concern, particularly for the development of reliable IoT systems, especially those powering smart city networks. In this research endeavor, we embark on exploring a cutting-edge pipeline that amalgamates federated deep learning with a trusted authority approach to tackle the intricate challenges associated with intrusion detection in smart city networks. To identify anomalies and intrusions effectively within the network, we devise an improved LSTM (Long Short-Term Memory) model. Additionally, we propose an intelligent swarm optimization solution to address dimensionality reduction concerns. Thorough evaluations of our federated learning-based approach are conducted, and these are juxtaposed with several basic approaches, utilizing the renowned NSL-KDD dataset. Encouragingly, our findings reveal that the proposed framework remarkably outperforms the baseline solutions, particularly when dealing with datasets containing a substantial volume of transactions. Furthermore, our method ensures robust data security for the model, as it becomes the pioneering endeavor to incorporate the principle of trusted authority into the realm of federated learning for the management of smart city networks.

This paper presents a feasibility analysis of the ORB-SLAM [1] for real-time vehicle localization and tracking using a monocular visual camera providing 360° panoramic views. This method described in [1] was initially designed and developed for conventional cameras, making use of a method for detection and tracking visual features and estimating the camera trajectory while reconstructing the environment. The accuracy of the tracking depends on the ability of this method to robustly detect and match sufficient visual features. This work aims to extend this method to large monocular round views using fish-eye-like cameras allowing an increase of visual features with the aim of improving localization robustness. The main challenge in using a standard fish-eye camera for generating panoramic views is the reduction of visual performance due to a potential higher distortion and lower spatial resolution compared to that using a standard camera lens. The objective of this research is to perform a feasibility analysis of a method combining a camera equipped with a panomorph lens to generate real-time panoramic views at minimal distortion and ORB-SLAM to robustly detect and track visual features for real-time camera localization and tracking. A quantitative evaluation is performed on a vehicle driving in an outdoor natural scene with the monocular panomorph camera mounted on-front and without any other additional sensors. The results with analysis and a concluding summary are included as well.

This paper presents a stereo matching approach for a novel multi-perspective panoramic stereo vision system, making use of asynchronous and non-simultaneous stereo imaging towards real-time 3D 360° vision. The method is designed for events representing the scenes visual contrast as a sparse visual code allowing the stereo reconstruction of high resolution panoramic views. We propose a novel cost measure for the stereo matching, which makes use of a similarity measure based on event distributions. Thus, the robustness to variations in event occurrences was increased. An evaluation of the proposed stereo method is presented using distance estimation of panoramic stereo views and ground truth data. Furthermore, our approach is compared to standard stereo methods applied on event-data. Results show that we obtain 3D reconstructions of 1024 × 3600 round views and outperform depth reconstruction accuracy of state-of-the-art methods on event data.

This paper presents a system for Braille learning support using real-time panoramic views generated from the novel smart panorama camera 360SCAN. The system makes use of the modern image processing libraries and state-of-the-art features extraction and clustering methods. We compare the real-time frames recorded by the bio-inspired camera to the reference images in order to determine particular figures. One contribution of the proposed method is that image edges can be transformed to the presentation on Braille display directly without any image processing. It is possible due to the bio-inspired construction of camera sensor. Another contribution is that our approach provides Braille users with images recorded from natural scenes. We conducted several experiments that verify the methods that demonstrate learning figures captured by the smart camera. Our goal is to process such images and present them on the Braille Display in a form appropriate for visually impaired people. All evaluations were performed in the natural environment with ambient illumination of 200 lux, which demonstrates high camera reliability in difficult light conditions. The system can be optimized by applying additional filters and features algorithms and by decreasing the rotational speed of the camera. The presented Braille learning support system is a building block for a rich and qualitative educational system for the efficient information transfer focused on visually impaired people.

This paper presents a system for quality control of real-time panoramic views generated from the novel smart panorama camera 360SCAN. The system makes use of the modern image processing library OpenIMAJ and state-of-the-art features extraction and clustering methods. We compare a real-time frame collection recorded by the camera to a reference image collection in order to determine camera readiness. We conducted several experiments that verify the methods that demonstrate smart camera operational status and evaluate changes in the position or number of objects in the working location. All evaluations were performed in the natural environment with ambient illumination of 200 lux, which demonstrates high camera reliability in difficult light conditions. The system can be optimized for embedded applications by applying additional filters and features algorithms and by decreasing the rotational speed of the camera. The presented quality control system is a building block for a rich and qualitative expert system for the efficient control and support of the smart camera.

This paper presents a recently developed dynamic stereo vision sensor system and its application for fall detection towards safety for elderly at home. The system consists of (1) two optical detector chips with 304×240 event-driven pixels which are only sensitive to relative light intensity changes, (2) an FPGA for interfacing the detectors, early data processing, and stereo matching for depth map reconstruction, (3) a digital signal processor for interpreting the sensor data in real-time for fall recognition, and (4) a wireless communication module for instantly alerting caring institutions. This system was designed for incident detection in private homes of elderly to foster safety and security. The two main advantages of the system, compared to existing wearable systems are from the application's point of view: (a) the stationary installation has a better acceptance for independent living comparing to permanent wearing devices, and (b) the privacy of the system is systematically ensured since the vision detector does not produce real images such as classic video sensors. The system can actually process about 300 kevents per second. It was evaluated using 500 fall cases acquired with a stuntman. More than 90% positive detections were reported. We will show a live demonstration during ISCAS2012 of the sensor system and its capabilities.

This paper presents a novel smart camera BiCa360 for real-time 360° panoramic views using a rotating dynamic vision sensor at up to 10 rotations per sec. The system consists of (1) a dual-line dynamic vision sensor generating events at high temporal resolution, on-chip time stamping (1μs resolution), having a high dynamic range and the sparse visual coding of the information, (2) a high-speed mechanical device rotating at up to 10 revolutions per sec (rps) where the sensor is mounted and (3) a real-time embedded software for panoramic reconstruction of the 360° panoramic views. Within this work, we show the capabilities of the system in terms of data quality (scene reconstruction). We made several experiments to assess the angular resolution as well as the visual quality of the data for rotations ranging from 1 to 10 rps. All evaluations were performed on natural scene with ambient illuminations. Within the live demonstration, we will show BiCa360 providing 360° panoramic views of the natural scene in real-time and at different rotations.

This paper presents an evaluation of spatio-temporal data generated by a dynamic stereo vision sensor in a highdimensional space (3D volume and time) for motion analysis and gesture recognition. In contrast to traditional frame-based (synchronous) stereo cameras, dynamic stereo vision sensors asynchronously generates events upon scene dynamics. Motion activities are intrinsically (on-chip) segmented by the sensor, such that activity, gesture recognition and tracking can be intuitively and efficiently performed. In this work, we investigated the applicability of this sensor for gesture recognition. We developed a machine learning method based on the Hidden Markow Model for training and automated classifications of gestures using the event data generated by the sensor. By training eight different activities (dance figures) with 15 persons we build up a library of 580 recorded activities. An average recognition rate of 97% has been reached.

This live demonstration aims to show a real-time people tracking using an embedded smart stereo vision system (3DVS) consisting of two asynchronous (biologically-inspired) dynamic vision sensors and a processing unit (DSP) embedding a detection and tracking algorithm.

Biologically-inspired dynamic vision sensors have been introduced in 2002 which asynchronously detect the significant relative light intensity changes in a scene and output them in a form of Address-Event representation. These vision sensors capture dynamical discontinuities on-chip for a reduced data volume compared to that from intensity images. Therefore, they support detection, segmentation and tracking of moving objects in the Address-Event space by exploiting the generated events, as a reaction to intensity changes, resulting from the scene dynamics. Object tracking has been previously demonstrated and reported in scientific publications using monocular dynamic vision sensors. This paper contributes with presenting and demonstrating a tracking algorithm using the 3D sensing technology based on the stereo dynamic vision sensor. This system is capable of detecting and tracking persons within a 4m range at an effective refresh rate of the depth map of up to 200 per second. The 3D system is evaluated for people tracking and the tests showed that up to 60k Address-Events/s can be processed for real-time tracking.

Although artificial neural networks have been applied to problems within hydrology for over ten years, there is little consensus on the 'best' type of neural network model to use and the most effective means of training the chosen model. In order to explore the different approaches neural network modellers use to forecasting river stage, an international comparison study was undertaken during 2004. This research was based on a set of rainfall and river stage data covering three winter periods for an unidentified river basin in England (with a catchment of 331,500 Ha in the north of the country), sampled at 15 minute intervals. Several neural network enthusiasts took part in the study from a number of different countries. The preferred methodologies and forecasting outputs from a number of 'blind' models of river stage developed by the participants have been collated and are presented in this paper.