We present a training-free, plug-and-play method, namely VFace, for high-quality face swapping in videos. It can be seamlessly integrated with image-based face swapping approaches built on diffusion models. First, we introduce a Frequency Spectrum Attention Interpolation technique to facilitate generation and intact key identity characteristics. Second, we achieve Target Structure Guidance via plug-and-play attention injection to better align the structural features from the target frame to the generation. Third, we present a Flow-Guided Attention Temporal Smoothening mechanism that enforces spatiotemporal coherence without modifying the underlying diffusion model to reduce temporal inconsistencies typically encountered in frame-wise generation. Our method requires no additional training or video-specific fine-tuning. Extensive experiments show that our method significantly enhances temporal consistency and visual fidelity, offering a practical and modular solution for video-based face swapping. Our code is available at VFace.

With the rapid deployment of vision systems on edge devices, energy-efficient and temporally aware image deraining models are increasingly needed. We propose SpikeRain, a spiking neural network (SNN) that achieves competitive deraining performance with substantially lower computational cost than conventional artificial neural networks (ANNs). Unlike ANN-based approaches with dense activations and high memory demands, SpikeRain leverages the event-driven sparse-firing nature of spiking neurons for efficient temporal integration and contextual learning. Built on an encoder-decoder framework, SpikeRain incorporates three spiking native modules: a Dense Spiking Residual Block (DSRB) for temporal integration and feature reuse, a Multi-Dimensional Spiking Attention (MDSA) module to model temporal channel spatial dependencies, and an Adaptive Residual Feature Enhancement (ARFE) block with gated attention to refine salient features. Experiments on synthetic and real-world benchmarks show that SpikeRain achieves state-of-the-art PSNR and SSIM while reducing parameters by approximately 40% and FLOPs by approximately 89%, with energy efficiency on par with existing SNN methods. These results highlight the potential of SNNs for real-time low-power image restoration on neuromorphic platforms. SpikeRain code is available on GitHub.

The unpredictable nature of rip currents makes them a leading cause of coastal drowning incidents globally. Traditional methods fall short, necessitating an advanced surveillance system that can prioritize critical threats, enable autonomous decision-making with adaptive network control, and optimize resource allocation for enhanced coastal safety. Intent-based networking (IBN) plays a pivotal role in converting high-level intents into automated processes, enabling dynamic control and intelligent resource allocation in critical applications such as the Internet of Things (IoT) and unmanned aerial vehicle (UAV)-based coastal surveillance. This study proposes an artificial intelligence (AI)-powered, IBN-driven framework for coastal surveillance that leverages UAVs and IoT devices to enable real-time rip current analysis through advanced segmentation techniques. In our framework, UAVs with AI-powered IoT systems perform initial rip current analysis using lightweight deep-learning models. High-risk detections are prioritized through the closed-loop feedback mechanism of the IBN and transmitted to control rooms for validation and response, ensuring efficient resource utilization and adaptive surveillance. We expanded the rip current dataset to enhance the segmentation accuracy by incorporating additional samples from open-source platforms and applying diverse environmental conditions. We trained YOLO models and Mask R-CNN, which are suitable for real-time rip current analysis. In addition, we introduced a modified YOLOv11n-seg model, replacing the C3K2 block with C2F and optimizing the channels to reduce the parameters while maintaining accuracy. The best-performing models were tested on edge devices to evaluate the time complexity and reliability.

Federated learning (FL) enables privacy-preserving analytics at the medical IoT (IoMT) edge but is vulnerable to model poisoning and distribution shift. We presentATTA-FL-Lite, a lightweight aggregation rule that admits a client update only when three tests are jointly satisfied: (i) scale conformity via a median–absolute–deviation (MAD)z-score, (ii) directional alignment via cosine similarity to a coordinate-wise median reference, and (iii) non-degradation via a validation-lossz-score computed on a small, centrally held clean set. If no update passes, a coordinate-wise median fallback is used. We provide sub-Gaussian tail bounds for the loss test and an expected one-step descent bound forL-smooth objectives under benign mean-alignment and an accepted-set composition assumption; the analysis does not require a positive cosine threshold. Experiments on MNIST, Fashion-MNIST, and PathMNIST, under IID and non-IID partitions with up to 40% adversaries across four attack families, show that ATTA-FL-Lite maintains accuracy representatively ≈ 0.78–0.98 across Tiny/Small/Medium CNNs, reliably filters magnitude/noise attacks, and remains competitive against sign-flip. Server runtime scales approximately linearly with the number of participating clients at fixed model and validation sizes. These results indicate that ATTA-FL-Lite offers practical robustness for FL in resource-constrained IoMT deployments without cryptographic overhead or trusted root data beyond a small validation set.

• Novel QCERN framework clusters and ranks image quality effectively. • Incorporates Order, Metric, and Center Loss for precise alignment. • Demonstrates superior generalization across diverse datasets. • Offers applications in photography, medical imaging, and surveillance. • Utilizes dynamic score anchors for improved accuracy and adaptability. This paper presents a new technology that focuses on blind image quality assessment (BIQA) through a framework known as Quality-Centric Embedding and Ranking Network (QCERN). The framework ensures maximum efficiency when processing images under various possible scenarios. QCERN is entirely different from contemporary BIQA techniques, which focus solely on regressing quality scores without structured embeddings. In contrast, the proposed model features a well-defined embedding space as its principal focus, in which picture quality is both clustered and ordered. This dynamic quality of images enables QCERN to utilize several adaptive ranking transformers along a geometric space populated by dynamic score anchors representing images of equivalent quality QCERN features a distinct advantage since unlabeled images of interest can be placed by evaluation of their distance to these specified score anchors inductively in the embedding space, improving accuracy as well as generalization across disparate datasets. Multiple loss functions are utilized in this instance, including order and metric loss, to ensure that images are positioned correctly according to their quality while maintaining distinct divisions of quality. With the application of QCERN, numerous experiments have demonstrated its ability to outperform existing models by consistently delivering high-quality predictions across various datasets, making it a competitive option. This quality-centric embedding and ranking methodology is excellent for reliable quality assessment applications, such as in photography, medical imaging, and surveillance.

Cloth-changing Person Re-Identification (CC-ReID) is a challenging data modeling task that involves identifying specific pedestrians wearing different outfits. Existing methods primarily focus on altering clothing color and directly reconstructing appearance to extract features independent of the clothes. Real pedestrians differ in height, body shape, etc. Such methods are prone to losing the intrinsic information of the original sample (i.e., the person identity) owing to the absence of contextual phenomena (e.g., texture structure and local correlation), which decreases the recognition performance. To address this problem, we propose a framework called HPRNet, or ”Human Parsing Reconstruction with Non-Local Multi-Scale Perception Network,” which includes a non-local weighted multi-scale perception (NWMP) module and a parsing reconstruction exploration (PRE) module. In particular, the proposed NWMP module effectively captures the global receptive field of a sample and obtains a contextual correlation between non-neighboring pixels within the sample image. The PRE module was used to achieve a more accurate reconstruction of human body components with a clothing parsing model to better distinguish features related to or unrelated to clothes. Extensive experiments were conducted on CC-ReID public datasets (LTCC, PRCC, and CCVID) to demonstrate the effectiveness and competitiveness of the proposed method with state-of-the-art (SOTA) baselines for this complex modeling task.

Timely and accurate crop damage classification (CDC) is vital for informed decision-making in the industry of precision agriculture. Traditional manual methods are slow and unreliable, whereas recent deep learning models, although accurate, are often too computationally intensive for resource-constrained environments. In this study, we present LNetCDC, a lightweight attention-based convolutional neural network tailored for CDC. The architecture integrates an EchoBlock for efficient feature extraction, combined with residual pathways enhanced by“Channelwise Refine”and“Dual Gate Attention”modules to emphasize critical spatial and channelwise features. Also, dilated convolutions are incorporated into deeper layers to capture multiscale contextual patterns. We evaluated our LNetCDC on a benchmark crop damage dataset, where it outperformed existing state-of-the-art (SOTA) models in terms of both accuracy and efficiency. Notably, it achieves around 2.3% gain in accuracy with only 0.86 million parameters compared with 1.13 million in the prior SOTA model for CDC. These results demonstrate the effectiveness and suitability of LNetCDC for real-time deployment on industrial edge devices.

Home security and surveillance systems are rapidly evolving, with Artificial Intelligence (AI) playing a transformative role in enhancing safety and threat detection. While several AI methods and datasets for intruder-related risk assessment exist, they predominantly focus on face detection and recognition, leaving a significant gap in addressing high-risk scenarios involving malicious intent, such as theft or harm. The lack of dedicated datasets for recognizing complex intruder activities, such as carrying weapons or engaging in destructive actions like kicking doors or breaking locks, limits the development of robust solutions. This work bridges this gap by introducing the Intruder Activity Recognition Dataset (IARD), a video dataset specifically designed to recognize four critical intruder activities: Armed Intruder, Door Kick, Intruder Inside and Lock Breaking. Leveraging IARD, we thoroughly benchmark various state-of-the-art methods, among which a Vision Transformer is found to achieve an impressive 93.3% accuracy in recognizing intruder actions. Our contribution highlights the potential of IARD in advancing AI-driven surveillance systems, providing a foundational dataset and benchmark for recognizing complex intruder activities.

Image super-resolution (SR) focuses on reconstructing high-resolution images from their low-resolution counter-parts, often affected by sensor limitations or environmental factors. Convolutional Neural Networks (CNNs) are state-of-the-art for SR tasks but computationally heavy. This paper introduces a novel CRMN (Convolutional Recurrent Mixer Network), a hybrid deep learning-based SR technique designed to address the complexity of CNNs, which is validated in the context of meteorological radar images. Experiments on public benchmark datasets (Berkley432 and T291) and our newly manually collected precipitation dataset from the Meteorological Research Institute (IPMET) show that our CRMN model provides competitive results compared to leading SR methods with significantly fewer parameters, making it a promising and practical solution for SR applications, particularly radar meteorology.

Watermarking embeds imperceptible patterns into images for authenticity verification. However, existing methods often lack robustness against various transformations primarily including distortions, image regeneration, and adversarial perturbation, creating real-world challenges. In this work, we introduce SpecGuard, a novel watermarking approach for robust and invisible image watermarking. Unlike prior approaches, we embed the message inside hidden convolution layers by converting from the spatial domain to the frequency domain using spectral projection of a higher frequency band that is decomposed by wavelet projection. Spectral projection employs Fast Fourier Transform approximation to transform spatial data into the frequency domain efficiently. In the encoding phase, a strength factor enhances resilience against diverse attacks, including adversarial, geometric, and regeneration-based distortions, ensuring the preservation of copyrighted information. Meanwhile, the decoder leverages Parseval's theorem to effectively learn and extract the watermark pattern, enabling accurate retrieval under challenging transformations. We evaluate the proposed SpecGuard based on the embedded watermark's invisibility, capacity, and robustness. Comprehensive experiments demonstrate the proposed SpecGuard outperforms the state-of-the-art models. To ensure reproducibility, the full code is released on \href{https://github.com/inzamamulDU/SpecGuard_ICCV_2025}{\textcolor{blue}{\textbf{GitHub}}}.

Detecting hazardous activities is essential for ensuring safety. However, existing datasets often lack coverage of the nuanced and diverse hazards present in indoor environments, which hinders the development of a specialized model. To address this, we introduce the Real-World Hazardous Activities Dataset (RHAD), a novel and diverse video dataset specifically curated for recognizing hazardous activities in real-world indoor settings. Leveraging RHAD, we introduce HazardNet, a hybrid deep-learning architecture designed for hazardous activity recognition. HazardNet integrates local and global spatial-temporal representation modules to effectively capture complex patterns, enabling a robust understanding of the activity. We perform comprehensive evaluations by benchmarking against a range of state-of-the-art activity recognition models. Experimental results show that our proposed model performs significantly better, surpassing the latest model, VideoMamba, with a 9.2% accuracy gain. Moreover, by providing the dataset and an effective recognition model, our work lays the foundation for further research, paving the way for enhanced safety measures and preventive interventions. The dataset and code are available at https://github.com/ShehzadCS18/RHAD.

Facial emotion recognition (FER) plays a critical role in understanding human behavior, especially for individuals suffering from neurological disorders (NDs) like Parkinson’s disease (PD), Multiple Sclerosis (MS), and Stroke. Early and accurate detection of emotions is crucial for both the diagnosis of associated mood disorders and continuous monitoring. However, traditional methods often fall short in providing noninvasive, real-time solutions and lack the clinical expertise necessary to identify the specific emotion types associated with each ND category. In response, this research conducted under the ALAMEDA consortium presents an Internet of Things-based FER AI Toolkit designed to enhance early diagnosis and treatment for brain diseases. The toolkit is in line with the consortium’s clinical guidelines and provides a personalized, patient-focused solution that supports the goals of Industry 5.0 in healthcare. In line with Industry 5.0 principles, the FER AI Toolkit uses edge devices to collect real-time facial data while deep learning models running on cloud servers process this data. The recognized emotions are uploaded to the Semantic Knowledge Graph (SemKG) server. This allows healthcare professionals to make informed decisions based on real-time data. Additionally, the toolkit integrates seamlessly with key components of the ALAMEDA, including the Identity Authentication Manager (IAM) for secure access and the ALAMEDA Innovation Hub (AIH) for efficient resource management. By offering continuous and personalized healthcare insights, the FER AI Toolkit helps bridge the gap between diagnosis and patient well-being, ultimately advancing healthcare systems. Training materials and video demonstrations are available athttps://drive.google.com/drive/folders/1-iUz7FE2IrKHt5nMl3oGjsrCtk7Ps2bM?usp=sharingfor further learning.

In IoT-enabled smart infrastructure, accurate and real-time road damage detection is crucial for enhancing road safety and optimizing maintenance processes. However, detecting road damage in complex and dynamic environments presents significant challenges, such as varying lighting conditions, diverse damage types, and the need for fast processing to enable real-time decision-making. This study introduces an advanced approach utilizing the YOLOv9s-Fusion model to overcome these challenges. Leveraging the RDD2022 dataset, which comprises 1976 annotated images of road damage from China, we employ comprehensive data preprocessing to create optimal conditions for model training. The YOLOv9s-Fusion model integrates innovative features, including a Transformer-based auxiliary module and enhanced feature extraction layers, specifically designed to detect fine-grained damage patterns accurately. Experimental results demonstrate that the model outperforms existing approaches, achieving notable improvements in mean average precision (mAP) and F1-score. Ablation studies further validate the impact of our modifications, highlighting the model’s robustness in real-time detection across diverse conditions. This IoT-centric approach sets a new standard for autonomous road damage detection, significantly advancing vehicle navigation and smart infrastructure management capabilities.

The task of associating photographs/videos of an individual obtained from the same camera on various occasions or across cameras is called Person Re-identification (PRId). Computer-aided monitoring of persons of interest is an active research area in automated visual surveillance. It becomes more substantial in emergencies like natural disasters, contrived incidents, and public health crises. Part-level features of a pedestrian image hold significant importance in person retrieval. Traditionally, part-based PRId tasks required pose estimators or body part detectors for the hard partition of the pedestrian image. However, such approaches attract additional issues due to their dependency on external cues. This article emphasized employing the convolutional partition of body parts to learn discriminative part features. We focus on two significant contributions: (I) A parallel architecture called Convolutional Part Refine (CPR) and (II) Three different convolutional part refine strategies of outliers to handle the existing inconsistencies of uniform partition. The experiments confirm that CPR achieves competitive performance with state-of-the-art methods.Note to Practitioners—This work is motivated by the need to quickly recognize a target person captured across multiple camera views in a crowded environment. Presently, there is no ideal person re-identification solution. This article highlights the future requirements of smart visual surveillance through person detection and re-identification (re-id). The live feed of CCTV cameras can simultaneously detect and re-identify the target person to proactively handle the surveillance issues. The method detects and identifies a person’s identity captured in a CCTV by searching across an extensive database of images known as a gallery set. However, it is difficult to automatically recognize an individual across multiple camera views due to challenging scenarios such as low resolution, occlusion, background clutter, viewpoint, and illumination variations. Thanks to the deep learning-based body-part partition strategies that facilitate learning discriminative features of the target person. Traditionally, hard and soft partition strategies were used to partition the body parts. However, the proposed method focuses on a recent convolutional part partition strategy. Most part-based approaches assume that all the pixels in each part partition are homogeneous. However, the proposed method highlights the within-part-inconsistency problem. Against this background, this paper provides researchers and practitioners with a short review of the part-based body partition strategies. The proposed method demonstrates the effectiveness of convolutional part-partition over the hard and soft partition of body parts over three publicly available benchmark datasets. The proposed process also introduces a refine strategy to reduce the within-part-inconsistency issues in the part partitions.

Deep-learning-based human activity recognition (HAR) methods have significantly transformed a wide range of domains over recent years. However, the adoption of Big Data techniques in industrial applications remains challenging due to issues such as generalized weight optimization, diverse viewpoints, and the complex spatiotemporal features of videos. To address these challenges, this work presents an industrial HAR framework consisting of two main phases. First, a squeeze bottleneck attention block (SBAB) is introduced to enhance the learning capabilities of the backbone model for contextual learning, which allows for the selection and refinement of an optimal feature vector. In the second phase, we propose an effective sequential temporal convolutional network (STCN), which is designed in parallel fashion to mitigate the issues of exploding and vanishing gradients associated with sequence learning. The high-dimensional spatiotemporal feature vectors from the STCN undergo further refinement through our proposed SBAB in a sequential manner, to optimize the features for HAR and enhance the overall performance. The efficacy of the proposed framework is validated through extensive experiments on six datasets, including data from industrial and general activities.

Recent years have witnessed tremendous advancements in Al tools (e.g., ChatGPT, GPT-4, and Bard), driven by the growing power, reasoning, and efficiency of Large Language Models (LLMs). LLMs have been shown to excel in tasks ranging from poem writing and coding to essay generation and puzzle solving. Despite their proficiency in general queries, specialized tasks such as metaphor understanding and fake news detection often require finely tuned models, posing a comparison challenge with specialized Deep Learning (DL). We propose an assessment framework to compare task-specific intelligence with general-purpose LLMs on suicide and depression tendency identification. For this purpose, we trained two DL models on a suicide and depression detection dataset, followed by testing their performance on a test set. Afterward, the same test dataset is used to evaluate the performance of four LLMs (GPT-3.5, GPT-4, Google Bard, and MS Bing) using four classification metrics. The BERT-based DL model performed the best among all, with a testing accuracy of 94.61%, while GPT-4 was the runner-up with accuracy 92.5%. Results demonstrate that LLMs do not outperform the specialized DL models but are able to achieve comparable performance, making them a decent option for downstream tasks without specialized training. However, LLMs outperformed specialized models on the reduced dataset.

Unsupervised person re-identification (ReID) has recently gained significant attention from researchers. ReID matches images of the same person from different camera views in various scenes without any labels. Existing clustering methods primarily rely on a fixed threshold (the maximum distance between sample points and clustering centroids) and overlook the importance of adjusting this threshold during continuous model optimization. This mismatch between clustering thresholds and inter- or intra-class spacing reduces clustering accuracy. To address this issue, this study proposes an Adaptive Clustering and Weighted Regularization Contrastive Learning (ACWRCL) framework for unsupervised person ReID. The ACWRCL framework comprises two main components: (1) the Clustering Threshold Adaptive Adjustment (CTAA) module, and (2) the Weighted Regularization Contrastive Learning (WRCL) module. The CTAA module dynamically adjusts the clustering threshold to align with model optimization, ensuring that the threshold remains within an appropriate range to prevent under- or over-robustness in the clustering model. The WRCL module uses the similarity ratio between the query sample and the clustering centroid relative to the overall similarity of all samples with the same labels as the query sample. This ratio is used as the weight in the loss function to penalize incorrect clustering and improve pseudo-label generation accuracy. Extensive experiments on public ReID datasets—Market-1501, MSMT17, Veri776, CUHK03, and PersonX—demonstrate the effectiveness of the proposed method.

Neural networks have demonstrated remarkable effectiveness in solving distinct real-world vision problems pertaining to activity recognition and violence detection in surveillance scenarios. The broad reliance on practicing a single network for spatial and motion information collection has made them less effective for long-term dependency analysis in video snippets. Our work solves this issue through a multi-network fusion strategy suitable for real-world surveillance. Initially, the spatial information is accessed from a compound coefficient strategy inspired by a robust convolutional neural network (ConvNet). Next, the pyramidal convolutional features from two consecutive frames are obtained through LiteFlowNet. The output from both the networks (ConvNet and LiteFlowNet) is separately passed into a deep-gated recurrent Unit (GRU) that is assembled for a skip connection. The latter obtained from each GRU is fused and further propagated to the dense layer for final decision. The results on the datasets and the ablation study confirm our method’s efficiency, outperforming the state-of-the-art methods. (Code: GitHub)

Reducing the atmospheric haze and enhancing image clarity is crucial for computer vision applications. The lack of real-life hazy ground truth images necessitates synthetic datasets, which often lack diverse haze types, impeding effective haze type classification and dehazing algorithm selection. This research introduces the HazeSpace2M dataset, a collection of over 2 million images designed to enhance dehazing through haze type classification. HazeSpace2M includes diverse scenes with 10 haze intensity levels, featuring Fog, Cloud, and Environmental Haze (EH). Using the dataset, we introduce a technique of haze type classification followed by specialized dehazers to clear hazy images. Unlike conventional methods, our approach classifies haze types before applying type-specific dehazing, improving clarity in real-life hazy images. Benchmarking with state-of-the-art (SOTA) models, ResNet50 and AlexNet achieve 92.75\% and 92.50\% accuracy, respectively, against existing synthetic datasets. However, these models achieve only 80% and 70% accuracy, respectively, against our Real Hazy Testset (RHT), highlighting the challenging nature of our HazeSpace2M dataset. Additional experiments show that haze type classification followed by specialized dehazing improves results by 2.41% in PSNR, 17.14% in SSIM, and 10.2\% in MSE over general dehazers. Moreover, when testing with SOTA dehazing models, we found that applying our proposed framework significantly improves their performance. These results underscore the significance of HazeSpace2M and our proposed framework in addressing atmospheric haze in multimedia processing. Complete code and dataset is available on \href{https://github.com/tanvirnwu/HazeSpace2M} {\textcolor{blue}{\textbf{GitHub}}}.

Action detection and understanding provide the foundation for the generation and interaction of multimedia content. However, existing methods mainly focus on constructing complex relational inference networks, overlooking the judgment of detection effectiveness. Moreover, these methods frequently generate detection results with cognitive abnormalities. To solve the above problems, this study proposes a cognitive effectiveness network based on fuzzy inference (Cefdet), which introduces the concept of 'cognition--based detection' to simulate human cognition. First, a fuzzy-driven cognitive effectiveness evaluation module (FCM) is established to introduce fuzzy inference into action detection. FCM is combined with human action features to simulate the cognition-based detection process, which clearly locates the position of frames with cognitive abnormalities. Then, a fuzzy cognitive update strategy (FCS) is proposed based on the FCM, which utilizes fuzzy logic to re-detect the cognition-based detection results and effectively update the results with cognitive abnormalities. Experimental results demonstrate that Cefdet exhibits superior performance against several mainstream algorithms on the public datasets, validating its effectiveness and superiority.

Digital publishing’s exponential growth has created vast scholarly collections. Guiding researchers to relevant resources is crucial, and knowledge graphs (KGs) are key tools for unlocking hidden knowledge. However, current methods focus on external links between concepts, ignoring the rich information within individual papers. Challenges like insufficient multi-relational data, name ambiguity, and cold-start issues further limit existing KG-based methods, failing to capture the intricate attributes of diverse entities. To solve these issues, we propose GLAMOR, a robust KG framework encompassing entities e.g., authors, papers, fields of study, and concepts, along with their semantic interconnections. GLAMOR uses a novel random walk-based KG text generation method and then fine-tunes the language model using the generated text. Subsequently, the acquired context-preserving embeddings facilitate superior top@k predictions. Evaluation results on two public benchmark datasets demonstrate our GLAMOR’s superiority against state-of-the-art methods especially in solving the cold-start problem.

Wireless capsule endoscopy (WCE) is a revolutionary diagnostic method for small bowel pathology. The manual perusal of the resulting lengthy and redundant videos is cumbersome. Automated analysis of WCE video frames is an intricate data modeling task because of the diverse representations of anomalies caused by inappropriate capture conditions. Deep neural networks require training to learn diverse pathological manifestations utilizing heterogeneous data collected from multiple institutions. However, the accessibility of WCE data poses privacy concerns for multiple centers. The efficient learning of heterogeneous data distributed over multiple institutions in a privacy-preserving fashion has become a challenge hampering the adoption of AI-based diagnoses in clinical practice. Prior studies have contrived extensive data augmentation and the generation of synthetic images from the same center. However, models trained at one center are at risk of a lack of generalization for a global deployment. Federated learning (FL) is a novel paradigm in which models learn from distributed data and share knowledge without accessing the data themselves. This study proposes an FL framework for multiple anomaly classifications of WCE frames, elaborating on the potential of collaborative learning from multiple data centers on the edge. Our empirical results prove that the proposed decentralized approach can learn the generalized features of WCE frames. Validating heterogeneous test sets revealed a 10–12% improvement in performance for decentralized models based on FL compared to the best-case performance of centralized models, demonstrating the potential of the federated framework to support multiple anomaly classification of WCE frames while preserving data privacy across various clinical setups.

For Unmanned Aerial Vehicles (UAVs) with Tiny Machine Learning (TML), there is mutual exclusivity between the energy consumption for flight and the energy consumption to support their computation and processing. IoUAVs integrated with TML systems often consume substantial amounts of energy during flights, particularly when engaged in extended coverage and surveillance missions. The energy consumption of a UAV with TML performing long, wide-area coverage patrols and monitoring missions in complex areas is significant for the flight itself, and the energy required for the TML to perform calculations and processing is not guaranteed. Therefore, to better support TML computations, this study optimizes flight paths to reduce the energy consumption of UAVs while ensuring coverage. Specifically, in this study, the use of concave point elimination algorithms, enhanced convex decomposition algorithms, and determination of flight direction significantly reduced the frequency of UAV turns. The computational cost of obtaining a complete path is reduced by merging the subconvex regions and the weighted minimum traversal of the graph. This novel bidirectional forwarding path coverage path-planning (BFP-CPP) algorithm maximizes the reduction in the number of turns, reduces energy consumption, and achieves global coverage. The simulation experimental results show that compared with the existing methods without concave point elimination, the BFP-CPP algorithm can effectively reduce the number of subregions, minimize the number of drone turns, and lower energy consumption.

Road damage detection (RDD) through computer vision and deep learning techniques can ensure the safety of vehicles and humans on the roads. Integrating unmanned aerial vehicles (UAVs) in RDD and infrastructure evaluation (IE) has also emerged as a key enabler, contributing significantly to data acquisition and real-time monitoring of road damages such as potholes, cracks, and surface anomalies, facilitating proactive maintenance and improved road conditions. These UAVs are low-powered and resource-constrained devices that work autonomously to perform pattern detection and decision-making leveraging tiny machine learning (Tiny ML) algorithms. These Tiny ML algorithms are designed to run on edge devices, IoT devices, UAVs, etc. In this study, the RDD2022 dataset collected using UAVs and dashboard cameras of vehicles was utilized to train pure and mixed models that exhibit class instance imbalance in certain classes which is addressed by implementing data augmentation as a regularization technique. State-of-the-art two-stage detectors; Faster R-CNN ResNet101 and one-stage detectors; SSD MobileNet V1 FPN, YOLOv5, and Efficientdet D1 are employed. The results indicate that the two-stage detector achieved an impressive mAP of 88.49% overall and 96.62% for focused classes. Notably, the state-of-the-art Efficientdet D1 approach achieved a competitive mAP of 86.47% overall and 95.12% for focused classes, with significantly lower computational cost. These findings highlight the potential of advanced object detection techniques, particularly Efficientdet D1, to enhance the accuracy and efficiency of RDD systems, thereby improving passenger safety and overall performance.

Deepfake detection has become increasingly important in recent years owing to the widespread availability of deepfake generation technologies. Existing deepfake detection methods present two primary limitations; i.e., they are trained on a specific type of deepfake dataset, which renders them vulnerable to unseen deepfakes, and they regard deepfakes as a “black box” with limited explainability, making it difficult for non-AI experts to understand and trust the decisions. Hence, this article proposes a novel neurosymbolic deepfake detection framework that exploits the fact that human emotions cannot be imitated easily owing to their complex nature. We argue that deep fakes typically exhibit inter- or intra-modality inconsistencies in the emotional expressions of the person being manipulated. Thus, the proposed framework performs inter- and intra-modality reasoning on emotions extracted from audio and visual modalities using a psychological and arousal-valence model for deepfake detection. In addition to fake detection, the proposed framework provides textual explanations for its decisions. The results obtained using the Presidential Deepfakes Dataset and World Leaders Dataset of real and manipulated videos demonstrate the effectiveness of our approach in detecting deepfakes and highlight the potential of a neurosymbolic approach for expandability.

User counterparts, such as user attributes in social networks or user interests, are the keys to more natural Human–Computer Interaction (HCI) . In addition, users’ attributes and social structures help us understand the complex interactions in HCI. Most previous studies have been based on supervised learning to improve the performance of HCI. However, in the real world, owing to signal malfunctions in user devices, large amounts of abnormal information, unlabeled data, and unsupervised approaches (e.g., the clustering method) based on mining user attributes are particularly crucial. This paper focuses on improving the clustering performance of users’ attributes in HCI and proposes a deep graph embedding network with feature and structure similarity (called DGENFS ) to cluster users’ attributes in HCI applications based on feature and structure similarity. The DGENFS model consists of a Feature Graph Autoencoder (FGA) module, a Structure Graph Attention Network (SGAT) module, and a Dual Self-supervision (DSS) module. First, we design an attributed graph clustering method to divide users into clusters by making full use of their attributes. To take full advantage of the information of human feature space, a k-neighbor graph is generated as a feature graph based on the similarity between human features. Then, the FGA and SGAT modules are utilized to extract the representations of human features and topological space, respectively. Next, an attention mechanism is further developed to learn the importance weights of different representations to effectively integrate human features and social structures. Finally, to learn cluster-friendly features, the DSS module unifies and integrates the features learned from the FGA and SGAT modules. DSS explores the high-confidence cluster assignment as a soft label to guide the optimization of the entire network. Extensive experiments are conducted on five real-world data sets on user attribute clustering. The experimental results demonstrate that the proposed DGENFS model achieves the most advanced performance compared with nine competitive baselines.

Unsupervised person re-identification (Re-ID) has garnered significant attention because of its data-friendly nature, as it does not require labeled data. Existing approaches primarily address this challenge by employing feature-clustering techniques to generate pseudo-labels. In addition, camera-proxy-based methods have emerged because of their impressive ability to cluster sample identities. However, these methods often blur the distinctions between individuals within inter-camera views, which is crucial for effective person re-ID. To address this issue, this study introduces an inter-camera-identity-difference-based contrastive learning framework for unsupervised person Re-ID. The proposed framework comprises two key components: (1) a different sample cross-view close-range penalty module and (2) the same sample cross-view long-range constraint module. The former aims at penalizing excessive similarity among different subjects across inter-camera views, whereas the latter mitigates the challenge of excessive dissimilarity among the same subject across camera views. To validate the performance of our method, we conducted extensive experiments on three existing person Re-ID datasets (Market-1501, MSMT17, and PersonX). The results demonstrate the effectiveness of the proposed method, which shows a promising performance. The code is available at https://github.com/hooldylan/IIDCL .

Authentic Assessments are generally seen as alternate to traditional assessments though the scope of an authentic assessment is much larger than that of a traditional assessment. Authentic assessments help students in comprehending the subject matter and, if properly designed, can also ensure student workplace readiness. During the authentic assessment design, five most important aspects are considered and their incorporation is sought. These aspects include; the assessment objectives, the physical context of an assessment, the social context, the outcome of the assessment and the assessment criteria. Emergence of the Generative Artificial Intelligence (GAI) supported applications and the Large Language Model (LLM) tools has posed new challenges to the authentic assessment design. Student access to these new applications and tools has also changed the socio-technological realities of the Learning and Teaching (L&T) practices. Therefore, we need to reimagine both, the L&T practices and the design and execution of authentic assessments. Keeping the prevailing socio-technological context in perspective, this work in progress paper proposes extending the scope of authentic assessments. The aim is to use them for quelling the growing problem of plagiarism as plagiarism can be facilitated by the use of GAI and LLM tools. Instead of considering authentic assessments as merely ‘an alternate to the traditional examination’ or ‘a tool for evaluating student workplace readiness,’ we propose adding ‘GAI redundancy’ to the scope of authentic assessments. For incorporating GAI redundancy we propose using either the game-based learning environment or a simulation environment. These two environments can be used for generating ‘close to real life’ problem-solving scenarios while assessing student comprehension and workplace readiness. In order to help practitioners, this paper also presents two examples of authentic assessments that were developed for combating plagiarism vis-à-vis enhancing student learning and evaluating their workplace readiness. In the first example, we show how to use a game environment and in the second example we demonstrate use of a simulation environment. The two examples also show how course contents can be embedded and how GAI redundancy can be incorporated in authentic assessments. The reported teaching assessment data and student feedback suggest that the proposed authentic assessment design and implementation strategies were able to engage students, help their comprehension and evaluate their workforce readiness.

Video capsule endoscopy (VCE) is a revolutionary technology for the early diagnosis of gastric disorders. However, owing to the high redundancy and subtle manifestation of anomalies among thousands of frames, the manual construal of VCE videos requires considerable patience, focus, and time. The automatic analysis of these videos using computational methods is a challenge as the capsule is untamed in motion and captures frames inaptly. Several machine learning (ML) methods, including recent deep convolutional neural networks approaches, have been adopted after evaluating their potential of improving the VCE analysis. However, the clinical impact of these methods is yet to be investigated. This survey aimed to highlight the gaps between existing ML-based research methodologies and clinically significant rules recently established by gastroenterologists based on VCE. A framework for interpreting raw frames into contextually relevant frame-level findings and subsequently merging these findings with meta-data to obtain a disease-level diagnosis was formulated. Frame-level findings can be more intelligible for discriminative learning when organized in a taxonomical hierarchy. The proposed taxonomical hierarchy, which is formulated based on pathological and visual similarities, may yield better classification metrics by setting inference classes at a higher level than training classes. Mapping from the frame level to the disease level was structured in the form of a graph based on clinical relevance inspired by the recent international consensus developed by domain experts. Furthermore, existing methods for VCE summarization, classification, segmentation, detection, and localization were critically evaluated and compared based on aspects deemed significant by clinicians. Numerous studies pertain to single anomaly detection instead of a pragmatic approach in a clinical setting. The challenges and opportunities associated with VCE analysis were delineated. A focus on maximizing the discriminative power of features corresponding to various subtle lesions and anomalies may help cope with the diverse and mimicking nature of different VCE frames. Large multicenter datasets must be created to cope with data sparsity, bias, and class imbalance. Explainability, reliability, traceability, and transparency are important for an ML-based diagnostics system in a VCE. Existing ethical and legal bindings narrow the scope of possibilities where ML can potentially be leveraged in healthcare. Despite these limitations, ML based video capsule endoscopy will revolutionize clinical practice, aiding clinicians in rapid and accurate diagnosis.

The main challenge faced by video‐based real‐world anomaly detection systems is the accurate learning of unusual events that are irregular, complicated, diverse, and heterogeneous in nature. Several techniques utilizing deep learning have been created to detect anomalies, yet their effectiveness on real‐world data is often limited due to the insufficient incorporation of motion patterns. To address these problems and enhance the traditional functionality of anomaly detection systems for surveillance video data, we propose a weakly supervised graph neural‐network‐assisted video anomaly detection framework called AD‐Graph. To identify temporal information from a series of frames, we extract 3D visual and motion features and represent these in a language‐based knowledge graph format. Next, a robust clustering strategy is applied to group together meaningful neighbourhoods of the graph with similar vertices. Furthermore, spectral filters are applied to these graphs, and spectral graph theory is used to generate graph signals and detect anomalous events. Extensive experimental results over two challenging datasets, UCF‐Crime and ShanghaiTech, show improvements of 0.35% and 0.78% against a state‐of‐the‐art model.

Deep neural networks (DNNs) are currently being deployed as machine learning technology in a wide range of important real-world applications. DNNs consist of a huge number of parameters that require millions of floating-point operations (FLOPs) to be executed both in learning and prediction modes. A more effective method is to implement DNNs in a cloud computing system equipped with centralized servers and data storage sub-systems with high-speed and high-performance computing capabilities. This paper presents an up-to-date survey on current state-of-the-art deployed DNNs for cloud computing. Various DNN complexities associated with different architectures are presented and discussed alongside the necessities of using cloud computing. We also present an extensive overview of different cloud computing platforms for the deployment of DNNs and discuss them in detail. Moreover, DNN applications already deployed in cloud computing systems are reviewed to demonstrate the advantages of using cloud computing for DNNs. The paper emphasizes the challenges of deploying DNNs in cloud computing systems and provides guidance on enhancing current and new deployments.

As the high-mobility nature of the vehicles results in frequent leaving and joining the transportation network, real-time data must be collected and shared in a timely manner. In such a transportation network, malicious vehicles can disrupt services and create serious issues, such as deadlocks and accidents. The blockchain is a technology that ensures traceability, consistency, and security in transportation networks. In this study, we integrated edge computing and blockchain technology to improve the optimal utilization of resources, especially in terms of computing, communication, security, and storage. We propose a novel, edge-integrated, blockchain-based vehicle platoon security scheme. For the vehicle platoon, we developed the security architecture, implemented smart contracts for practical network scenarios in network simulator version 3, and integrated them with the simulation urban mobility traffic control interface API. We exhaustively simulated all the scenarios and analyzed the communication performance metrics, such as throughput, delay, and jitter, and the security performance metrics, such as mean squared error, communication, and computational cost. The performance results demonstrate that the developed scheme can solve security-related issues more effectively and efficiently in smart cities.

The industrial demands of immersive videos for virtual reality/augmented reality applications are crescendo, where the video stream provides a choice to the user viewing object of interest with the illusion of “being there.” However, in industry 4.0, streaming of such huge-sized video over the network consumes a tremendous amount of bandwidth, where the users are only interested in specific regions of the immersive videos. Furthermore, for delivering full excitement videos and minimizing the bandwidth consumption, the automatic selection of the user’s Region of Interest in a 360° video is very challenging because of subjectivity and difference in contentment. To tackle these challenges, we employ two efficient convolutional neural networks for salient object detection and memorability computation in a unified framework to find the most prominent portion of a 360° video. The proposed system is four-fold: 1) preprocessing; 2) intelligent visual interest predictor; 3) final viewport selection; and 4) virtual camera steerer. First, an input 360° video frame is split into three Field of Views (FoVs), each with a viewing angle of 120°. Next, each FoV is passed to the object detection and memorability prediction model for visual interestingness computation. Furthermore, the FoV is supplied as a viewport, containing the most salient and memorable objects. Finally, a virtual camera steerer is designed using enriched salient features from YOLO and LSTM that are forwarded to the dense optical flow to follow the salient object inside the immersive video. Performance evaluation of the proposed system over our own collected data from various Websites as well as on public data sets indicates the effectiveness for diverse categories of 360° videos and helps in the minimization of the bandwidth usage, making it suitable for industry 4.0 applications.

The main objective of this study is to propose a cyber–physical system (CPS)-based person reidentification (P-ReID) framework for smart surveillance. The Internet of Things (IoT)-based interconnected vision sensors in smart cities are considered essential elements of a CPS, and contribute significantly to urban security. However, the reidentification of targeted persons using emerging edge AI techniques still faces certain challenges. To improve efficiency at the edge and overcome the traditional sensing of video cameras, we employed an AI-based P-ReID framework for CPS that is functional in IoT environments. In addition, we present dual attention dilated network (DADNet), which integrates an energy-efficient convolutional neural network (CNN) with a self-attention module to substantially improve the person matching probability. Furthermore, we applied dual feature fusion to intelligently integrate discriminative and robust features using early and late fusion strategies that allow DADNet to significantly consider the foreground and marginally utilize the background information. Furthermore, we impose diversity orthogonality regularization over several CNN layers, which boosts the performance of DADNet, resulting in an appropriate usage over IoT networks. A comprehensive set of ablation studies, comparison with other state-of-the-art approaches, and a time complexity analysis confirm the strength of our DADNet for reidentification tasks in AI-enabled IoT settings that are well suited for a CPS.

Computer vision has always been a hot field of research by contemporary scholars due to its wide range of applications. As an important branch of this field, the visual monitoring technology has shown superior vitality in the actual monitoring environment of the Internet of Things (IoT). However, when the monitoring environment is complex, once the target monitoring fails, the important information related to the target also disappears. At this time, if the existing monitoring method is used, the target cannot be monitored again. Moreover, the current filtering monitoring algorithm also has the problem of poor interpretability. Therefore, this article combines the relevant characteristics of human inertial thinking when dealing with such problems. First, our method screens the movement information of the target and introduces a fuzzy reasoning mechanism to infer the location area of the target through fuzzy thinking. Then, an alternative selection strategy based on the thinking set is applied, which alternates between the location of thinking reasoning and the location of memory to further obtain the effective visual monitoring of the target. The filtering and monitoring algorithm fused with the new mechanism in the OTB-2015 data set, the UVA123 data set, and the TC128 data set all show that the proposed fuzzy inference mechanism has good robustness and universality. Furthermore, our results confirm that it can not only ensure the monitoring speed and overall accuracy but also improve the stability of monitoring in the IoT-assisted monitoring environment, showing its effectiveness compared to state-of-the-art methods. In addition, our results confirm that the integration of the proposed edge learning method with the IoT can be well applied to the construction of smart cities and future generation systems.

The salient event recognition of soccer matches in the next-generation Internet of Things (Nx-IoT) environment aims to analyze the performance of players/teams by the sports analytics and managerial staff. The embedded Nx-IoT devices carried by the soccer players during the match capture and transmit data to an artificial intelligence (AI)-assisted computing platform. The interconnectivity of data acquisition devices with an AI-assisted computing platform in the Nx-IoT environment will not only allow the spectators to track the formation of their favorite players during a soccer match but will also enable the managerial staff to evaluate the players’ performance in the soccer match as well as in practice sessions. This Nx-IoT-enabled salient event detection feature can be provided to spectators and sports’ managerial staff as a financial technology (FinTech) service. In this article, we propose an efficient deep-learning-based framework for multiperson salient soccer event recognition in IoT-enabled FinTech. The proposed framework performs event recognition in three steps: 1) frames preprocessing; 2) frame-level discriminative features extraction; and 3) high-level events recognition in soccer videos. Moreover, we introduce a new soccer video events (SVE) data set containing videos of six salient events of soccer games. To provide a strong baseline, we evaluate our newly created SVE data set using different traditional machine learning and deep learning algorithms. We also perform event recognition on untrimmed soccer videos using our proposed framework and compare the results with state-of-the-art methods. The obtained results validate the suitability of our proposed framework for salient event recognition in Nx-IoT environments.

Nowadays, emerging sixth-generation (6G) mobile networks, the Internet of Things (IoT), and mobile-edge computing (MEC) technologies have played significant roles in developing a sustainable computing network. In sustainable computing networks, with the increasing scale of data-driven applications, massive privacy-sensitive data are generated. How to effectively process such data on resource-limited IoT devices is challenging. Although edge intelligence (EI) is designed to maintain an appropriate level of ultradelay reliability, low-latency communication (URLLC), real-time data processing, and security and privacy are concerning. In this article, we propose a novel blockchain-supported hierarchical digital twin IoT (HDTIoT) framework, which combines the digital twin to edge network and adopts blockchain technology to achieve secure and reliable real-time computation. We first propose a data and knowledge dual-driven learning solution to ensure real-time interaction and efficient optimization between the physical and the digital worlds. To improve communication and computation efficiency with data and knowledge dual-driven learning, the optimization goal is to minimize the system delay and energy consumption and ensure system reliability and the learning accuracy of IoT devices. Moreover, we propose a proximal policy optimization (PPO)-based multiagent reinforcement learning (MARL) algorithm to solve the resource allocation (RA) problem. Experimental results show that the proposed RA scheme can improve the efficiency of the HDTIoT system, guarantee learning accuracy, reliability, and security, and make a balance between system delay and energy consumption.

No system entity within a contemporary distributed cyber system can be entirely trusted. Hence, the classic centralized trust management method cannot be directly applied to it. Blockchain technology is essential to achieving decentralized trust management, its consensus mechanism is useful in addressing large-scale data sharing and data consensus challenges. Herein, an n-party quantum detectable Byzantine agreement (DBA) based on the GHZ state to realize the data consensus in a quantum blockchain is proposed, considering the threat posed by the growth of quantum information technology on the traditional blockchain. Relying on the nonlocality of the GHZ state, the proposed protocol detects the honesty of nodes by allocating the entanglement resources between different nodes. The GHZ state is notably simpler to prepare than other multi-particle entangled states, thus reducing preparation consumption and increasing practicality. When the number of network nodes increases, the proposed protocol provides better scalability and stronger practicability than the current quantum DBA. In addition, the proposed protocol has the optimal fault-tolerant found and does not rely on any other presumptions. A consensus can be reached even when there are n−2 traitors. The performance analysis confirms viability and effectiveness through exemplification. The security analysis also demonstrates that the quantum DBA protocol is unconditionally secure, effectively ensuring the security of data and realizing data consistency in the quantum blockchain.

Video summarization (VS) suppresses high-dimensional (HD) video data by only extracting the important information. However, prior research has not focused on the need for surveillance VS, that is used for many applications to assist video surveillance experts, including video retrieval and data storage. In addition, mainstream techniques commonly use two-dimensional (2-D) deep models for VS, ignoring event occurrences. Accordingly, we present a two-fold 3-D deep learning-assisted VS framework. First, we employ an inflated 3-D ConvNet model to extract temporal features; these features are optimized using a proposed encoder mechanism. The input video is temporally segmented using a feature comparison technique for selecting a single frame from each video segment. The segmented shots are evaluated using our novel shot segmentation evaluation scheme and are input into a saliency computation mechanism for keyframe selection in a second fold. Qualitative and quantitative analyses over VS benchmarks and surveillance videos demonstrate the superior performance of our framework, with 0.3- and 4.2-unit increases in the F1 scores for YouTube and title-based video summarization datasets, respectively. Along with accurate VS, a key contribution of our study is the novel shot segmentation criterion prior to VS, which can be used as a benchmark in future research to effectively prioritize HD visual data.

Cyberattacks pose a serious threat to the wireless transfer of sensitive healthcare data, hampering the level of privacy offered. Numerous cyber-security modules have been developed. However, many of these methods are unsuitable for real-time medical data processing. In this article, we present a cybersecurity framework developed for medical images in a smart healthcare system. We propose two novel two-dimensional chaotic maps, called the logistic regulated quadratic map (LRQ) and quadratic regulated quadratic map (QRQ), which have compound chaotic properties and a large chaotic range. We present an encryption technique based on the LRQ and QRQ map and demonstrate that the exceedingly chaotic pseudorandom number sequence generated by the maps results in a highly robust cipher image. Our fast cryptosystem can encrypt an image of size 128 × 128 in approximately 0.03 s. The proposed cybersecurity solution protects data against cyberattacks and ensures a seamless treatment experience.

Remote monitoring is an important application of intelligent transportation systems (ITSs). The combination of monitoring equipment and tracking algorithms can be used to automatically track moving targets. The tracking algorithm based on the Siamese network is both accurate and efficient, and its development potential is better than that of other algorithms. Its output is a detection map that reflects the probability that any position in the search area is the center of the target’s bounding box, and the maximum value of the detection map is the center of the target’s bounding box predicted by the algorithm. Owing to partial occlusion, target deformation, out-of-view, and background clutter, local maxima in the detection map may also be the center of the target’s bounding box. A tracker’s ability to make accurate judgments is currently limited. Furthermore, previous trackers extracted only the target features in the initial frame as the matching template. Although this matching template is highly reliable, it cannot effectively combine the target features available in the subsequent frames. Therefore, in this study, fuzzy inference is introduced into the tracking process to analyze the reliability of the detection map. When this map is reliable, the target feature of the search area is transformed into a substitute template; otherwise, multiple substitute templates are selected from the template pool for parallel matching as per the set rules. The optimal result is selected from multiple detection results, based on the priority of the detection results when the initial frame is used as the matching template. Experimental results on multiple datasets show that the proposed algorithm is superior to other similar algorithms in terms of multiple assessment metrics and can improve the robustness of remote monitoring tasks in ITSs.

Rapid developments in deep learning (DL) and the Internet-of-Things (IoT) have enabled vision-based systems to efficiently detect fires at their early stage and avoid massive disasters. Implementing such IoT-driven fire detection systems can significantly reduce the corresponding ecological, social, and economic destruction; they can also provide smart monitoring for intelligent transportation systems (ITSs). However, deploying these systems requires lightweight and cost-effective convolutional neural networks (CNNs) for real-time processing on artificial intelligence (AI)-assisted edge devices. Therefore, in this paper, we propose an efficient and lightweight CNN architecture for early fire detection and segmentation, focusing on IoT-enabled ITS environments. We effectively utilize depth-wise separable convolution, point-wise group convolution, and a channel shuffling strategy with an optimal number of convolution kernels per layer, significantly reducing the model size and computation costs. Extensive experiments on our newly developed and other benchmark fire segmentation datasets reveal the effectiveness and robustness of our approach against state-of-the-art fire segmentation methods. Further, the proposed method maintains a balanced trade-off between the model efficiency and accuracy, making our system more suitable for IoT-driven fire disaster management in ITSs.

In the Cooperative Intelligent Transportation System (C-ITS) paradigm, vehicles could communicate with roadside units to augment their traffic knowledge. Smart roadside units could provide second-order information (e.g., vehicle count) from raw first-order data (e.g., visual feed, point clouds), and this “smart” feature is usually provided using deep neural network models. However, implementing these useful models implies a cost for computational complexity that could hinder the future deployment of smart roadside units needed for sustainability in transportation systems. In this paper, we propose to use model compression on deep image processing models to promote its feasibility for usage in smart sensors. We formulated a controllable convolutional model compression (CCMC) algorithm that can perform filter-wise evolutionary pruning on image processing networks, along with a predefined compression ratio. CCMC is applicable for image processing networks, which have multiple possible traffic data sources (e.g., road camera surveillance). Furthermore, CCMC has a definable target compression ratio that is useful for controlling the trade-off between resource consumption and output performance. We tested our proposed method on depth estimation, which is useful for scene understanding and mapping the locations of objects in the 3D space. Our experiments show that the pruned model has minimal performance discrepancy from the original one, supporting the sustainability features needed for intelligent transportation systems.

The purpose of this study is to investigate Network Car Hailing (NCH) price or the deficiency in NCH Platform in Edge Computing (EC)-based Intelligent Transportation System. Aiming at the uncertain capacity and unbalanced load in the car-hailing platform, this work innovatively introduces the EC to unload, constructs an EC-based online car-hailing resource allocation and pricing optimization model by combining with factors such as the number of users and reputation in the network, and further analyzes the performance of the resource allocation and pricing optimization model in the constructed car-hailing platform through simulation experiments. The experimental results show that with the increase in the number of vehicles with computing tasks, the amount of resources purchased from various car-hailing vehicles also increases, the cost of paying is showing an increasing trend, and the utility function of NCH platforms and operators has declined. In the task resource analysis, the average unloading utility of the algorithm in this work is the highest, and the average unloading utility is basically stable at about 70% when the number of vehicles is 98. With the increase of the delay weight, the delay is smaller and the energy consumption is lower. Therefore, the model constructed in this work can minimize the average cost and consumes less energy while the delay is small. It can provide a reference for intelligent pricing and resource allocation of the online car-hailing platform in the later period of intelligent transportation.

Reliability is an important property in the applied engineering systems, especially in visual tracking. The supervised visual tracking method uses reliable ground truth that is manually annotated, which is hard to get in many applications. However, weakly supervised visual trackings are limited by the low-quality labels. Therefore, a reliable sample selection strategy is the most important issue for the weakly supervised visual trackings. In this article, we propose an optimal sample selection strategy and apply it to the visual tracking system. The strategy first assesses the reliability of the samples according to the score map, where the score map is the pseudolabel generated by the upstream task to meet the needs of the downstream task. Then, the unreliable pseudolabels are replaced by reliable ground truth or discarded to overcome the degraded modeling problem by filtering low-quality samples. Finally, through comparison with multiple selection strategies, it is verified that the model trained using this strategy has the best performance. The proposed visual tracking model achieves the best performance among multiple assessment metrics in multiple datasets. Experiments verify that the scientific sample quality assessment method is very important. It can guide the improvement of model performance, which is of great help to the weakly supervised learning systems based on data.

The dissemination of fake news is one of the most concerning issues in current digital media platforms, originating from the quick and easy spread of unverified information therethrough. Consequently, intense research efforts have been invested towards automating the process of identifying fake news from textual data by means of Artificial Intelligence methods. Among the manifold approaches proposed for this purpose to date, a large fraction of studies have examined the performance of modern deep neural network architectures, mostly relying on pretrained word embeddings and neural processing modules of diverse kind. Unfortunately, such sophisticated Deep Learning methods often require intense computational efforts for training. In this work we explore a novel approach based on randomization-based recurrent neural networks. Specifically, our proposal consists of a weighted ensemble of bidirectional Echo State Networks learned from word sequences processed through pretrained embeddings. Experiments over two fake news detection datasets reveal that competitive detection statistics are obtained by our proposed approach when compared to shallow learning and avant-garde Deep Learning models, but at a dramatically less computational complexity in their training phase.

Person reidentification (P-Reid) is an emerging research domain in the field of information retrieval that has gained exponential growth due to its wide range of applications in pedestrian tracking and crime prevention. The primary goal of P-Reid is to recognize a person based on previous appearance in multiview surveillance videos. The mainstream approaches apply fully supervised learning techniques that have poor scalability when deployed in complex real-world scenes, due to the overfitting problem, caused by the lack of sufficient annotated data. Further, optimization of these models for unlabeled data in real-time surveillance is a challenging task. To tackle these issues, an intelligent framework (LR-Net) is proposed, consisting of three tiers including fine-tuning (FT), siamese network (SN), and fusion strategy (FS). In the first tier, a deep learning model is fine-tuned for P-Reid that can handle both labeled and unlabeled data. Next, with the assistance of transfer learning, an SN is proposed that has a strong discriminative capability in terms of similarity between a pair of images. Finally, a learning-to-rank strategy is applied to optimize the learning capability of the SN, in which a triplet network extracts spatial-temporal patterns from unlabeled samples. In addition, a bayesian fusion model (BFM) is introduced to integrate the spatiotemporal and visual features, which yields 4.4%, 9.3%, and 0.8% improvement in the matching score over Market-1501, DukeMCMT-reID, and CUHK03 data sets, respectively. The conducted experiments and ablation study on the benchmark data sets empirically validate the proposed system, which obtains a high Rank-1 score as compared with the state-of-the-art (SOTA) methods.

Video surveillance has shown encouraging outcomes to monitor human activities and prevent crimes in real time. To this extent, violence detection (VD) has received substantial attention from the research community due to its vast applications, such as ensuring security over public areas and industrial settings through smart machine intelligence. However, because of changing illumination, complex background and low resolution, the analysis of violence patterns remains challenging in the industrial video surveillance domain. In this paper, we propose a computationally intelligent VD approach to precisely detect violent scenes through deep analysis of surveillance video sequential patterns. First, the video stream acquired through the vision sensor is processed by a lightweight convolutional neural network (CNN) for the segmentation of important shots. Next, temporal optical flow features are extracted from the informative shots via a residential optical flow CNN. These are concatenated with appearance-invariant features extracted from a Darknet CNN model. Finally, a multilayer long short-term memory network is plugged to generate the final feature map for learning the violence patterns in a sequence of frames. In addition, we contribute to the existing surveillance VD data set by considering its indoor and outdoor scenarios separately for the proposed method's evaluation, achieving a 2% increase in accuracy over surveillance fight data set. Experiments also show encouraging results over the state of the art on other challenging benchmark data sets.

In the industry 4.0 era, the visualization and real-time automatic monitoring of smart cities supported by the Internet of Things is becoming increasingly important. The use of filtering algorithms in smart city monitoring is a feasible method for this purpose. However, maintaining fast and accurate monitoring in complex surveillance environments with restricted resources remains a major challenge. Since the cognitive theory in visual monitoring is difficult to realize in practice, efficient monitoring of complex environments is accordingly hard to be achieved. Moreover, current monitoring methods do not consider the particularities of the human cognitive system, so the remonitoring ability of the process/target is weak in case of monitoring failure by the monitoring system. To overcome these issues, this article proposes a novel human short-long cognitive memory mechanism for video surveillance in smart cities. In this mechanism, a memory with a high reliability target is used as a “long-term memory,” whereas a memory with a low reliability target is used as a “short-term memory.” During the monitoring process, the “short-term memory” and “long-term memory” alternation strategy is combined with the stored target appearance characteristics, ensuring that the original model in the memory will not be contaminated or mislaid by changes in the external environment (occlusion, fast motion, motion blur, and background clutter). Extensive simulations showcase that the algorithm proposed in this article not only improves the monitoring speed without hindering its real-time operation but also monitors and traces the monitored target accurately, ultimately improving the robustness of the detection in complex scenery, and enabling its application to IoT-assisted smart cities.

Human interaction recognition (HIR) is challenging due to multiple humans’ involvement and their mutual interaction in a single frame, generated from their movements. Mainstream literature is based on three-dimensional (3-D) convolutional neural networks (CNNs), processing only visual frames, where human joints data play a vital role in accurate interaction recognition. Therefore, this article proposes a multistream network for HIR that intelligently learns from skeletons’ key points and spatiotemporal visual representations. The first stream localises the joints of the human body using a pose estimation model and transmits them to a 1-D CNN and bidirectional long short-term memory to efficiently extract the features of the dynamic movements of each human skeleton. The second stream feeds the series of visual frames to a 3-D convolutional neural network to extract the discriminative spatiotemporal features. Finally, the outputs of both streams are integrated via fully connected layers that precisely classify the ongoing interactions between humans. To validate the performance of the proposed network, we conducted a comprehensive set of experiments on two benchmark datasets, UT-interaction and TV human interaction, and found 1.15% and 10.0% improvement in the accuracy.

Analyzing surveillance videos is mandatory for the public and industrial security. Overwhelming growth in computer vision fields has been made to automate the surveillance system in terms of human activity recognition, such as behavior analysis and violence detection (VD). However, it is challenging to detect and analyze the violent scenes intelligently to fulfill the notion of Industrial Internet of Things (IIoT)-based surveillance buoyed by constrained resources to reduce computational power. To tackle this challenge, in this article, an artificial intelligence enabled IIoT-based framework with VD-Network (VD-Net) is proposed. First, the input video frames are passed to light-weight convolutional neural network model for important information collection including humans or suspicious objects such as knives/guns. Upon suspicious object detection, an alert is generated as an earlier VD in IIoT network while the information is shared with concern departments. Only the frames with objects are forwarded to cloud for detail investigation where features are extracted using convolutional long short-term memory (ConvLSTM). The latter from ConvLSTM is propagated to gated recurrent unit for final VD. The conducted experiments and ablation study on the existing surveillance and nonsurveillance datasets empirically validate the effectiveness of the proposed VD-Net by improving 3.9% increase in the accuracy compared with the state-of-the-art VD methods.

The Internet of Vehicles (IoV) is undoubtedly at the core of the future of intelligent transportation. It will prevail over the road ecosystem, and it will have a huge impact on our lives throughout the provision of seamless connectivity among diverse transportation means. For the network to operate efficiently, the data needs to be quickly spread throughout the network, which requires low computational and bandwidth overheads. However, the dynamics of vehicular environments due to frequent node mobility poses many challenges to realize efficient data dissemination. This work addresses this type of problem by proposing a novel clustering algorithm at the edge of the network and an efficient message routing approach, which is known as Group’n Route (GnR). Both mechanisms resort to machine learning and graph metrics that reflect the social relationships between the nodes. Our performance evaluation reveals that the clustering algorithm yields stable results with varying road scenarios, which are becoming an advisable approach in the presence of mobile IoV nodes. Also, the designed routing protocol achieves two orders of magnitude smaller overhead and almost double the delivery rate when it is compared to traditional routing protocols, which thereby justify that the combination of our two proposed clustering and routing methods are a plausible alternative to support IoV communications in real-world setups.

Scene understanding plays a crucial role in autonomous driving by utilizing sensory data for contextual information extraction and decision making. Beyond modeling advances, the enabler for vehicles to become aware of their surroundings is the availability of visual sensory data, which expand the vehicular perception and realizes vehicular contextual awareness in real-world environments. Research directions for scene understanding pursued by related studies include person/vehicle detection and segmentation, their transition analysis, lane change, and turns detection, among many others. Unfortunately, these tasks seem insufficient to completely develop fully-autonomous vehicles i.e., achieving level-5 autonomy, travelling just like human-controlled cars. This latter statement is among the conclusions drawn from this review paper: scene understanding for autonomous driving cars using vision sensors still requires significant improvements. With this motivation, this survey defines, analyzes, and reviews the current achievements of the scene understanding research area that mostly rely on computationally complex deep learning models. Furthermore, it covers the generic scene understanding pipeline, investigates the performance reported by the state-of-the-art, informs about the time complexity analysis of avant garde modeling choices, and highlights major triumphs and noted limitations encountered by current research efforts. The survey also includes a comprehensive discussion on the available datasets, and the challenges that, even if lately confronted by researchers, still remain open to date. Finally, our work outlines future research directions to welcome researchers and practitioners to this exciting domain.

Deep Learning models’ performance strongly correlate with availability of annotated data; however, massive data labeling is laborious, expensive, and error-prone when performed by human experts. Active Learning (AL) effectively handles this challenge by selecting the uncertain samples from unlabeled data collection, but the existing AL approaches involve repetitive human feedback for labeling uncertain samples, thus rendering these techniques infeasible to be deployed in industry related real-world applications. In the proposed Proxy Model based Active Learning technique (PMAL) , this issue is addressed by replacing human oracle with a deep learning model, where human expertise is reduced to label only two small subsets of data for training proxy model and initializing the AL loop. In the PMAL technique, firstly, proxy model is trained with a small subset of labeled data, which subsequently acts as an oracle for annotating uncertain samples. Secondly, active model's training, uncertain samples extraction via uncertainty sampling, and annotation through proxy model is carried out until predefined iterations to achieve higher accuracy and labeled data. Finally, the active model is evaluated using testing data to verify the effectiveness of our technique for practical applications. The correct annotations by the proxy model are ensured by employing the potentials of explainable artificial intelligence. Similarly, emerging vision transformer is used as an active model to achieve maximum accuracy. Experimental results reveal that the proposed method outperforms the state-of-the-art in terms of minimum labeled data usage and improves the accuracy with 2.2%, 2.6%, and 1.35% on Caltech-101, Caltech-256, and CIFAR-10 datasets, respectively. Since the proposed technique offers a highly reasonable solution to exploit huge multimedia data, it can be widely used in different evolutionary industrial domains.

Multi-view action recognition (MVAR) is an optimal technique to acquire numerous clues from different views data for effective action recognition, however, it is not well explored yet. There exist several challenges to MVAR domain such as divergence in viewpoints, invisible regions, and different scales of appearance in each view require better solutions for real world applications. In this paper, we present a conflux long short-term memory (LSTMs) network to recognize actions from multi-view cameras. The proposed framework has four major steps; 1) frame level feature extraction, 2) its propagation through conflux LSTMs network for view self-reliant patterns learning, 3) view inter-reliant patterns learning and correlation computation, and 4) action classification. First, we extract deep features from a sequence of frames using a pre-trained VGG19 CNN model for each view. Second, we forward the extracted features to conflux LSTMs network to learn the view self-reliant patterns. In the next step, we compute the inter-view correlations using the pairwise dot product from output of the LSTMs network corresponding to different views to learn the view inter-reliant patterns. In the final step, we use flatten layers followed by SoftMax classifier for action recognition. Experimental results over benchmark datasets compared to state-of-the-art report an increase of 3% and 2% on northwestern-UCLA and MCAD datasets, respectively.

Green energy management is an economical solution for better energy usage, but the employed literature lacks focusing on the potentials of edge intelligence in controllable Internet of Things (IoT). Therefore, in this article, we focus on the requirements of todays' smart grids, homes, and industries to propose a deep-learning-based framework for intelligent energy management. We predict future energy consumption for short intervals of time as well as provide an efficient way of communication between energy distributors and consumers. The key contributions include edge devices-based real-time energy management via common cloud-based data supervising server, optimal normalization technique selection, and a novel sequence learning-based energy forecasting mechanism with reduced time complexity and lowest error rates. In the proposed framework, edge devices relate to a common cloud server in an IoT network that communicates with the associated smart grids to effectively continue the energy demand and response phenomenon. We apply several preprocessing techniques to deal with the diverse nature of electricity data, followed by an efficient decision-making algorithm for short-term forecasting and implement it over resource-constrained devices. We perform extensive experiments and witness 0.15 and 3.77 units reduced mean-square error (MSE) and root MSE (RMSE) for residential and commercial datasets, respectively.

Multiview video summarization (MVS) has not received much attention from the research community due to inter-view correlations and views' overlapping, etc. The majority of previous MVS works are offline, relying on only summary, and require additional communication bandwidth and transmission time, with no focus on foggy environments. We propose an edge intelligence-based MVS and activity recognition framework that combines artificial intelligence with Internet of Things (IoT) devices. In our framework, resource-constrained devices with cameras use a lightweight CNN-based object detection model to segment multiview videos into shots, followed by mutual information computation that helps in a summary generation. Our system does not rely solely on a summary, but encodes and transmits it to a master device using a neural computing stick for inter-view correlations computation and efficient activity recognition, an approach which saves computation resources, communication bandwidth, and transmission time. Experiments show an increase of 0.4 unit in F-measure on an MVS Office dateset and 0.2% and 2% improved accuracy for UCF-50 and YouTube 11 datesets, respectively, with lower storage and transmission times. The processing time is reduced from 1.23 to 0.45 s for a single frame and optimally 0.75 seconds faster MVS. A new dateset is constructed by synthetically adding fog to an MVS dateset to show the adaptability of our system for both certain and uncertain IoT surveillance environments.

The significant evolution of the Internet of Things (IoT) enabled the development of numerous devices able to improve many aspects in various fields in the industry for smart cities where machines have replaced humans. With the reduction in manual work and the adoption of automation, cities are getting more efficient and smarter. However, this evolution also made data even more sensitive, especially in the industrial segment. The latter has caught the attention of many hackers targeting Industrial IoT (IIoT) devices or networks, hence the number of malicious software, i.e., malware, has increased as well. In this article, we present the IIoT concept and applications for smart cities, besides also presenting the security challenges faced by this emerging area. We survey currently available deep learning (DL) techniques for IIoT in smart cities, mainly deep reinforcement learning, recurrent neural networks, and convolutional neural networks, and highlight the advantages and disadvantages of security-related methods. We also present insights, open issues, and future trends applying DL techniques to enhance IIoT security.

Internet of Medical Things (IoMT)-driven smart health and emotional care is revolutionizing the healthcare industry by embracing several technologies related to multimodal physiological data collection, communication, intelligent automation, and efficient manufacturing. The authentication and secure exchange of electronic health records (EHRs), comprising of patient data collected using wearable sensors and laboratory investigations, is of paramount importance. In this article, we present a novel high payload and reversible EHR embedding framework to secure the patient information successfully and authenticate the received content. The proposed approach is based on novel left data mapping (LDM), pixel repetition method (PRM), RC4 encryption, and checksum computation. The input image of size [Formula: see text] is upscaled by using PRM that guarantees reversibility with lesser computational complexity. The binary secret data are encrypted using the RC4 encryption algorithm and then the encrypted data are grouped into 3-bit chunks and converted into decimal equivalents. Before embedding, these decimal digits are encoded by LDM. To embed the shifted data, the cover image is divided into [Formula: see text] blocks and then in each block, two digits are embedded into the counter diagonal pixels. For tamper detection and localization, a checksum digit computed from the block is embedded into one of the main diagonal pixels. A fragile logo is embedded into the cover images in addition to EHR to facilitate early tamper detection. The average peak signal to noise ratio (PSNR) of the stego-images obtained is 41.95 dB for a very high embedding capacity of 2.25 bits per pixel. Furthermore, the embedding time is less than 0.2 s. Experimental results reveal that our approach outperforms many state-of-the-art techniques in terms of payload, imperceptibility, computational complexity, and capability to detect and localize tamper. All the attributes affirm that the proposed scheme is a potential candidate for providing better security and authentication solutions for IoMT-based smart health.

Energy-efficient communication has become the center of attention from various interdisciplinary fields, such as industrial automation, healthcare, and transportation, among others. Besides, proliferation in the artificial intelligence (AI)-based sixth-generation (6G) technology for achieving the smart automation system has caught the attention of both academia and industry. Most of the intelligent automation systems are formed by IoT-based user terminal (UT) devices for multimedia (i.e., video, audio, image, and text) content delivery with high clarity and efficiency. Customer satisfaction/perception, i.e., Quality of Experience (QoE), is an essential factor to be analyzed because the Quality of Service (QoS) is not a suitable candidate to portray the feelings and expectations of users during multimedia transmission. Therefore, the energy-efficient, entropy-aware communication, and QoE analysis through IoT devices are the dire need. This article focuses on how the energy-efficient communication and user's QoE level can be captured through the UT device during multimedia transmission. Thus first, QoS-based joint energy and entropy optimization (QJEEO) algorithm is proposed. Second, the 6G-driven multimedia data structure model and framework are developed for modeling and evaluation of QoE with acquisition time. Third, the relationship between subjective test score (i.e., surveyed data) and objective performance metrics with mobility/speed of IoT-based devices for multimedia service is established. Fourth, the correlation model is proposed for integrating QoS parameters with estimated QoE perceptions. The experimental results indicate that QoE is modeled and evaluated with acquisition time and correlated with QoS parameter, i.e., packet loss ratio (PLR), and average transfer delay during energy-efficient multimedia transmission in 6G-based networks to improve the satisfaction level of customers.

Typical cloud systems are often prone to inherent wide area network (WAN) latency. To address this issue fog computing is proposed that enables resource-constrained Internet-of-Things (IoT) devices, to execute deadline-sensitive tasks at the edge of the network. These devices can extend their battery lifespan by intelligently offloading computations as tasks to fog nodes (FNs) in their vicinity. However, finding an optimal offloading plan in a densely connected IoT-fog network is proven to beNP-Hard. Hence, in this article, we propose a matching theory-based efficient task offloading strategy called METO that aims to reduce the total system energy and number of outages (number of tasks exceeding the deadline) in an IoT-fog interconnection network. As resource allocation involves multiple criteria, their weights are derived using criteria importance though inter criteria correlation (CRITIC). Furthermore, to rank the alternatives we use the technique for order of preference by similarity to ideal solution (TOPSIS). Based on this ranking, we formulate the overall offloading problem as a one-to-many matching game and utilize the deferred acceptance algorithm (DAA) to produce a stable assignment. Simulation is performed in two different settings comprising offloading of homogeneous and heterogeneous tasks. Extensive simulations across both environments confirm that the proposed algorithm outperforms the existing schemes with respect to improved energy consumption, completion time, and execution time. Moreover, METO also shows the reduced number of outages across baselines used for comparison.

Systems developed to work with computational intelligence have become very efficient, and in some cases obtain more accurate results than evaluations by humans. Hence, this work proposes a new online approach based on deep learning tools according to the concept of transfer learning to generate a computational intelligence framework for use with the Internet of Health Things (IoHT) devices. This framework allows the user to add their images and perform platform training almost as easily as creating folders and placing files in regular cloud storage services. The trials carried out with the tool showed that even people with no programming and image processing knowledge were able to set up projects in a few minutes. The proposed approach is validated using three medical databases, which include cerebral vascular accident images for stroke type classification, lung nodule images for malignant classification, and skin images for the classification of melanocytic lesions. The results show the efficiency and reliability of the framework, which reached 91.6% Accuracy in the stroke images and lung nodules databases, and 92% Accuracy in the skin images databases. This prove the immense contribution that this work can bring to assist medical professionals in analyzing complex examinations quickly and accurately, allowing a large medical examination database through a consolidated collaborative IoT platform.

The rapid development of the Industrial Internet of Things (IIoT) and artificial intelligence (AI) brings new security threats by exposing secret and private data. Thus, information security has become a major concern in the communication environment of IIoT and AI, where security and privacy must be ensured for the messages between a sender and the intended recipient. In this article, we propose a method called Harris hawks optimization-integer wavelet transform (HHO-IWT) for covert communication and secure data in the IIoT environment based on digital image steganography. The method embeds secret data in the cover images using a metaheuristic optimization algorithm called HHO to efficiently select image pixels that can be used to hide bits of secret data within integer wavelet transforms. The HHO-based pixel selection operation uses an objective function evaluation depending on the following two phases: exploitation and exploration. The objective function is employed to determine an optimal encoding vector to transform secret data into an encoded form generated by the HHO algorithm. Several experiments are conducted to validate the performance of the proposed method with respect to visual quality, payload capacity, and security against attacks. The obtained results reveal that the HHO-IWT method achieves higher levels of security than the state-of-the-art methods and that it resists various forms of steganalysis. Thus, utilizing this approach can keep unauthorized individuals away from the transmitted information and solve some security challenges in the IIoT.

Internet and other online media networks have emerged as the most important platforms for the sharing of digital information. However, the readily available editing tools provide an easy way for adversaries to manipulate the data and affect decision-making in various industrial applications. This malicious modification of the content, which has reduced the credibility of information delivery, is a commonly prevalent issue and hence needs serious attention. It also initiates an extreme need for industrial cyber–physical systems (ICPS), which can compare the transferred and received images for correct orientation to ensure that it conveys meaningful information and assists in correct decision-making in industrial automation. In this article, we propose “INDFORG”, which employs a novel and highly accurate automatic rotation angle detection and correction algorithm (ARADC) for intelligent detection of forgery in industrial images. ARADC uses basic geometrical concepts, such as Pythagorean theorem and intensity correlation computation and works without any digital signature or watermark. It performs accurately even under several simultaneous signal-processing manipulations. The proposed framework detects the rotation angles blindly with a 99% accuracy rate for rotation up to ±89°. Experimental results prove that the proposed algorithm is highly efficient compared to various state-of-the-art approaches and is a preferred ICPS for trustworthy media delivery in industrial automation.

The image data received through various sensors are of significant importance in Industry 4.0. Unfortunately, these data are highly vulnerable to various malicious attacks during its transit to the destination. Although the use of pervasive edge computing (PEC) with the Internet of Things (IoT) has solved various issues, such as latency, proximity, and real-time processing, but the security and authentication of data between the nodes is still a significant concern in PEC-based industrial-IoT scenarios. In this article, we present “DWFCAT,” a dual watermarking framework for content authentication and tamper localization for industrial images. The robust and fragile watermarks along with overhead bits related to the cover image for tamper localization are embedded in different planes of the cover image. We have used discrete cosine transform coefficients and exploited their energy compaction property for robust watermark embedding. We make use of a four-point neighborhood to predict the value of a predefined pixel and use it for embedding the fragile watermark bits in the spatial domain. Chaotic and deoxyribonucleic acid encryption is used to encrypt the robust watermark before embedding to enhance its security. The results indicate that DWFCAT can withstand a range of hybrid signal processing and geometric attacks, such as Gaussian noise, salt and pepper, joint photographic experts group (JPEG) compression, rotation, low-pass filtering, resizing, cropping, sharpening, and histogram equalization. The experimental results prove that the DWFCAT is highly efficient compared with the various state-of-the-art approaches for authentication and tamper localization of industrial images.

As a famous visual content perception and processing tool, structure tensor has been widely studied in the past decades. Among them, the anisotropic nonlocal structure tensor (ANLST) has received much attention, recently. However, the existing ANLST calculation methods fail to fully utilize the anisotropic characteristic of the tensor field, thus resulting in limited performance. For this problem, in this article, we present a novel ANLST construction method, by means of combining tensor decomposition with weighted line integral convolution (LIC) with the aim at deeply discovering and exploiting the spatial direction relevancy of the tensors for their regularization. At first, the tensors decomposition, computed by direction projection, yields multiple atomic vector fields, from which, for each point in the tensor field we obtain a family of integral curves that are associated with spatial direction related tensors. Then, LIC is employed with the nonlocal means filtering to smooth the tensors relevant to each integral curve, giving rise to curve-level structure tensor (CLST). At last, a weighted average scheme is carried out on the multiple CLSTs, leading to our proposed weighted anisotropic nonlocal structure tensor (WANST). Experimental results demonstrate that the proposed WANST is superior to the current representative nonlinear structure tensors. The proposed WANST can be applied to industrial surveillance system to enable it perceive image contents, such as flat regions, corners, textures, and edges. In addition, WANST can also help monitoring system improve its image quality.

Due to the rapid development of artificial intelligence technology, industrial sectors are revolutionizing in automation, reliability, and robustness, thereby significantly increasing quality and productivity. Most of the surveillance and industrial sectors are monitored by visual sensor networks capturing different surrounding environment images. However, during tempestuous weather conditions, the visual quality of the images is reduced due to contaminated suspended atmospheric particles that affect the overall surveillance systems. To tackle these challenges, this article presents a computationally efficient lightweight convolutional neural network referred to as Light-DehazeNet (LD-Net) for the reconstruction of hazy images. Unlike other learning-based approaches, which separately measure the transmission map and the atmospheric light, our proposed LD-Net jointly estimates both the transmission map and the atmospheric light using a transformed atmospheric scattering model. Furthermore, a color visibility restoration method is proposed to evade the color distortion in the dehaze image. Finally, we conduct extensive experiments using synthetic and natural hazy images. The quantitative and qualitative evaluation on different benchmark hazy datasets verify the superiority of the proposed method over other state-of-the-art image dehazing techniques. Moreover, additional experimentation validates the applicability of the proposed method in the object detection tasks. Considering the lightweight architecture with minimal computational cost, the proposed system is encouraged to be incorporated as an integral part of the vision-based monitoring systems to improve the overall performance.

Teledermatology is one of the most illustrious applications of telemedicine and e-health. In this field, telecommunication technologies are utilized to transfer medical information to the experts. Due to the skin's visual nature, teledermatology is an effective tool for the diagnosis of skin lesions especially in rural areas. Furthermore, it can also be useful to limit gratuitous clinical referrals and triage dermatology cases. The objective of this research is to classify the skin lesion image samples, received from different servers. The proposed framework is comprised of two module, which include the skin lesion localization/segmentation and the classification. In the localization module, we propose a hybrid strategy that fuses the binary images generated from the designed 16-layered convolutional neural network model and an improved high dimension contrast transform (HDCT) based saliency segmentation. To utilize maximum information extracted from the binary images, a maximal mutual information method is proposed, which returns the segmented RGB lesion image. In the classification module, a pre-trained DenseNet201 model is re-trained on the segmented lesion images using transfer learning. Afterward, the extracted features from the two fully connected layers are down-sampled using the t-distribution stochastic neighbor embedding (t-SNE) method. These resultant features are finally fused using a multi canonical correlation (MCCA) approach and are passed to a multi-class ELM classifier. Four datasets (i.e., ISBI2016, ISIC2017, PH2, and ISBI2018) are employed for the evaluation of the segmentation task, while HAM10000, the most challenging dataset, is used for the classification task. The experimental results in comparison with the state-of-the-art methods affirm the strength of our proposed framework.

Vehicle detection and tracking play an important role in autonomous vehicles and intelligent transportation systems. Adverse weather conditions such as the presence of heavy snow, fog, rain, dust or sandstorm situations are dangerous restrictions on camera’s function by reducing visibility, affecting driving safety. Indeed, these restrictions impact the performance of detection and tracking algorithms utilized in the traffic surveillance systems and autonomous driving applications. In this article, we start by proposing a visibility enhancement scheme consisting of three stages: illumination enhancement, reflection component enhancement, and linear weighted fusion to improve the performance. Then, we introduce a robust vehicle detection and tracking approach using a multi-scale deep convolution neural network. The conventional Gaussian mixture probability hypothesis density filter based tracker is utilized jointly with hierarchical data associations (HDA), which splits into detection-to-track and track-to-track associations. Herein, the cost matrix of each phase is solved using the Hungarian algorithm to compensate for the lost tracks caused by missed detection. Only detection information (i.e., bounding boxes with detection scores) is used in HDA without visual features information for rapid execution. We have also introduced a novel benchmarking dataset designed for research in applications of autonomous vehicles under adverse weather conditions called DAWN. It consists of real-world images collected with different types of adverse weather conditions. The proposed method is tested on DAWN, KITTI, and MS-COCO datasets and compared with 21 vehicle detectors. Experimental results have validated effectiveness of the proposed method which outperforms state-of-the-art vehicle detection and tracking approaches under adverse weather conditions.

Advances in information and signal processing technologies have a significant impact on autonomous driving (AD), improving driving safety while minimizing the efforts of human drivers with the help of advanced artificial intelligence (AI) techniques. Recently, deep learning (DL) approaches have solved several real-world problems of complex nature. However, their strengths in terms of control processes for AD have not been deeply investigated and highlighted yet. This survey highlights the power of DL architectures in terms of reliability and efficient real-time performance and overviews state-of-the-art strategies for safe AD, with their major achievements and limitations. Furthermore, it covers major embodiments of DL along the AD pipeline including measurement, analysis, and execution, with a focus on road, lane, vehicle, pedestrian, drowsiness detection, collision avoidance, and traffic sign detection through sensing and vision-based DL methods. In addition, we discuss on the performance of several reviewed methods by using different evaluation metrics, with critics on their pros and cons. Finally, this survey highlights the current issues of safe DL-based AD with a prospect of recommendations for future research, rounding up a reference material for newcomers and researchers willing to join this vibrant area of Intelligent Transportation Systems.

Autonomous vehicles rely on sophisticated hardware and software technologies for acquiring holistic awareness of their immediate surroundings. Deep learning methods have effectively equipped modern self-driving cars with high levels of such awareness. However, their application requires high-end computational hardware, which makes utilization infeasible for the legacy vehicles that constitute most of today's automotive industry. Hence, it becomes inherently challenging to achieve high performance while at the same time maintaining adequate computational complexity. In this paper, a monocular vision and scalar sensor-based model car is designed and implemented to accomplish autonomous driving on a specified track by employing a lightweight deep learning model. It can identify various traffic signs based on a vision sensor as well as avoid obstacles by using an ultrasonic sensor. The developed car utilizes a single Raspberry Pi as its computational unit. In addition, our work investigates the behavior of economical hardware used to deploy deep learning models. In particular, we herein propose a novel, computationally efficient, and cost-effective approach. The designed system can serve as a platform to facilitate the development of economical technologies for autonomous vehicles that can be used as part of intelligent transportation or advanced driver assistance systems. The experimental results indicate that this model can achieve real-time response on a resource-constrained device without significant overheads, thus making it a suitable candidate for autonomous driving in current intelligent transportation systems.

In the era of rapid development of artificial intelligence, the integration of multimedia and human-artificial intelligence has become an important research hotspot. Especially in the multimedia environment, effective remote visual monitoring has become the exploration direction of many scholars. The use of traditional correlation filtering (CF) algorithm for real-time monitoring in the context of multimedia is a practical strategy. However, most existing filtering-based visual monitoring algorithms still have the problem of insufficient robustness and effectiveness. Therefore, by considering the strategy of updating human memory, this paper proposes a multi-layer template update mechanism to achieve effective monitoring in a multimedia environment. In this strategy, the weighted template of the high-confidence matching memory is used as the confidence memory, and the unweighted template of the low-confidence matching memory is used as the cognitive memory. Through the alternate use of confidence memory, matching memory, and cognitive memory, it is ensured that the target will not be lost during the monitoring process. Experimental results show that this strategy does not affect the speed (still real-time) and improves the robustness in the multimedia background.

Brain tumor is one of the most dangerous cancers in people of all ages, and its grade recognition is a challenging problem for radiologists in health monitoring and automated diagnosis. Recently, numerous methods based on deep learning have been presented in the literature for brain tumor classification (BTC) in order to assist radiologists for a better diagnostic analysis. In this overview, we present an in-depth review of the surveys published so far and recent deep learning-based methods for BTC. Our survey covers the main steps of deep learning-based BTC methods, including preprocessing, features extraction, and classification, along with their achievements and limitations. We also investigate the state-of-the-art convolutional neural network models for BTC by performing extensive experiments using transfer learning with and without data augmentation. Furthermore, this overview describes available benchmark data sets used for the evaluation of BTC. Finally, this survey does not only look into the past literature on the topic but also steps on it to delve into the future of this area and enumerates some research directions that should be followed in the future, especially for personalized and smart healthcare.

In the last decade, twin support vector machine (TWSVM) classifiers have achieved considerable emphasis on pattern classification tasks. However, the TWSVM formulation still suffers from the following two shortcomings: (1) TWSVM deals with the inverse matrix calculation in the Wolfe-dual problems, which is intractable for large-scale datasets with numerous features and samples, and (2) TWSVM minimizes the empirical risk instead of the structural risk in its formulation. With the advent of huge amounts of data today, these disadvantages render TWSVM an ineffective choice for pattern classification tasks. In this article, we propose an efficient large-scale least squares twin support vector machine (LS-LSTSVM) for pattern classification that rectifies all the aforementioned shortcomings. The proposed LS-LSTSVM introduces different Lagrangian functions to eliminate the need for calculating inverse matrices. The proposed LS-LSTSVM also does not employ kernel-generated surfaces for the non-linear case, and thus uses the kernel trick directly. This ensures that the proposed LS-LSTSVM model is superior to the original TWSVM and LSTSVM. Lastly, the structural risk is minimized in LS-LSTSVM. This exhibits the essence of statistical learning theory, and consequently, classification accuracy on datasets can be improved due to this change. The proposed LS-LSTSVM is solved using the sequential minimal optimization (SMO) technique, making it more suitable for large-scale problems. We further proved the convergence of the proposed LS-LSTSVM. Exhaustive experiments on several real-world benchmarks and NDC-based large-scale datasets demonstrate that the proposed LS-LSTSVM is feasible for large datasets and, in most cases, performed better than existing algorithms.

One-shot image recognition has been explored for many applications in computer vision community. However, its applications in video analytics is not deeply investigated yet. For instance, surveillance anomaly recognition is an open challenging problem and one of its hurdles is the lack of accurate temporally annotated data. This paper addresses the lack of data issue using one-shot learning strategy and proposes an anomaly recognition framework which exploits a 3D CNN siamese network that yields the similarity between two anomaly sequences. This paper also investigates the existing 3D CNNs for this task and then proposes a lightweight 3D CNN model that efficiently handles one-shot anomaly recognition. Once our network is trained, then we can use the powerful discriminative 3D CNN features to predict anomalies not only for the new data but also for entirely new classes. The proposed model is trained using temporally annotated test set of UCF Crime dataset. Finally, the trained model is used to recognize the anomalies and produce temporal automatic labels for the video level weakly annotated training set of the dataset.

Video summarization (VS) has attracted intense attention recently due to its enormous applications in various computer vision domains, such as video retrieval, indexing, and browsing. Traditional VS researches mostly target at the effectiveness of the VS algorithms by introducing the high quality of features and clusters for selecting representative visual elements. Due to the increased density of vision sensors network, there is a tradeoff between the processing time of the VS methods with reasonable and representative quality of the generated summaries. It is a challenging task to generate a video summary of significant importance while fulfilling the needs of Internet of Things (IoT) surveillance networks with constrained resources. This article addresses this problem by proposing a new computationally effective solution through designing a deep CNN framework with hierarchical weighted fusion for the summarization of surveillance videos captured in IoT settings. The first stage of our framework designs discriminative rich features extracted from deep CNNs for shot segmentation. Then, we employ image memorability predicted from a fine-tuned CNN model in the framework, along with aesthetic and entropy features to maintain the interestingness and diversity of the summary. Third, a hierarchical weighted fusion mechanism is proposed to produce an aggregated score for the effective computation of the extracted features. Finally, an attention curve is constituted using the aggregated score for deciding outstanding keyframes for the final video summary. Experiments are conducted using benchmark data sets for validating the importance and effectiveness of our framework, which outperforms the other state-of-the-art schemes.

As one of the next-generation network technologies for data centers, wireless data center networks have important research significance. Smart architecture optimization and management are vital for wireless data center networks. With the ever-increasing demand for data center resources, the deployment of the data servers are on the rise. However, traditional wired links among servers are expensive and inflexible. Benefitting from the development of intelligent optimization and other techniques, this article studies a high-speed wireless topology for wireless data center networks. A radio propagation model based on a heat map is constructed. The line-of-sight issue and the interference problem are also discussed. By simultaneously considering the objectives of coverage, propagation intensity, and interference intensity, as well as the constraint of connectivity, the topology optimization problem is formulated as a multiobjective optimization problem. To seek the solutions, several state-of-the-art serial multiobjective evolutionary algorithms (MOEAs), as well as parallel MOEAs, are employed. Prior knowledge is preferred for the grouping, and parameter adaptation is conducted in the distributed parallel algorithms. Experimental results demonstrate that the parallel MOEAs perform effectively in the optimization results and efficiently in time consumption.

Nowadays, video sensors are used on a large scale for various applications, including security monitoring and smart transportation. However, the limited communication bandwidth and storage constraints make it challenging to process such heterogeneous nature of Big Data in real time. Multiview video summarization (MVS) enables us to suppress redundant data in distributed video sensors settings. The existing MVS approaches process video data in offline manner by transmitting them to the local or cloud server for analysis, which requires extra streaming to conduct summarization, huge bandwidth, and are not applicable for integration with industrial Internet of Things (IIoT). This article presents a light-weight convolutional neural network (CNN) and IIoT-based computationally intelligent (CI) MVS framework. Our method uses an IIoT network containing smart devices, Raspberry Pi (RPi) (clients and master) with embedded cameras to capture multiview video data. Each client RPi detects target in frames via light-weight CNN model, analyzes these targets for traffic and crowd density, and searches for suspicious objects to generate alert in the IIoT network. The frames of each client RPi are encoded and transmitted with approximately 17.02% smaller size of each frame to master RPi for final MVS. Empirical analysis shows that our proposed framework can be used in industrial environments for various applications such as security and smart transportation and can be proved beneficial for saving resources.11[Online]. Available: https://github.com/tanveer-hussain/Embedded-Vision-for-MVS.

The massive amount of video data produced by surveillance networks in industries instigate various challenges in exploring these videos for many applications, such as video summarization (VS), analysis, indexing, and retrieval. The task of multiview video summarization (MVS) is very challenging due to the gigantic size of data, redundancy, overlapping in views, light variations, and interview correlations. To address these challenges, various low-level features and clustering-based soft computing techniques are proposed that cannot fully exploit MVS. In this article, we achieve MVS by integrating deep neural network based soft computing techniques in a two-tier framework. The first online tier performs target-appearance-based shots segmentation and stores them in a lookup table that is transmitted to cloud for further processing. The second tier extracts deep features from each frame of a sequence in the lookup table and pass them to deep bidirectional long short-term memory (DB-LSTM) to acquire probabilities of informativeness and generates a summary. Experimental evaluation on benchmark dataset and industrial surveillance data from YouTube confirms the better performance of our system compared to the state-of-the-art MVS methods.

The exponential growth in the production of video contents in different industries causes an urgent need for effective video summarization (VS) techniques, in order to get an optimal storage and preservation of key information in the video. Compared to other domains, industrial videos are more challenging to process, as they usually contain diverse and complex events, which make their online processing a difficult task. In this article, we introduce an online system for intelligent video capturing, coarse and fine redundancy removal, and summary generation. First, we capture video data through resource-constrained devices in an industrial Internet of Things network, equipped with vision sensors and apply coarse redundancy removal through the comparison of low-level features. Second, we transmit the resulting frames to the cloud for detailed analysis, where sequential features are extracted for the selection of candidate keyframes. Finally, we refine the candidate keyframes in order to discriminate those with maximum information as part of the summary. The key contributions of this article include the coarse and fine refining of video data implemented over resource-restricted devices and the presentation of important data in the form of a summary. Experiments11[Online]. Available: https://github.com/tanveer-hussain/DeepRes-Video-Summarization. over publicly available datasets evince a 0.3-unit increase in the F1 score when compared to state-of-the-art and with reduced time complexity. Furthermore, we provide convincing results on our newly created dataset in an industrial environment, which is made publicly available for the research community along with its labeled ground truth.

Smoke detection in foggy surveillance environments is a challenging task and plays a key role in disaster management for industrial systems. The current smoke detection methods are applicable to only normal surveillance videos, providing unsatisfactory results for video streams captured from foggy environments, due to challenges related to clutter and unclear contents. In this paper, an energy-friendly edge intelligence-assisted smoke detection method is proposed using deep convolutional neural networks for foggy surveillance environments. Our method uses a light-weight architecture, considering all necessary requirements regarding accuracy, running time, and deployment feasibility for smoke detection in an industrial setting, compared to other complex and computationally expensive architectures including AlexNet, GoogleNet, and visual geometry group (VGG). Experiments are conducted on available benchmark smoke detection datasets, and the obtained results show better performance of the proposed method over state-of-the-art for early smoke detection in foggy surveillance.

Training gene expression data with supervised learning approaches can provide an alarm sign for early treatment of lung cancer to decrease death rates. However, the samples of gene features involve lots of noises in a realistic environment. In this study, we present a random forest with self-paced learning bootstrap for improvement of lung cancer classification and prognosis based on gene expression data. To be specific, we propose an ensemble learning with random forest approach to improving the model classification performance by selecting multi-classifiers. Then, we investigate the sampling strategy by gradually embedding from high- to low-quality samples by self-paced learning. The experimental results based on five public lung cancer datasets show that our proposed method could select significant genes exactly, which improves classification performance compared to that of existing approaches. We believe that our proposed method has the potential to assist doctors in gene selections and lung cancer prognosis.

SCADA (Supervisory Control and Data Acquisition) networks are built to efficiently provide supervisory and control of national and international critical infrastructures. SCADA networks represent a challenging domain for forensic investigators who have the responsibility to discover the main causes of the catastrophic incidents that could happen in these critical mission systems and provide precise and logical evidences supported with comprehensive technical reports to the legal organizations. They urgently need technological tools and frameworks that enable them to effectively do their mission without affecting the running state of SCADA networks which must be sustainable and robust against technical and disruptive incidents. This paper discusses the challenges and opportunities towards achieving that goal and highlights the emerging technological approaches and paradigms that can be considered as promising for the realization of such a framework taking into account the efficient consumption of computational resources. Further, this paper proposes a conceptual framework for automated and secure forensic investigation in modern complex SCADA networks accompanied with a possible realization architecture based on the Multi-Agent Systems (MAS) and Wireless Sensor Networks (WSN) promising technological paradigms. The proposed framework is intentionally designed to be compliant with the currently active motivation towards promoting green computing requirements.

Nowadays the widespread adoption of cryptocurrencies (also referred to as Altcoins) has universalized the access of the society to trading opportunities in alternative markets, thereby laying a rich substrate for the development of new applications and services aimed at easing the management of personal investment portfolios. When selecting how much to invest and in which asset it is often the case that multiple criteria conflict with each other within a single decision making process, which calls for efficient means to optimally balance such contradicting objectives. In this paper we report initial findings around the combination of Deep Learning (DL) models and Multi-Objective Evolutionary Algorithms (MOEAs) for allocating cryptocurrency portfolios. Technical rationale and details are given on the design of a stacked DL recurrent neural network, and how its predictive power can be exploited for yielding accurate ex ante estimates of the return and risk of the portfolio. These two objectives are complemented by a measure of the diversity of the investment. Results are presented and discussed with real cryptocurrency data, showcasing the potential of our technical approach to produce near-optimal portfolios by balancing the aforementioned objectives. Our study stimulates further research towards incorporating other factors in the design of predictive portfolios, such as the confidence of the DL model output.

Smoke detection in Internet of Things (IoT) environment is a primary component of early disaster-related event detection in smart cities. Recently, several smoke and fire detection methods are presented with reasonable accuracy and running time for normal IoT environment. However, these methods are unable to detect smoke in foggy IoT environment, which is a challenging task. In this paper, we propose an energy-efficient system based on deep convolutional neural networks for early smoke detection in both normal and foggy IoT environments. Our method takes advantage of VGG-16 architecture, considering its sensible stability between the accuracy and time efficiency for smoke detection compared to the other computationally expensive networks, such as GoogleNet and AlexNet. Experiments performed on benchmark smoke detection datasets and their results in terms of accuracy, false alarms rate, and efficiency reveal the better performance of our technique compared to state-of-the-art and verifies its applicability in smart cities for early detection of smoke in normal and foggy IoT environments.

The advanced computational capabilities of many resource constrained devices, such as smartphones have enabled various research areas including image retrieval from big data repositories for numerous Internet of Things (IoT) applications. The major challenges for image retrieval using smartphones in an IoT environment are the computational complexity and storage. To deal with big data in IoT environment for image retrieval, this paper proposes a light-weighted deep learning-based system for energy-constrained devices. The system first detects and crops face regions from an image using Viola-Jones algorithm with additional face and nonface classifier to eliminate the miss-detection problem. Second, the system uses convolutional layers of a cost effective pretrained CNN model with defined features to represent faces. Next, features of the big data repository are indexed to achieve a faster matching process for real-time retrieval. Finally, Euclidean distance is used to find similarity between query and repository images. For experimental evaluation, we created a local facial images dataset, including both single and group facial images. This dataset can be used by other researchers as a benchmark for comparison with other real-time facial image retrieval systems. The experimental results show that our proposed system outperforms other state-of-the-art feature extraction methods in terms of efficiency and retrieval for IoT-assisted energy-constrained platforms.

Tactile Internet can combine multiple technologies by enabling intelligence via mobile edge computing and data transmission over a 5G network. Recently, several convolutional neural networks (CNN) based methods via edge intelligence are utilized for fire detection in certain environment with reasonable accuracy and running time. However, these methods fail to detect fire in uncertain Internet of Things (IoT) environment having smoke, fog, and snow. Furthermore, achieving good accuracy with reduced running time and model size is challenging for resource constrained devices. Therefore, in this paper, we propose an efficient CNN based system for fire detection in videos captured in uncertain surveillance scenarios. Our approach uses light-weight deep neural networks with no dense fully connected layers, making it computationally inexpensive. Experiments are conducted on benchmark fire datasets and the results reveal the better performance of our approach compared to state-of-the-art. Considering the accuracy, false alarms, size, and running time of our system, we believe that it is a suitable candidate for fire detection in uncertain IoT environment for mobile and embedded vision applications during surveillance.

Due to large volume and high variability of editing tools, protecting multimedia contents, and ensuring their privacy and authenticity has become an increasingly important issue in cyber-physical security of industrial environments, especially industrial surveillance. The approaches authenticating images using their principle content emerge as popular authentication techniques in industrial video surveillance applications. But maintaining a good tradeoff between perceptual robustness and discriminations is the key research challenge in image hashing approaches. In this paper, a robust image hashing method is proposed for efficient authentication of keyframes extracted from surveillance video data. A novel feature extraction strategy is employed in the proposed image hashing approach for authentication by extracting two important features: the positions of rich and nonzero low edge blocks and the dominant discrete cosine transform (DCT) coefficients of the corresponding rich edge blocks, keeping the computational cost at minimum. Extensive experiments conducted from different perspectives suggest that the proposed approach provides a trustworthy and secure way of multimedia data transmission over surveillance networks. Further, the results vindicate the suitability of our proposal for real-time authentication and embedded security in smart industrial applications compared to state-of-the-art methods.

Convolutional neural networks (CNNs) have yielded state-of-the-art performance in image classification and other computer vision tasks. Their application in fire detection systems will substantially improve detection accuracy, which will eventually minimize fire disasters and reduce the ecological and social ramifications. However, the major concern with CNN-based fire detection systems is their implementation in real-world surveillance networks, due to their high memory and computational requirements for inference. In this paper, we propose an original, energy-friendly, and computationally efficient CNN architecture, inspired by the SqueezeNet architecture for fire detection, localization, and semantic understanding of the scene of the fire. It uses smaller convolutional kernels and contains no dense, fully connected layers, which helps keep the computational requirements to a minimum. Despite its low computational needs, the experimental results demonstrate that our proposed solution achieves accuracies that are comparable to other, more complex models, mainly due to its increased depth. Moreover, this paper shows how a tradeoff can be reached between fire detection accuracy and efficiency, by considering the specific characteristics of the problem of interest and the variety of fire data.

Exponential growth of multimedia data has been witnessed in recent years from various industries, such as e-commerce, health, transportation, and social networks, etc. Access to desired data in such gigantic datasets require sophisticated and efficient retrieval methods. In the last few years, neuronal activations generated by a pretrained convolutional neural network (CNN) have served as generic descriptors for various tasks including image classification, object detection and segmentation, and image retrieval. They perform incredibly well compared to hand-crafted features. However, these features are usually high dimensional, requiring a lot of memory and computations for indexing and retrieval. For very large datasets, utilization of these high dimensional features in raw form becomes infeasible. In this paper, a highly efficient method is proposed to transform high dimensional deep features into compact binary codes using bidirectional Fourier decomposition. This compact bit code saves memory and eases computations during retrieval. Further, these codes can also serve as hash codes, allowing very efficient access to images in large datasets using approximate nearest neighbor (ANN) search techniques. Our method does not require any training and achieves considerable retrieval accuracy with short length codes. It has been tested on features extracted from fully connected layers of a pretrained CNN. Experiments conducted with several large datasets reveal the effectiveness of our approach for a wide variety of datasets.

This paper proposes a secure surveillance framework for Internet of things (IoT) systems by intelligent integration of video summarization and image encryption. First, an efficient video summarization method is used to extract the informative frames using the processing capabilities of visual sensors. When an event is detected from keyframes, an alert is sent to the concerned authority autonomously. As the final decision about an event mainly depends on the extracted keyframes, their modification during transmission by attackers can result in severe losses. To tackle this issue, we propose a fast probabilistic and lightweight algorithm for the encryption of keyframes prior to transmission, considering the memory and processing requirements of constrained devices that increase its suitability for IoT systems. Our experimental results verify the effectiveness of the proposed method in terms of robustness, execution time, and security compared to other image encryption algorithms. Furthermore, our framework can reduce the bandwidth, storage, transmission cost, and the time required for analysts to browse large volumes of surveillance data and make decisions about abnormal events, such as suspicious activity detection and fire detection in surveillance applications.