This research explores the positive application of deepfake technology for upper body generation, specifically sign language for the D(d)eaf and hard of hearing (DHoH) community. Given the complexity of sign language and the scarcity of experts, the generated videos are vetted by a sign language expert for accuracy. We construct a reliable deepfake dataset, evaluating its technical and visual credibility using computer vision and natural language processing models. The dataset, consisting of over 1200 videos featuring both seen and unseen individuals to the generation model, is also used to detect deepfake videos targeting vulnerable individuals. Expert annotations confirm that the generated videos are comparable to real sign language content. Linguistic analysis, using textual similarity scores and interpreter evaluations, shows that the interpretation of generated videos is at least 90% similar to authentic sign language. Visual analysis demonstrates that convincingly realistic deepfakes can be produced, even for new subjects. Using a pose/style transfer model, we pay close attention to detail, ensuring hand movements are accurate and align with the driving video. We also apply machine learning algorithms to establish a baseline for deepfake detection on this dataset, contributing to the detection of fraudulent sign language videos.

Adaptive video streaming in cellular networks faces challenges due to the dynamic nature of network conditions, particularly in environments with varying mobility patterns. This study introduces a novel multi-stage mobility-aware throughput prediction engine (MATH-P) designed to enhance the Quality of Experience (QoE) in adaptive bitrate (ABR) streaming. MATH-P integrates a classification-driven model selection mechanism that dynamically adjusts to different mobility scenarios, such as stationary, vehicular, and high-speed mobility, allowing for precise throughput predictions. By replacing the conventional throughput estimation module in existing ABR systems with MATH-P, our experiments demonstrate significant improvements across key QoE models and streaming performance metrics across different mobility scenarios. The results highlight MATH-P's ability to deliver smoother video playback, reduced interruptions, and better overall streaming quality. The comprehensive evaluation shows that MATH-P outperforms traditional estimation methods, making it a robust solution for enhancing video streaming in mobility-rich environments.

The rapid surge of text-to-speech and face-voice reenactment models makes video fabrication easier and highly realistic. To encounter this problem, we require datasets that rich in type of generation methods and perturbation strategy which is usually common for online videos. To this end, we propose AV-Deepfake1M++, an extension of the AV-Deepfake1M having 2 million video clips with diversified manipulation strategy and audio-visual perturbation. This paper includes the description of data generation strategies along with benchmarking of AV-Deepfake1M++ using state-of-the-art methods. We believe that this dataset will play a pivotal role in facilitating research in Deepfake domain. Based on this dataset, we host the 2025 1M-Deepfakes Detection Challenge. The challenge details, dataset and evaluation scripts are available online under a research-only license at https://deepfakes1m.github.io/2025.

E-commerce platforms face the critical challenge of adversary events, including fraudulent transactions and fake reviews, which can lead to significant financial and reputational damage. Addressing this, our research introduces a hybrid Deep Learning model, tailored for the detection of such adversarial activities. This innovative approach leverages spatial and sequential data processing capabilities, enhancing the identification of subtle adversarial manipulations across diverse e-commerce contexts. Our findings indicate a high detection rate of 93 percent for adversarial attacks, with precision, recall, and Matthews Correlation Coefficient metrics underscoring the model’s efficacy. This work significantly contributes to e-commerce security by advancing the robustness of detection systems against a spectrum of adversarial threats, including account takeovers and deceptive reviews. While demonstrating a notable improvement over existing methods, our research also acknowledges the potential for evasion by sophisticated attacks, highlighting areas for future work in enhancing model resilience. This balance of innovation and critical insight provides a solid foundation for further advancements in the field of e-commerce security

With the ever-growing power of generative artificial intelligence, deepfake and artificially generated (synthetic) media have continued to spread online, which creates various ethical and moral concerns regarding their usage. To tackle this, we thus present a novel exploration of the trends and patterns observed in real, deepfake and synthetic facial images. The proposed analysis is done in two parts: firstly, we incorporate eight deep learning models and analyze their performances in distinguishing between the three classes of images. Next, we look to further delve into the similarities and differences between these three sets of images by investigating their image properties both in the context of the entire image as well as in the context of specific regions within the image. ANOVA test was also performed and provided further clarity amongst the patterns associated between the images of the three classes. From our findings, we observe that the investigated deep-learning models found it easier to detect synthetic facial images, with the ViT Patch-16 model performing best on this task with a class-averaged sensitivity, specificity, precision, and accuracy of 97.37%, 98.69%, 97.48%, and 98.25%, respectively. This observation was supported by further analysis of various image properties. We saw noticeable differences across the three category of images. This analysis can help us build better algorithms for facial image generation, and also shows that synthetic, deepfake and real face images are indeed three different classes.

The detection and localization of deepfake content, particularly when small fake segments are seamlessly mixed with real videos, remains a significant challenge in the field of digital media security. Based on the recently released AV-Deepfake1M dataset, which contains more than 1 million manipulated videos across more than 2,000 subjects, we introduce the 1M-Deepfakes Detection Challenge. This challenge is designed to engage the research community in developing advanced methods for detecting and localizing deepfake manipulations within the large-scale high-realistic audio-visual dataset. The participants can access the AV-Deepfake1M dataset and are required to submit their inference results for evaluation across the metrics for detection or localization tasks. The methodologies developed through the challenge will contribute to the development of next-generation deepfake detection and localization systems. Evaluation scripts, baseline models, and accompanying code will be available on https://github.com/ControlNet/AV-Deepfake1M.

The integration of various computational paradigms including the Internet of Things (IoT), and Edge-Cloud platforms, have the potential to enhance the efficiency of evacuation during emergencies. Conspicuously, this study presents an intelligent evacuation framework by integrating the IoT-Edge-Cloud (IEC) computing paradigm. The proposed framework utilizes IoT technology to collect ambient data and track occupant movement based on the location. Edge computing involves the incorporation of a Support Vector Machine (SVM) to identify emergency events. Additionally, it facilitates real-time Spatio-temporal data processing. Furthermore, cloud computing enables the implementation of an evacuation algorithm for efficiently computing a secure and expeditious path based on environmental and occupant data. The presented algorithm generates a comprehensive evacuation map, which serves as a guidance tool to direct individuals toward the designated exit point. Based on experimental simulations, enhanced results were obtained for performance enhancement in terms of Temporal delay (5.23s), Decision-making Efficiency (Precision (95.56%), Sensitivity (96.44%), Specificity (96.97%), F-Measure (96.69%)), Energy Efficiency (4.56mJ), Reliability (92.69%) and Stability (73%).

Abstract Mental stress is a common problem that affects individuals all over the world. Stress reduces human functionality during routine work and may lead to severe health defects. Early detection of stress is important for preventing diseases and other negative health-related consequences of stress. Several neuroimaging techniques have been utilized to assess mental stress, however, due to its ease of use, robustness, and non-invasiveness, electroencephalography (EEG) is commonly used. This paper aims to fill a knowledge gap by reviewing the different EEG-related deep learning algorithms with a focus on Convolutional Neural Networks (CNNs) and Long Short-Term Memory networks (LSTMs) for the evaluation of mental stress. The review focuses on data representation, individual deep neural network model architectures, hybrid models, and results amongst others. The contributions of the paper address important issues such as data representation and model architectures. Out of all reviewed papers, 67% used CNN, 9% LSTM, and 24% hybrid models. Based on the reviewed literature, we found that dataset size and different representations contributed to the performance of the proposed networks. Raw EEG data produced classification accuracy around 62% while using spectral and topographical representation produced up to 88%. Nevertheless, the roles of generalizability across different deep learning models and individual differences remain key areas of inquiry. The review encourages the exploration of innovative avenues, such as EEG data image representations concurrently with graph convolutional neural networks (GCN), to mitigate the impact of inter-subject variability. This novel approach not only allows us to harmonize structural nuances within the data but also facilitates the integration of temporal dynamics, thereby enabling a more comprehensive assessment of mental stress levels.

Workers with demanding jobs are at risk of experiencing mental stress, leading to decreased performance, mental illness, and disrupted sleep. To detect elevated stress levels in the workplace, studies have explored stress measurement from physiological, psychological, and behavioral perspectives. This paper reviews the assessment methods and strategies for mitigating mental stress in the workplace and provides recommendations for early detection and mitigation of mental stress. Among the modalities, Electroencephalography (EEG), Electrocardiography (ECG) and Galvanic Skin Response (GSR) were found to be the most used in assessing mental stress in the workplace. Nevertheless, these modalities are sensitive to motion artifacts and are difficult to be integrated into real work environments. To further improve stress level assessment in the workplace, multimodality integration with a reduced number of sensors such as EEG, GSR and Functional near infrared spectroscopy (fNIRS) can be utilized. This would lead to developing strategies for stress management in real-time. Furthermore, combining EEG with fNIRS would improve source localization of mental stress. To mitigate stress, we recommend developing a closed loop system that incorporates brain data acquisition systems and machine learning with physical stimulations such as audio Binaural Beats Stimulation and/or Transcranial Electric Stimulation.

Detection of human relationships from their interactions on social media is a challenging problem with a wide range of applications in different areas, like targeted marketing, cyber-crime, fraud, defense, planning, and human resource, to name a few. All previous work in this area has only dealt with the most basic types of relationships. The proposed approach goes beyond the previous work to efficiently handle the hierarchy of social relationships. This article introduces a novel technique named Quantifiable Social Relationship (QSR) analysis for quantifying social relationships to analyze relationships between agents from their textual conversations. QSR uses cross-disciplinary techniques from computational linguistics and cognitive psychology to identify relationships. QSR utilizes sentiment and behavioral styles displayed in the conversations for mapping them onto level II relationship categories. Then, for identifying the level III relationship categories, QSR uses level II relationships, sentiments, interactions, and word embeddings as key features. QSR employs natural language processing techniques for feature engineering and state-of-the-art embeddings generated by word2vec, global vectors (glove), and bidirectional encoder representations from transformers (bert). QSR combines the intrinsic conversational features with word embeddings for classifying relationships. QSR achieves an accuracy of up to 89% for classifying relationship subtypes. The evaluation shows that QSR can accurately identify the hierarchical relationships between agents by extracting intrinsic and extrinsic features from textual conversations between agents.

In this work we propose additions to the COSMOS and COSMOS on Steroids pipelines for the detection of Cheapfakes for Task 1 of the ACM Grand Challenge for Detecting Cheapfakes. We compute sentiment features, namely polarity and subjectivity, using the news image captions. Multiple logistic regression results show that these sentiment features are significant in prediction of the outcome. We then combine the sentiment features with the four image-text features obtained in the aforementioned previous works to train an MLP. This classifies sets of inputs into being out-of-context (OOC) or not-out-of-context (NOOC). On a test set of 400 samples, the MLP with all features achieved a score of 87.25%, and that with only the image-text features a score of 88%. In addition to the challenge requirements, we also propose a separate pipeline to automatically construct caption pairs and annotations using the images and captions provided in the large, un-annotated training dataset. We hope that this endeavor will open the door for improvements, since hand-annotating cheapfake labels is time-consuming. To evaluate the performance on the test set, the Docker image with the models is available at: https://hub.docker.com/repository/docker/malkaddour/mmsys22cheapfakes. The open-source code for the project is accessible at: https://github.com/malkaddour/ACMM-22-Cheapfake-Detection-Sentiment-aware-Classifier-for-Out-of-Context-Caption-Detection.

Mental stress is a significant factor in the development of a wide variety of psychological, emotional, behavioral, and physical illnesses. It is critical to accurately quantify mental stress, which needs reliable neuroimaging to monitor stress levels. In this work, we used a modified Stroop Color Word Task (SCWT) with time constraints and negative feedback to elicit two distinct degrees of stress in the workplace. We then used salivary alpha amylase (SAA) concurrently with functional near-infrared spectroscopy (fNIRS) to quantify the level of stress. We propose Long Short-Term Memory (LSTM) to decode the two classes of mental stress based on the fNIRS time series. Five-fold cross validation, two LSTM layers, as well as many trainable characteristics were used to construct the network. The induced mental stress increased the level of salivary alpha amylase significantly (p<0.001) by 32.09%. Likewise, we found that LSTM classified mental stress with an average accuracy, sensitivity, and specificity, equal to 72.5%, 72%, and 73% respectively. The findings indicated that the developed LSTM could be used to effectively classify mental stress using fNIRS time series.

Abstract The conventional technique of leukocyte cell classification involves segmenting the required portion of cells from input image, extracting features of the segmented nuclei, reducing and optimizing these features and then implements the classifier. Thus, designing a good classifier by using such techniques increases the time complexity of the system. In order to resolve such issues, the proposed work implements the deep convolutional neural network (DCNN)‐based models for classifying malignant versus normal WBCs. The proposed system is validated on 108 images of ALL‐IDB 1. Due to limited number of training samples, data augmentation is used to create a similar type of virtual image. In this work, experimentation is carried out for discrimination between normal and infected WBC using DCNN with four different activation functions. By using this method, a set of 6000 samples are generated and used for proper training of the DL model for all activation functions. The performance of each trained model is evaluated in terms of accuracy, recall, precision and F‐measure with the maximum values of 98.1%, 98.3%, 98.3% and 98.3% are achieved, respectively. Finally, it has been concluded that the defined DCNN model and ReLu activation function yield outstanding performance for lymphoblast characterization using microscopic blood images.

Computing optical flow is a fundamental problem in computer vision. However, deep learning-based optical flow techniques do not perform well for non-rigid movements such as those found in faces, primarily due to lack of the training data representing the fine facial motion. We hypothesize that learning optical flow on face motion data will improve the quality of predicted flow on faces. This work aims to: (1) exploring self-supervised techniques to generate optical flow ground truth for face images; (2) computing baseline results on the effects of using face data to train Convolutional Neural Networks (CNN) for predicting optical flow; and (3) using the learned optical flow in micro-expression recognition to demonstrate its effectiveness. We generate optical flow ground truth using facial key-points in the BP4D-Spontaneous dataset. This optical flow is used to train the FlowNetS architecture to test its performance on the Extended Cohn-Kanade dataset and a portion of the generated dataset. The performance of FlowNetS trained on face images surpassed that of other optical flow CNN architectures. Our optical flow features are further compared with other methods using the STSTNet micro-expression classifier, and the results indicate that the optical flow obtained using this work has promising applications in facial expression analysis.

With recent developments in communication technologies, vehicular networks have become a reality with various applications. However, the cybersecurity aspect of vehicular networks is still an open issue that needs to be addressed with novel defence mechanisms against attacks. This paper first presents the state-of-the-art communication technologies in vehicular networks (either inter-vehicle networking or in-vehicle networking) along with their applications. Then we explore novel technologies including machine learning and blockchain as cybersecurity defence mechanisms in vehicular networks. Based on the extensive survey, we highlight some insights for future research to secure vehicular networks.

Digital Twin (DT) has become the key technology in the Intelligent Transportation Systems (ITS) in smart cities to keep the health and reliability of various DT requesters, such as private vehicles, public transportation, energy systems, etc. The combination of DT and ITS can further release the potential of participants in smart cities and guarantee their efficiency and reliability. Despite the advantages of DT-enabled ITS, not all requesters need the same level of DT service due to the highly dynamic nature of ITS. Safe and reliable matching between DT and ITS still needs to be resolved. To address these issues, we propose the blockchain-enabled Digital Twin as a Service (DTaaS) for ITS. First, we propose an on-demand DTaaS architecture to fully utilize the sensing capabilities of ITS and the macro perspective of DT. Second, a double-auction model and a price adjustment algorithm are proposed to realize the optimal DT matching for ITS requesters and ensure the benefits of participants. Third, a permissioned blockchain and a novel DT-DPoS consensus mechanism are established to enhance the security and efficiency of DTaaS. Simulation shows that the proposed DTaaS and double-auction can efficiently stimulate and facilitate DT transactions. The proposed DT-DPoS also has obvious advantages.

This paper investigates binaural beat stimulation (BBs) on mitigating mental stress levels at the workplace. We developed an experimental protocol to induce stress levels by performing Stroop Color-Word Task (SCWT) under time pressure and negative feedback. Then, we mitigated the levels of stress using 16 Hz BBs. The level of stress was assessed by utilizing Functional Near-Infrared Spectroscopy (fNIRS), salivary alpha-amylase, and behavioral responses. We quantified the level of stress using statistical analysis, functional connectivity based on Phase Locking Value (PLV), and support vector machines (SVM) classifier. We found that BBs has significantly improved the accuracy of target detection by 27.35 %, (p<0.005) and reduced the cortisol level. The classification results showed that the SVM technique with PLV features differentiates between three levels of mental states (control, stress and mitigation) with an average accuracy of 65.22%, and sensitivity of 81.79% and specificity of 80.10%.

Image inpainting is a non-trivial task in computer vision due to multiple possibilities for filling the missing data, which may be dependent on the global information of the image. Most of the existing approaches use the attention mechanism to learn the global context of the image. This attention mechanism produces semantically plausible but blurry results because of incapability to capture the global context. In this paper, we introduce hypergraph convolution on spatial features to learn the complex relationship among the data. We introduce a trainable mechanism to connect nodes using hyperedges for hypergraph convolution. To the best of our knowledge, hypergraph convolution have never been used on spatial features for any image-to-image tasks in computer vision. Further, we introduce gated convolution in the discriminator to enforce local consistency in the predicted image. The experiments on Places2, CelebA-HQ, Paris Street View, and Facades datasets, show that our approach achieves state-of-the-art results.

The massive increase in computing and network capabilities has resulted in a paradigm shift from vehicular networks to the Internet of Vehicles (IoV). Owing to the dynamic and heterogeneous nature of IoV, it requires efficient resource management using smart technologies, such as software-defined network (SDN), machine learning (ML), and so on. Roadside units (RSUs) in software-defined-IoV (SD-IoV) networks are responsible for network efficiency and offer several safety functions. However, it is not viable to deploy enough RSUs, and also the existing RSU placement lacks universal coverage within a region. Furthermore, any disruption in network performance or security impacts vehicular activities severely. Thus, this work aims to improve network efficiency through optimal RSU placement and enhance security with a malicious IoV detection algorithm in an SD-IoV network. Therefore, the memetic-based RSU (M-RSU) placement algorithm is proposed to reduce communication delay and increase the coverage area among IoV devices through an optimum RSU deployment. Besides the M-RSU algorithm, the work also proposes a distributed ML (DML)-based intrusion detection system (IDS) that prevents the SD-IoV network from disastrous security failures. The simulation results show that M-RSU placement reduces the transmission delay. The DML-based IDS detects the malicious IoV with an accuracy of 89.82% compared to traditional ML algorithms.

Summary Smartphones are equipped with precise hardware sensors including accelerometer, gyroscope, and magnetometer. These devices provide real‐time semantic data that can be used to recognize daily life physical activities for personalized smart health assessment. Existing studies focus on the recognition of simple physical activities but they lacked in providing accurate recognition of physical activities having complex interclass variations. Therefore, this research focuses on the accurate recognition of physical activities having complex interclass variations. We propose a two‐layered approach calledPARCIVthat first clusters similar activities based on semantic data and then recognize them using a machine learning classifier. Our two‐layered approach first bounds the highly indistinguishable activities in clusters to avoid misclassification with other distinguishable activities and thereafter recognize them on a fine‐grained level within each cluster. To evaluate our approach, we make an android application that collects labeled data by using smartphone sensors from 10 participants, while performing activities.PARCIVrecognizes distinguishable as well as indistinguishable activities with high accuracy of 99% on the self‐collected dataset. Furthermore,PARCIVachieve 95% accuracy on the publicly available dataset used by state‐of‐the‐art studies.PARCIVoutperforms various state‐of‐the‐art studies by 8%‐17% for simple activities as well as complex activities.

A large scale fast-growing data generated in intelligent transportation systems (ITS) has become a ponderous burden on the coordination of heterogeneous transportation networks, which makes the traditional cloud-centric storage architecture no longer satisfy new data analytics requirements. Meanwhile, the lack of storage trust between ITS devices and edge servers could lead to security risks in the data storage process. However, a unified data distributed storage architecture for ITS with intelligent management and trustworthiness is absent in the previous works. To address these challenges, this paper proposes a distributed trustworthy storage architecture with reinforcement learning in ITS, which also promotes edge services. We adopt an intelligent storage scheme to store data dynamically with reinforcement learning based on trustworthiness and popularity, which improves resource scheduling and storage space allocation. Besides, trapdoor hashing based identity authentication protocol is proposed to secure transportation network access. Due to the interaction between cooperative devices, our proposed trust evaluation mechanism is provided with extensibility in the various ITS. Simulation results demonstrate that our proposed distributed trustworthy storage architecture outperforms the compared ones in terms of trustworthiness and efficiency.

Volunteer computing uses computers volunteered by the general public to do distributed scientific computing. Volunteer computing is being used in high-energy physics, molecular biology, medicine, astrophysics, climate study, and other areas. These projects have attained unprecedented computing power. However, with the development of information technology, the traditional defense system cannot deal with the unknown security problems of volunteer computing . At the same time, Cyber Mimic Defense (CMD) can defend the unknown attack behavior through its three characteristics: dynamic, heterogeneous, and redundant. As an important part of the CMD, the dynamic scheduling algorithm realizes the dynamic change of the service centralized executor, which can enusre the security and reliability of CMD of volunteer computing . Aiming at the problems of passive scheduling and large scheduling granularity existing in the existing scheduling algorithms, this article first proposes a scheduling algorithm based on time threshold and task threshold and realizes the dynamic randomness of mimic defense from two different dimensions; finally, combining time threshold and random threshold, a dynamic scheduling algorithm based on multi-level queue is proposed. The experiment shows that the dynamic scheduling algorithm based on multi-level queue can take both security and reliability into account, has better dynamic heterogeneous redundancy characteristics, and can effectively prevent the transformation rule of heterogeneous executors from being mastered by attackers.

Blockchain is a distributed operation and information supervision technology programmed initially for Bitcoin cryptocurrency. The awareness in Blockchain technology is rapidly growing since the notion was invented in the year 2008. The motivation for the concentration in Blockchain is its significant characteristics that deliver security, privacy, and information reliability devoid of any additional system regulating the communications, and consequently it generates fascinating research domains, specifically from the viewpoint of methodological difficulties and restrictions. This study discovers the wide‐ranging Blockchain technology and studies it's perspective with respect to ‘ internet‐of‐things ’ controlled nodes. A resilient prototype method has been programmed that reveals a basic system exhausting Blockchain. The outcome illustrates that the established method is functional in test‐bed environment.

This paper details the authors' efforts to push the baseline of emotion recognition performance on the Geneva Multimodal Emotion Portrayals (GEMEP) Facial Expression Recognition and Analysis database. Both subject-dependent and subject-independent emotion recognition scenarios are addressed in this paper. The approach toward solving this problem involves face detection, followed by key-point identification, then feature generation, and then, finally, classification. An ensemble of features consisting of hierarchical Gaussianization, scale-invariant feature transform, and some coarse motion features have been used. In the classification stage, we used support vector machines. The classification task has been divided into person-specific and person-independent emotion recognitions using face recognition with either manual labels or automatic algorithms. We achieve 100% performance for the person-specific one, 66% performance for the person-independent one, and 80% performance for overall results, in terms of classification rate, for emotion recognition with manual identification of subjects.

This work details the authors' efforts to push the baseline of expression recognition performance on a realistic database. Both subject-dependent and subject-independent emotion recognition scenarios are addressed in this work. These two happen frequently in real life settings. The approach towards solving this problem involves face detection, followed by key point identification, then feature generation and then finally classification. An ensemble of features comprising of Hierarchial Gaussianization (HG), Scale Invariant Feature Transform (SIFT) and Optic Flow have been incorporated. In the classification stage we used SVMs. The classification task has been divided into person specific and person independent emotion recognition. Both manual labels and automatic algorithms for person verification have been attempted. They both give similar performance.

This paper demonstrates, to our best knowledge, the first attempt on gender and ethnicity identification from silhouetted face profiles using a computer vision technique. The results achieved, after testing on 441 images, show that silhouetted face profiles have a lot of information, in particular, for ethnicity identification. Shape context based matching was employed for classification. The test samples were multi-ethnic. Average accuracy for gender was 71.20% and for ethnicity 71.66%. However, the accuracy was significantly higher for some classes, such as 83.41% for females (in case of gender identification) and 80.37% for East and South East Asians (in case of ethnicity identification).

Mobile IPv6 provides route optimization mechanism for fast communication by lessening the overhead of indirection. Although it ameliorates the communication latency but it also needs good authentication mechanism to make route optimization more effective and reliable. In this paper, we improve one of the route optimization security mechanisms called bombing resistant protocol, and propose a new binding update authentication scheme. Both mechanisms perform the care of address validation of the mobile node and maintain the integrity of the binding update message during binding update process, while the latter performs better in terms of latency and computation. They also resolve reflection and amplification, intensive computation problem.