Open-set face forgery detection poses significant security threats and presents substantial challenges for existing detection models. These detectors primarily have two limitations: they cannot generalize across unknown forgery domains or inefficiently adapt to new data. To address these issues, we introduce an approach that is both general and parameter-efficient for face forgery detection. Our method builds on the assumption that different forgery source domains exhibit distinct style statistics. Specifically, we design a forgery-style-mixture formulation that augments the diversity of forgery source domains, enhancing the model’s generalizability across unseen domains. In addition, previous methods typically require fully fine-tuning pretrained networks, consuming substantial time and computational resources. Drawing on recent advancements in vision transformers (ViT) for face forgery detection, we develop a parameter-efficient ViT-based detection model that includes lightweight forgery feature extraction modules and enables the model to extract global and local forgery clues simultaneously. We only optimize the inserted lightweight modules during training, maintaining the original ViT structure with its pre-trained weights. This training strategy effectively preserves the informative pre-trained knowledge while flexibly adapting the model to the task of Deepfake detection. Extensive experimental results demonstrate that the designed model achieves state-of-the-art generalizability with significantly reduced trainable parameters, representing an important step toward open-set Deepfake detection in the wild.

Deepfakes have recently raised significant trust issues and security concerns among the public. Compared to CNN-based face forgery detectors, ViT-based methods take advantage of the expressivity of transformers, achieving superior detection performance. However, these approaches still exhibit the following limitations: (1) Fully fine-tuning ViT-based models from ImageNet weights demands substantial computational and storage resources; (2) ViT-based methods struggle to capture local forgery clues, leading to model bias; (3) These methods limit their scope on only one or few face forgery features, resulting in limited generalizability. To tackle these challenges, this work introduces Mixture-of-Experts modules for Face Forgery Detection (MoE-FFD), a generalized yet parameter-efficient ViT-based approach. MoE-FFD only updates lightweight Low-Rank Adaptation (LoRA) and Adapter layers while keeping the ViT backbone frozen, thereby achieving parameter-efficient training. Moreover, MoE-FFD leverages the expressivity of transformers and local priors of CNNs to simultaneously extract global and local forgery clues. Additionally, novel MoE modules are designed to scale the model's capacity and smartly select optimal forgery experts, further enhancing forgery detection performance. Our proposed learning scheme can be seamlessly adapted to various transformer backbones in a plug-and-play manner. Extensive experimental results demonstrate that the proposed method achieves state-of-the-art face forgery detection performance with significantly reduced parameter overhead in cross-dataset, cross-manipulation, and robustness evaluations. Our ablation studies further validate the effectiveness of the designed components and the proposed learning scheme. The code is available at: https://github.com/LoveSiameseCat/MoE-FFD.

The pervasive spread of fake news, particularly through manipulated images, presents a consequential negative impact on society. To prevent fake news images from misleading the public, existing methods focus on verifying the authenticity of news images but ignore source traceability, leaving a gap in creating a complete forensic chain for reliable fake news detection. To simultaneously achieve the goals of authenticity verification and source tracing, we propose a proactive image tagging approach based on a design of Disentangled Invertible Neural Networks (DINN). It can simultaneously embed the dual-tags,i.e., authenticable tag and traceable tag, into each news image prior to publication, allowing for separate extraction for authenticity verification and source tracing. Within the proposed DINN, we design a parallel Feature Aware Projection Module (FAPM) to assist DINN in preserving essential tag information, thereby improving extraction accuracy. In addition, we introduce a Distance Metric-Guided Module (DMGM) that learns asymmetric one-class representations, enabling the dual-tags to exhibit different robustness performances under malicious manipulations. Extensive experiments on diverse datasets and unseen manipulations demonstrate that the proposed tagging approach achieves promising performances on both authenticity verification and source tracing for reliable fake news detection and outperforms the prior works.

We propose a novel Greedy Graph Cut (GGC) algorithm to address the graph partitioning problem. The algorithm begins by treating each data point as an individual cluster and iteratively merges cluster pairs that maximize the reduction in the global objective function until the desired number of clusters is achieved. We provide a theoretical proof of the monotonic convergence of the objective function values throughout this process. To improve computational efficiency, the algorithm restricts merging operations to adjacent clusters, resulting in a computational complexity that scales nearly linearly with the sample size. A significant advantage of our greedy approach is its deterministic nature, which ensures consistent results across multiple runs. This stands in contrast to many existing algorithms that are sensitive to random initialization effects. We demonstrate the effectiveness of the proposed algorithm by applying it to the Normalized Cut (N-Cut) problem, a well-studied variant of graph partitioning. Extensive experimental results show that GGC consistently outperforms the conventional two-stage optimization approach-which involves eigendecomposition followed by k-means clustering-in solving the N-Cut problem. Furthermore, comparative analyses reveal that GGC achieves superior performance compared to several state-of-the-art clustering algorithms.

Major Depressive Disorder (MDD) is a prevalent and severe mental disease. Functional Magnetic Resonance Imaging (fMRI)-based diagnostic methods, which analyze Functional Connectivity (FC) to identify abnormal functional connections, have shown promise as biomarker-based approaches for diagnosing depression. However, the high costs of fMRI data result in small sample sizes, hindering the effective identification of abnormal FC patterns. Moreover, existing methods often overlook the potential benefits of incorporating domain knowledge into their models. In this paper, we propose KnowMDD, a novel knowledge-guided cross contrastive learning framework for MDD diagnosis. By incorporating domain knowledge and employing data augmentation, KnowMDD addresses data sparsity while improving robustness and interpretability. Specifically, multiple atlases are used to construct complementary brain graph representations. The default mode network, closely associated with depression, is introduced into the contrastive learning paradigm for diverse subgraph augmentations, while an attention mechanism captures global semantic relationships between brain regions. Based on them, a cross contrastive learning is designed to learn robust representations for accurate diagnosis. Extensive experiments demonstrate the effectiveness, robustness, and interpretability of KnowMDD, which outperforms state-of-the-art methods. We also develop a demonstration system to show its practical application.

Current scene parsers have effectively distilled abstract relationships among refined instances, while overlooking the discrepancies arising from variations in scene depth. Hence, their potential to imitate the intrinsic 3D perception ability of humans is constrained. In accordance with the principle of perspective, we advocate first grading the depth of the scenes into several slices, and then digging semantic correlations within a slice or between multiple slices. Two attention-based components, namely the Scene Depth Grading Module (SDGM) and the Edge-oriented Correlation Refining Module (EoCRM), comprise our framework, the Line-of-Sight Depth Network (LoSDN). SDGM grades scene into several slices by calculating depth attention tendencies based on parameters with explicit physical meanings, e.g., albedo, occlusion, specular embeddings. This process allocates numerous multi-scale instances to each scene slice based on their line-of-sight extension distance, establishing a solid groundwork for ordered association mining in EoCRM. Since the primary step in distinguishing distant faint targets is boundary delineation, EoCRM implements edge-wise saliency quantification and association digging. Quantitative and diagnostic experiments on Cityscapes, ADE20K, and PASCAL Context datasets reveal the competitiveness of LoSDN and the individual contribution of each highlight. Visualizations display that our strategy offers clear benefits in detecting distant, faint targets.

Facial recognition technique is used extensively in areas like online payments, education, and social media. Traditionally, these applications relied on powerful cloud-based systems, but advancements in edge computing have changed this, enabling fast and reliable local processing in complex and extreme environment. However, new challenges arise in availability and durability insurance to make the system running 24/7 with acceptable performance. This paper proposes a novel solution to these challenging settings. First, we use edge device for local data processing, reducing the need for cloud communication and enhancing user privacy. Second, we implement an adaptive control strategy to improve energy management in these devices. Lastly, we establish a solar-powered energy system to facilitate long-term device operation. Our approach strikes a balance between performance, quality, and durability, enabling facial recognition systems to work consistently and efficiently in complex environments.

With the development of multiaccess edge computing (MEC) technology, an increasing number of researchers and developers are deploying their computation-intensive and IO-intensive services (especially AI services) on edge devices. These devices, being close to end users, provide better performance in mobile environments. By constructing a service provisioning system at the network edge, latency is significantly reduced due to short-distance communication with edge servers. However, since the MEC-based service provisioning system is resource-sensitive and the network may be unstable, careful resource allocation and traffic scheduling strategies are essential. This paper investigates and quantifies the cost-effectiveness and robustness of the MEC-based service provisioning system with the applied resource allocation and traffic scheduling strategies. Based on this analysis, acost-effective androbust service provisioningalgorithm, termedCERA, is proposed to minimize deployment costs while maintaining system robustness. Extensive experiments are conducted to compare the proposed approach with well-known baseline algorithms and evaluate factors impacting the results. The findings demonstrate thatCERAachieves at least 15.9% better performance than other baseline algorithms across various instances.

In recent years, the burgeoning urban population, coupled with expanding urban dimensions and various sociodemographic factors, has led to an alarming escalation in criminal activities within urban centers. This escalation has further exacerbated the existing strain on police resources, rendering them increasingly inadequate for effective law enforcement. Concurrently, police patrol operations have emerged as a pivotal instrument in the ongoing battle against violent criminal activities. The judicious planning of patrol routes has the potential to markedly enhance the efficiency of police patrolling endeavors, thereby bolstering the overall security infrastructure within the jurisdiction while simultaneously conserving invaluable police resources. The formulation of an efficient patrol strategy within the intricate and ever-evolving landscape of urban regions replete with crime hotspots represents an intellectually taxing challenge. To address this exigent problem, this paper proffers a novel real-time patrol route planning algorithm tailored to dynamic environments, employing the principles of deep reinforcement learning, specifically the Integrated Double Q-Network (IDQN) method. Subsequently, the efficacy of the proposed method is empirically substantiated through experimentation, attesting to its practical viability and utility in the field of computer science and urban security management.

The proliferation and ubiquity of GPS-enabled mobile Internet of Things (IoT) devices (e.g., drones and unmanned robotic devices) across many disciplines has generated substantial interest in the analysis and mining of trajectory data for enhancing public services such as air pollution monitoring and road safety. Among all trajectory analysis techniques in mobile IoT, classifying moving device trajectories is a fundamental research problem due to its importance for numerous important mobile IoT applications such as travel demand analysis and animal migration patterns. However, existing classification methods for moving object trajectories either are dynamic programming problems (such as similarity-based methods with dynamic time wrapping or Fréchet distance) with a quadratic time complexity in all cases, or need abundant labeled samples to training classification model, thus limit the scalability of these methods for given resource-constrained mobile IoT devices. In this study, we propose a deep learning-based approach for trajectory classification in mobile IoT with linear time complexity by firstly encoding a trajectory as a vector via deep representation learning, then utilizing a meta-learning based approach to learn the ability of classifying trajectory with few annotated samples. Experiments on a massive trajectory dataset show that the proposed approach outperforms state-of-the-art baselines consistently and significantly.

Crime risk prediction is crucial for city safety and residents’ life quality. However, without labeled data, it is challenging to predict crime risk in cities. Due to municipal regulations and maintenance costs, it is not trivial for many cities to collect high-quality labeled crime data. In particular, some cities have lots of labeled data while others may have few. It has been possible to develop a crime prediction model for a city without labeled crime data by learning knowledge from a city with abundant data. Nevertheless, the inconsistency of relevant context data between cities exacerbates the difficulty of this prediction task. To this end, this article proposes an effective unsupervised domain adaptation model (UDAC) for crime risk prediction across cities while addressing the contexts’ inconsistency issue. More specifically, we first identify several similar source city grids for each target city grid. Based on these source city grids, we then construct auxiliary contexts for the target city, to make contexts consistent between the two cities. A dense convolutional network with unsupervised domain adaptation is designed to learn high-level representations for accurate crime risk prediction and simultaneously learn domain-invariant features for domain adaptation. The effectiveness of our model is verified through extensive experiments using three real-world datasets.

The multi-access edge computing (MEC) paradigm has emerged as a critical solution to address the exponential growth in mobile web services and devices. By implementing an edge-based service provisioning system (EPS) with servers located at the network’s edge, both transmission and computation efficiency can be significantly enhanced. Nevertheless, it is also essential to carefully consider the resource allocation for services, the traffic management of requests, and the path arrangement for data delivery to ensure the cost-effective operation of the EPS. Therefore, we investigate and quantify the relationship between the performance and cost of the EPS in this paper, and model the cost-effective service provisioning problem as a multi-phase convex optimization problem. An online algorithm whose name isRDCbased on the Lyapunov framework is proposed to decompose this problem into several sub-problems.Additionally, a heuristic approach that partitions edge servers into several clusters, calledRDC-NePand based onRDC, has also been proposed to reduce computational complexity. A series of experiments were conducted to evaluate the proposed approach. The results demonstrate thatRDCcan effectively balance expense and performance, whileRDC-NePsignificantly simplifies the processing ofRDCwhen the problem scale increases.

Sarcasm is important to sentiment analysis on social media.Sarcasm Target Identification (STI) deserves further study to understand sarcasm in depth.However, text lacking context or missing sarcasm target makes target identification very difficult.In this paper, we introduce multimodality to STI and present Multimodal Sarcasm Target Identification (MSTI) task.We propose a novel multi-scale crossmodality model that can simultaneously perform textual target labeling and visual target detection.In the model, we extract multi-scale visual features to enrich spatial information for different sized visual sarcasm targets.We design a set of convolution networks to unify multi-scale visual features with textual features for cross-modal attention learning, and correspondingly a set of transposed convolution networks to restore multi-scale visual information.The results show that visual clues can improve the performance of TSTI by a large margin, and VSTI achieves good accuracy.

While it is well understood that edge computing can significantly facilitate IoT-related applications by deploying edge servers close to IoT devices, it also faces many challenges with numerous IoT devices connected and interacted. One of the most important issues is how to efficiently deploy edge servers under a certain budget with the explosive growth of data scale and user base. Existing studies for edge server placement fail to consider user’s query preferences since individual users may be interested in events in particular regions and are keen to receive up-to-date data streams that originate in regions of interest. In this article, we present a preference-aware edge server placement approach that offers better workload distribution in terms of both minimizing query latency and balancing the load of edge servers. To achieve this, we formulate edge server placement with multiobjective optimization as a${p}$-center problem and design two progressive approaches. We first propose quadratic integer programming (QIP) for small-scale data sets. Since the${p}$-center problem is an NP-hard problem, we thus propose a heuristic algorithm named TAKG (TAbu search with$K$-means and Genetic algorithm) for large-scale data sets. To evaluate the utility of the proposed models, we have conducted a comprehensive evaluation on a large data set that is collected by more than 1900 IoT devices during 30 days. Experimental results indicate our approaches outperform all baselines significantly in terms of both query latency and load balancing.

Trajectory similarity computation is a fundamental problem for many intelligent applications such as trip trajectory mining to find the most popular routes and road anomaly detection. Existing similarity computation methods for vehicle trajectory, such as dynamic time warping (DTW) and Fréchet distance, are dynamic programming problems with a quadratic time complexity in all cases and need to handle local time shifts when computing the distance between trajectories, thus limiting the scalability of these methods. In addition, GPS-based vehicle trajectories usually contain errors, such as noise and outliers, due to poor satellite visibility in urban regions and nonuniform sampling rates. To this end, we propose an embedding-based method for trajectory similarity computation with linear time complexity, which encodes a trajectory as a vector via deep representation learning and learns the similarity between trajectories with an attention-based learning to rank model. We use an interpolation-based approach to reduce noise and outliers by considering vehicle trajectory is physically constrained to the road network. Experiments on a massive vehicle trajectory data set show that the proposed approach outperforms state-of-the-art baselines consistently and significantly.

Abstract Recently, the Internet-of-Things technique is believed to play an important role as the foundation of the coming Artificial Intelligence age for its capability to sense and collect real-time context information of the world, and the concept Artificial Intelligence of Things (AIoT) is developed to summarize this vision. However, in typical centralized architecture, the increasing of device links and massive data will bring huge congestion to the network, so that the latency brought by unstable and time-consuming long-distance network transmission limits its development. The multi-access edge computing (MEC) technique is now regarded as the key tool to solve this problem. By establishing a MEC-based AIoT service system at the edge of the network, the latency can be reduced with the help of corresponding AIoT services deployed on nearby edge servers. However, as the edge servers are resource-constrained and energy-intensive, we should be more careful in deploying the related AIoT services, especially when they can be composed to make complex applications. In this paper, we modeled complex AIoT applications using directed acyclic graphs (DAGs), and investigated the relationship between the AIoT application performance and the energy cost in the MEC-based service system by translating it into a multi-objective optimization problem, namely the CA $$^3$$ 3 D problem — the optimization problem was efficiently solved with the help of heuristic algorithm. Besides, with the actual simple or complex workflow data set like the Alibaba Cloud and the Montage project, we conducted comprehensive experiments to evaluate the results of our approach. The results showed that the proposed approach can effectively obtain balanced solutions, and the factors that may impact the results were also adequately explored.

To discover the condition of roads, a large number of detection algorithms have been proposed, most of which apply machine learning methods by time and frequency processing in acceleration and velocity data. However, few of them pay attention to the similarity of the data itself when the vehicle passes over the road anomalies. In this article, we propose a method to detect road anomalies by comparing the data windows with various length using Dynamic Time Warping(DTW) method. We propose a model to prove that the maximum acceleration of a vehicle passing through a road anomaly is linear with the height of the road barrier, and it’s verified by an experiment. This finding suggests that it is reasonable to divide the window by threshold detection. We also apply a brief random forest filter to roughly distinguish normal windows from anomaly windows using the aforementioned theory, in order to reduce the time consumption. From our study, a system is proposed that utilizes a series of acceleration data to discover where might be anomalies on the road, named as Quick Filter Based Dynamic Time Warping (QFB-DTW). We show that our method performs clearly beyond some existing methods. To support this conclusion, experiments are conducted based on three data sets and the results are statistically analyzed. We expect to lay the first step to some new thoughts to the field of road anomalies detection in subsequent work.

The rapid development of the Internet-of-Things (IoT) makes it convenient to sense and collect real-world information with different kinds of widely distributed sensors. With plenty of web services providing diverse functions on the cloud, the collected information can be sufficiently used to complete complex tasks after being uploaded. However, the latency brought by long-distance communication and network congestion limits the development of IoT platforms. A feasible approach to solve this problem is to establish an edge-cloud collaboration (ECC) system based on the multi-access edge computing (MEC) paradigm where the collected information can be refined with the services deployed on nearby edge servers. However, as the edge servers are resource-limited, we should be more careful in allocating the edge resource to services, as well as designing the traffic scheduling strategy. In this paper, we investigated the edge-cloud cooperation mechanism of service provisioning in ECC systems, and to that end, proposed an energy-consumption model for it; we also proposed a performance model and balancing model to quantify the running state of ECC systems. Based on these, we further formulated the energy-effective ECC system optimization problem as a joint optimization problem whose decision variables are the resource allocation strategy and traffic scheduling strategy. With the convexity of this problem proved, we proposed an algorithm to solve it and conducted a series of experiments to evaluate its performance. The results showed that our approach can improve at least 4.3 % of the performance compared with representative baselines.

Multimodal named entity recognition (MNER) for tweets has received increasing attention recently.Most of the multimodal methods used attention mechanisms to capture the text-related visual information.However, unrelated or weakly related text-image pairs account for a large proportion in tweets.Visual clues unrelated to the text would incur uncertain or even negative effects for multimodal model learning.In this paper, we propose a novel pre-trained multimodal model based on Relationship Inference and Visual Attention (RIVA) for tweets.The RIVA model controls the attention-based visual clues with a gate regarding the role of image to the semantics of text.We use a teacher-student semi-supervised paradigm to leverage a large unlabeled multimodal tweet corpus with a labeled data set for text-image relation classification.In the multimodal NER task, the experimental results show the significance of text-related visual features for the visual-linguistic model and our approach achieves SOTA performance on the MNER datasets.

With a large number of possible smart objects in Social Internet of Things (SIoT), a recommendation system is of great necessity to help users find smart objects they need. However, traditional recommendation techniques usually exploit user's rating or feedback information, which are impractical as such kind of user preference information is difficult to collect in the SIoT environment. In addition, temporal context plays an important role in smart object recommendation since most users tend to utilize different objects at different time slots in a day, e.g., making coffee at morning and playing games on weekends. In this article, we propose a time-aware smart object recommendation model by jointly considering user's preference over time and smart object's social similarity. We first learn user's preference over time from his/her object usage events with a latent probabilistic model. Then, we estimate the smart object's social similarity by embedding their heterogeneous social relationships into a shared lower dimensional space. Finally, we generate the recommendation list with an item-based collaborative filtering. We conduct a comprehensive experimental study based on two real-world data sets, and the experimental results show our method outperforms all baselines significantly in terms of recommendation effectiveness.

Trajectory similarity computation is one of the most fundamental functionality, which is applied in many fields, such as trip trajectory mining to find the most popular routes and similar ones, and identify the routes of animal migration and even the stock trends.There are two different kinds of search thoughts, the advanced deep learning method and traditional points matching methods.However, the exiting methods are not totally perfect to solve the trajectory similarity computation problem.The deep-learning method has original problem that it needs a large size of dataset resulting in the requirement of the training time much more than we expected.While the traditional points matching method often suffer from noise and non-uniform sampling rates, because points matching often treats it as two different sequences when the unequal points turn up.In other word, it is often sensitive to the noise which lowers the correct rate of the similarity computation.Based of the statement upon, we propose a new method -applying the interpolation and deformed integration to similarity computation.Experiments shows that our method is robust to the noise and non-uniform sampling rate.

Wireless sensor networks (WSNs) are an area of emerging networking research. One obstacle is its limited supply of energy. Therefore, minimizing energy consumption and maximizing system lifetime have been a major design goal for WSNs. This paper presents an energy-efficient event-driven clustering routing algorithm (EDC algorithm) based on unique features of the event-driven data model of WSNs. The algorithm can decide which nodes would become cluster-head nodes according to the maximum remainder energy of nodes, which are sensing an event occurred and are firstly switched to the active state if their several components are in the sleeping state. This strategy can keep sensor nodes with lower remainder energy out of being used up quickly. Detailed simulations of sensor network environments demonstrate that EDC algorithm saves node energy, prolongs system lifetime, and improves evenness of dissipated network energy.