Single image super-solution (SR) aims to restore a high-resolution (HR) image from a degraded low-resolution (LR) image. However, existing SR models still face a significant domain gap between synthetic and real-world datasets due to the mismatched degradation distributions, hindering SR models from achieving optimal results. In this paper, we propose an unsupervised diffusion-based degradation modeling framework (UDDM) to effectively capture real-world degradation distributions. Specifically, given unpaired LR and HR images, a diffusion-based degradation module (DDM) first models the degradation distribution by diffusing real-world LR images to downsampled LR images, which does not require HR images. It then applies reverse diffusion to generate real-world LR images from extremely downsampled HR images. This approach allows DDM to model and generate real-world degradation distributions without requiring paired data, by using extreme downsampling to link unpaired LR and HR images. Additionally, we introduce a physics-based dynamic degradation module (P-DDM) that adaptively models content-aware degradation, ensuring both content and structural accuracy. Finally, the LR images generated by DDM and P-DDM are adaptively weighted to produce the final LR images, which are paired with the given HR images for training the SR network. Extensive experiments across multiple real-world datasets demonstrate that our framework achieves state-of-the-art performance in both qualitative and quantitative comparison.

Scene Graph Generation (SGG) aims to detect all objects and identify their pairwise relationships existing in the scene. Considering the substantial human labor costs, existing scene graph annotations are often sparse and biased, which result in confusion training with low-frequency predicates. In this work, we design a Semi-Supervised Clustering framework for Scene Graph Generation (SSC-SGG) that uses the sparse labeled data to guide the generation of effective pseudo-labels from unlabeled object pairs, thus enriching the labeled sample space, especially for low-frequency interaction samples. We approach from the perspective of clustering, reducing the problem of confirmation bias in a self-training manner. Specifically, we first enhance the model's robustness to feature extraction via prototype-based clustering, aggregating different relationship augmented features onto the same prototype. Secondly, we design a dynamic pseudo-label assignment algorithm based on a mini-batch, which adjusts the detection sensitivity to different frequency samples from the historical assignment. Finally, we conduct joint training on the pseudo-labels and the labeled data. We conduct experiments on various SGG models and achieve substantial overall performance improvements, demonstrating the effectiveness of SSC-SGG.

Traditional video instance segmentation (VIS) models rely on extensive per-frame video annotations, which are both time-consuming and costly. In this paper, we present MinMaxVIS, a novel VIS framework that reduces the dependency on fully labeled video datasets by utilizing a small set of labeled images from the target domain along with a large volume of general-domain, unlabeled images. MinMaxVIS operates in three stages: first, a preliminary segmentation model is trained on the small labeled set from the target domain; this model then retrieves relevant instances from the unlabeled dataset to build a high-quality pseudo-labeled set, ensuring a rich content alignment with the target domain while avoiding the inefficiencies of large-scale semi-supervised learning across the entire unlabeled set. Finally, we train MinMaxVIS on a combination of labeled and pseudo-labeled data, addressing challenges such as noise in pseudo-labels and instance association across frames. To simulate object continuity, we augment static images to create paired frames, allowing MinMaxVIS to capture instance associations effectively. MinMaxVIS outperforms the prior image-driven approach, MinVIS, achieving superior mAP scores with significantly reduced labeled data. For instance, MinMaxVIS with a Swin-L backbone attains 62.2 mAP on YouTube-VIS 2019 using only 2% labeled data and additional unlabeled images from SA-1B. This surpasses MinVIS, which uses the same backbone trained on fully labeled YouTube-VIS 2019, by 0.6 mAP.

Event-aware information flow security in cyber–physical systems (CPSs) emphasizes the correlation among events. Event-aware noninterference is a security property capable to describe such a correlation. In light of the event-aware noninterference assessment problem, system modeling is a common means, and many studies have been done on such a problem by using formal modeling tools, i.e., Petri nets (PNs), at present. However, classical PNs suffer from the lack of modeling for patterns with semantic sharing of events, which can be modeled by labeled PNs (LPNs). In this article, we present the concept of semantics-based noninterference for the CPSs modeled by LPNs and focus on the assessment problem of semantics-based noninterference properties represented by semantics-based strong nondeterministic noninterference (SNNI) and extended bisimulation SNNI (EBSNNI). To this end, we first give the formal definitions of semantics-based SNNI and EBSNNI. Then, we analyze the assessment mechanisms of them according to the characteristics of semantic sharing of events in LPNs. On this basis, we provide the semantics-based noninterference assessment method involving the coarse and fine assessments to reveal the event-aware security of CPSs. Finally, a case study is provided to explain the significance of our research and the effectiveness of our method.

With the transformative impact of the Transformer, DETR pioneered the application of the encoder-decoder ar-chitecture to object detection. A collection of follow-up research, e.g., Deformable DETR, aims to enhance DETR while adhering to the encoder-decoder design. In this work, we revisit the DETR series through the lens of Faster R-CNN. We find that the DETR resonates with the underlying principles of Faster R-CNN's RPN-refiner design but benefits from end-to-end detection owing to the incorpo-ration of Hungarian matching. We systematically adapt the Faster R-CNN towards the Deformable DETR, by in-tegrating or repurposing each component of Deformable DETR, and note that Deformable DETR's improved per-formance over Faster R-CNN is attributed to the adoption of advanced modules such as a superior proposal refiner (e.g., deformable attention rather than RoI Align). When viewing the DETR through the RPN-refiner paradigm, we delve into various proposal refinement techniques such as deformable attention, cross attention, and dynamic convo-lution. These proposal refiners cooperate well with each other; thus, we synergistically combine them to estab-lish a Hybrid Proposal Refiner (HPR). Our HPR is ver-satile and can be incorporated into various DETR de-tectors. For instance, by integrating HPR to a strong DETR detector, we achieve an AP of 54.9 on the COCO benchmark, utilizing a ResNet-50 backbone and a 36-epoch training schedule. Code and models are available at https://github.com/ZhaoJingjing713IHPR.

Large language models (LLMs) often demonstrate inconsistencies with human preferences. Previous research typically gathered human preference data and then aligned the pre-trained models using reinforcement learning or instruction tuning, a.k.a. the finetuning step. In contrast, aligning frozen LLMs without requiring alignment data is more appealing. This work explores the potential of the latter setting. We discover that by integrating self-evaluation and rewind mechanisms, unaligned LLMs can directly produce responses consistent with human preferences via self-boosting. We introduce a novel inference method, Rewindable Auto-regressive INference (RAIN), that allows pre-trained LLMs to evaluate their own generation and use the evaluation results to guide rewind and generation for AI safety. Notably, RAIN operates without the need of extra data for model alignment and abstains from any training, gradient computation, or parameter updates. Experimental results evaluated by GPT-4 and humans demonstrate the effectiveness of RAIN: on the HH dataset, RAIN improves the harmlessness rate of LLaMA 30B from 82% of vanilla inference to 97%, while maintaining the helpfulness rate. On the TruthfulQA dataset, RAIN improves the truthfulness of the already-well-aligned LLaMA-2-chat 13B model by 5%.

Prior research in human-centric AI has primarily addressed single-modality tasks like pedestrian detection, action recognition, and pose estimation. However, the emergence of large multimodal models (LMMs) such as GPT-4V has redirected attention towards integrating language with visual content. Referring expression comprehension (REC) represents a prime example of this multimodal approach. Current human-centric REC benchmarks, typically sourced from general datasets, fall short in the LMM era due to their limitations, such as insufficient testing samples, overly concise referring expressions, and limited vocabulary, making them inadequate for evaluating the full capabilities of modern REC models. In response, we present HC-RefLoCo (Human-Centric Referring Expression Comprehension with Long Context), a benchmark that includes 13,452 images, 24,129 instances, and 44,738 detailed annotations, encompassing a vocabulary of 18,681 words. Each annotation, meticulously reviewed for accuracy, averages 93.2 words and includes topics such as appearance, human-object interaction, location, action, celebrity, and OCR. HC-RefLoCo provides a wider range of instance scales and diverse evaluation protocols, encompassing accuracy with various IoU criteria, scale-aware evaluation, and subject-specific assessments. Our experiments, which assess 24 models, highlight HC-RefLoCo’s potential to advance human-centric AI by challenging contemporary REC models with comprehensive and varied data. Our benchmark, along with the evaluation code, are available at https://github.com/ZhaoJingjing713/HC-RefLoCo.

Contrastive learning has recently been applied to enhancing the BERT-based pre-trained models for code search. However, the existing end-to-end training mechanism cannot sufficiently utilize the pre-trained models due to the limitations on the number and variety of negative samples. In this paper, we propose MoCoCS, a multi-modal momentum contrastive learning method for code search, to improve the representations of query and code by constructing large-scale multi-modal negative samples. MoCoCS increases the number and the variety of negative samples through two optimizations: integrating multi-batch negative samples and constructing multi-modal negative samples. We first build momentum contrasts for query and code, which enables the construction of large-scale negative samples out of a mini-batch. Then, to incorporate multi-modal code information, we build multi-modal momentum contrasts by encoding the abstract syntax tree and the data flow graph with a momentum encoder. Experiments on CodeSearchNet with six programming languages demonstrate that our method can further improve the effectiveness of pre-trained models for code search.

One of the noteworthy investigations in workflows is the implementation of authorization-related security requirements. There are two typical security requirements, i.e., separation of duty (SoD) and binding of duty (BoD). However, most of the previous works are only focused on SoD while ignoring BoD. In this article, we consider both of them by proposing an extended-structural implementation approach, namely beacon-based firing control, to enforce security requirements. Thanks to the flexibility of beacon-based firing control, both BoD and SoD can be enforced in a straightforward way with no sophisticated operations for their implementation, although they are a pair of security requirements in conflict. As a preparation of beacon-based firing control, we define the beacon-extended Petri nets (PNs) by introducing a new object, namely beacon, to PNs so as to lay the foundation. In addition, we present the firing-based linear equations and inequalities for BoD and SoD to provide the standardized descriptions for their implementation. For the sake of expansibility, the applicability analysis is provided for the more general security requirements. Ultimately, the comparative experiments and discussions are presented to show the effiectiveness and efficiency of the proposed approach.

Authorization-related security requirements are of great significance in workflow management systems. Existing studies are restricted in their scopes of research. There is no unified principle for their implementation. In this paper, we focus on the unification of authorization-related security requirements using Petri nets (PNs). These security requirements are expressed by constraints, being imposed on tasks, namely task-based security requirements (TSRs). By downgrading TSRs to a kind of authorization-conflict relationship, we provide a standardized expression for TSRs. Such a standardized expression can be transformed to firing-based linear inequalities which are a more general representation of constraints. Then, we propose the firing control for the unified implementation of TSRs based on firing-based linear inequalities. In fact, firing control is enforced by structural controllers namely monitors which are structurally consistent with PNs. For the sake of conciseness, simplification techniques are provided for the monitors. Ultimately, the experiments and discussions are presented to show the performance and advantages of the proposed approach.

Data leakage prevention has recently become the most important concern for both personal users and corporate users. Most existing feasible data leakage preventers are built with the Dynamic Binary Instrumentation (DBI) technology. Such mechanism suffers from poor application compatibility issue, especially for the large scale ones. In this paper, we propose Gemini, an instrumentation-free approach, to track data propagation dynamically and then prevent data leakage. Gemini leverages the page fault interrupt mechanism of the operating system, instead of DBI, to track memory page accesses, and then thwart the data leakage. As a result, Gemini is application transparent, i.e., it solves the application compatibility issue. Besides, Gemini is implemented on the most prevalent operating system-Windows, while most of previous approaches are built on Linux. Our evaluation results demonstrate Gemini's feasibility and effectiveness.

The virtual machine (VM) based introspection on the operating system (OS) holds predominance over previous host-based introspectors for being more resistant to attack while suffering the difficulty of retrieving the semantic view of the OS. Previous approaches addressing this limitation highly depend on the explicit guest information which is still subvertable to the privileged malware. Moreover, they only deal with the OS deployed in the VM instead of our daily used native OS. In this paper, we present a new VM-based introspecting approach called Pisces which accurately reproduces the execution environment of the underlying preinstalled OS within the Pisces VM and provides an OS-level semantic view. With our novel local-booting virtualization technology, Pisces VM just boots from the underlying host OS but not a newly installed OS image. Thus, Pisces provides a feasible way to introspect on the existing OS. In addition, instead of relying on the explicit guest information, Pisces adopts a set of unique techniques to implicitly construct the semantic view of the OS from within the virtualized hardware layer. The evaluation results demonstrate its practicality and effectiveness.

Deadlock is a common error for multithread Java programs. Existing Java thread deadlock detection solutions either require source code, or are built on non-official JVMs. In a consequence, a great number of Java programs cannot be evaluated with these solutions. This paper proposes a new Java thread deadlock detection approach, namely JDeadlockDetector. JDeadlockDetector is built on the official Java Virtual Machine (JVM), viz., OpenJDK's HotSpot. Compared to existing methods, JDeadlockDetector archieves three unique advantages, i.e., application transparency, detection accuracy and minimized performance overhead. Our functionality evaluation shows JDeadlockDetector achieves no false negative and minimized false positive while the performance evaluation shows the workloads generated by SPECjbb2005 achieve 96.7% of official JVM speed on average.

With security researchers relying on the virtual machine (VM) in their analysis work, malware has a significant stake in detecting the presence of a VM to avoid executing its vicious behavior. But hiding a VM from malware by building a transparent virtual machine monitor (VMM) is fundamentally infeasible, as well as impractical from a performance and engineering standpoint. This paper proposes a new idea from another perspective: hiding the "real" machine from the VMM-aware malware. We propose a minimal VMM called MiniVMM which can migrate a booted OS, our protecting concern, to this VMM on demand. In our protection model, all the untrusted code, although having been verified by VMM-based malware detectors, should be executed in this migrated OS. Instead of building a transparent VMM, MiniVMM advisedly exposes the VMM fingerprints to prevent the computer against VMM-aware malicious programs by deceiving them into deactivating their destructive behavior by themselves. MiniVMM has two key features: dynamic OS migration and commodity VMM fingerprints emulation. Unlike existing VMM solutions, MiniVMM can make the protected OS transfer between VMM mode and native mode dynamically. MiniVMM can also emulate the fingerprints of prevalent VMMs to make the protected computer more like a "real" VM. MiniVMM might be deployed as a considerable complement of the existing VMM-based security approaches to make the native OSes immune to the VMM-aware malware.

experimental Mbone for a number of years. The practical pricing mechanism is the foundation for the deploying of IP multicast in the inter-domain Internet. The IP multicast service model and its pricing mechanism are discussed in this paper. Three models are proposed for all applications in the real environments. They are ICP-USER model, ICP-ISP model and ICP-ISP-USER model. Here, the Internet is considered as an ecosystem, and the entities construct a supply chain with the welfare maximum purpose. Our work gives a general discussion on the practical pricing mechanism for the stability of the economic relationship between ICPs, ISPs and users in the Internet. Key Words—IP multicast, pricing mechanism, Cost-sharing mechanism

This paper analyzes the relationship between BGP convergence and the power-law of the Internet. The inter-domain routing system is classified into three hierarchies based on the power-law and commercial relations of autonomous systems. The relation of network topology and three convergence parameters-convergence time T, affected ASs set Nc and affected paths factor mu is presented for all sorts of convergence events in different layers. The result shows that the power-law nature of network influences the BGP convergence greatly