Vision-Language Models (VLMs) are widely used in tasks like Open-Vocabulary Object Detection and zero-shot Classification, owing to their powerful generalization. However, recent research reveals that VLMs exhibit significant performance instability when tasked with recognizing concepts at varying granularities (e.g., ``animal'' vs. ``dog''). Prevailing methods inject external knowledge from Large Language Models, but this unconstrained approach distorts the VLM's inherent hierarchical orthogonal geometry, leading to performance collapse on general concepts. To address this, we introduce GeCoin, an innovative Geometrically Constrained framework that safely enhances existing VLMs with external knowledge for improved hierarchical understanding, without additional training. By projecting knowledge into the null-space of a query concept's feature space, GeCoin mathematically guarantees the preservation of general knowledge while integrating specialized information. Extensive experiments across large-scale benchmarks, diverse VLMs, and knowledge from various LLMs (e.g., GPT-3.5, Claude-3, Gemini-Pro) show that GeCoin boosts performance by an average of 3.9% over the strongest baseline—crucially eradicating performance collapse on general concepts.

In federated learning (FL), multiple clients train a global model by sharing gradients. Since the client data remains locally, federated learning is considered privacy-safe. Although previous studies have demonstrated the feasibility of recovering client data from shared gradients, most studies assume that there is no privacy defense in the federated setting. In this work, we propose Deep Diffusion Gradients Leakage, which uses the prior information of the diffusion model to compensate for the gradient information degradation caused by privacy defense, and reconstructs high-quality original images from deep neural networks under privacy defense. We also propose a step-by-step optimization strategy to solve the gradient matching problem in the process of reconstructing the image by sequentially optimizing the latent variables of the diffusion model, and more finely control the reconstruction results. We hope that our method can promote the development of privacy-preserving methods for federated learning.

The past few years have witnessed the immense development of small object detection, which is aimed at detecting size-limited targets in high-resolution images. The prevailing methods focus on extracting fine-grained information by expanding the receptive fields and then generating the potential small object region. However, these solutions inevitably add sophisticated detectors and extra learning components, which incur time-consuming and computation-costing. Meanwhile, we observe that it’s suboptimal to extract fine features in such a generic way. To alleviate the issues, we propose a multi-branch small object detection framework with a regular-scale detection branch and a small-scale detection branch. Specifically, we design and pre-train a reinforcement learning agent to control feature extractors in both branches according to the results of small object areas. Moreover, we present a region clipping algorithm to rebuild the small object to regular size, which can be input into a mature detector directly. The extensive experiments on COCO, VisDrone, SODA-D, and our collecting TVDS datasets demonstrate our method outperforms the state-of-the-art methods in several metrics.

Large image-language models(LLM) have made significant progress in zero-shot anomaly detection(ZSAD), however, the semantic gap between images and text limits their performance in hierarchical learning. In this paper, we propose the hierarchical alignment clip(HieClip) framework, to achieve hierarchical alignment between images and text. Specifically, we introduce learnable hierarchical textual(LHT) to reduce the representation differences between various levels of images and text, while performing multi-level comprehensive discrimination. Additionally, the dynamically adjusting the weights of features at different levels, improving the model’s ability to capture both global and local information. Experiments on public industrial datasets demonstrate HieClip’s effectiveness, showing significant accuracy improvement, and its strong generalization capabilities were further validated on medical datasets. Compared to existing methods, HieClip excels in anomaly detection tasks, particularly in industrial inspection and medical diagnosis scenarios.

Defect detection is a fundamental task in industrial image analysis, crucial for identifying and delineating defect regions. However, existing models, often struggle to learn critical features effectively under conditions of noisy interference. In this study, we introduce the Fine-Grained Hierarchical Interaction Learning (FINet) framework, designed to enhance the learning process by harmonizing feature interactions at multiple scales. Specifically, FINet incorporates the Adaptive Tensor Interaction (ATI) to facilitate high-order feature interactions amidst noise in a high-dimensional frequency space. Additionally, the FlexiFocus network is developed to dynamically balance feature focus across scales, further enhancing defect feature visibility and providing an effective trade-off between computational speed and performance. Extensive experiments on the PVEL-AD dataset show FINet’s superior accuracies (90.50% mAP50, 62.80% mAP50:5:95 ), surpassing DDQ-DETR by 8.3% and 3.6%, respectively. The code is available at https://github.com/zhongzee/FINet-master.

Pioneer efforts have been dedicated to action segmentation that predicts what step is occurring in a video frame. Existing studies focus on improving the accuracy of video segmentation, but neglect the temporal continuity of intersegments and semantic consistency of intra-segments, which are necessary for developing computer-assisted systems. Meanwhile, Temporal Convolutional Networks have shown good performance in action segmentation tasks, but their high layers tend to lose fine-grained information and impact the results. Toward this end, we devise a multi-feature and multi-branch action segmentation framework for modeling long-term and short-term dependencies. Specifically, we present a multi-feature fusion to enhance temporal video representation and design a multi-branch predictor for extracting both segment-level and frame-level information. We justify our framework over three datasets and experimental results demonstrate its superiority, especially in Edit and F1 metrics, which means our framework is more applicable to computer-assisted systems.

Medical Visual Question Answering (Med-VQA) interprets complex medical imagery using user instructions for precise diagnostics, yet faces challenges due to diverse, inadequately annotated images. In this paper, we introduce the Universal Instruction-Vision Navigator (Uni-Med) framework for extracting instruction-to-answer relationships, facilitating the understanding of visual evidence behind responses. Specifically, we design the Instruct-to-Answer Clues Interpreter (IAI) to generate visual explanations based on the answers and mark the core part of instructions with "real intent" labels. The IAI-Med VQA dataset, produced using IAI, is now publicly available to advance Med-VQA research. Additionally, our Token-Level Cut-Mix module dynamically aligns visual explanations with image patches, ensuring answers are traceable and learnable. We also implement intention-guided attention to minimize non-core instruction interference, sharpening focus on ’real intent’. Extensive experiments on SLAKE datasets show Uni-Med’s superior accuracies (87.52% closed, 86.12% overall), outperforming MedVInT-PMC-VQA by 1.22% and 0.92%. Code and dataset are available at: https://github.com/zhongzee/Uni-Med-master.

Based on data synthesis, anomaly detection (AD) methods often rely on external data for data synthesis. However, most external abnormal data exhibits strong randomness, which may lead to a reduced range of diversity among the synthesized data. In order to achieve a broader diversity in data synthesis, it is necessary to not only have highly diverse data but also to incorporate low-diversity noise data. To enhance the diversity range of the synthesized data, this study proposes a diversity measurement assisted by image self-representation: measuring the distance between noise data and normal data and quantitatively synthesizing diversified data by selecting diverse noise data for synthesis, namely, Diversified Synthesis (DS). Diversified Synthesis introduces patch measurement and a controllable enhancement module to establish controllable diversified enhanced data. The contribution of this study lies in proposing a novel diversified synthesis method, which achieves a broader diversity synthesis through the introduction of image self-representation-assisted diversity measurement and quantitative synthesis. Furthermore, through the self-enhancement data augmentation method, the use of image intrinsic features for enhancement achieves diversity and multi-scale characteristics in the synthesized data, thereby improving the training performance of the discriminative model. This provides an effective optimization solution for comprehensive anomaly detection methods.

Vehicular Edge Computing (VEC) is a promising paradigm that leverages the vehicles to offload computation tasks to the nearby VEC server with the aim of supporting the low latency vehicular application scenarios. Incentivizing VEC servers to participate in computation offloading activities and make full use of computation resources is of great importance to the success of intelligent transportation services. In this paper, we formulate the competitive interactions between the VEC servers and vehicles as a two-stage Stackelberg game with the VEC servers as the leader players and the vehicles as the followers. After obtaining the full information of vehicles, the VEC server calculates the unit price of computation resource. Given the unit prices announced by VEC server, the vehicles determine the amount of computation resource to purchase from VEC server. In the scenario that vehicles do not want to share their computation demands, a deep reinforcement learning based resource management scheme is proposed to maximize the profits of vehicles and VEC server. The extensive experimental results have demonstrated the effectiveness of our proposed resource management scheme based on Stackelberg game and deep reinforcement learning.

The development of the Internet of Things (IoT) and intelligent vehicles brings a comfortable environment for users. Various emerging vehicular applications using artificial intelligence (AI) technologies are expected to enrich users' daily life. However, how to execute computation-intensive applications on resource-constrained vehicles based on AI still faces great challenges. In this article, we consider the vehicular computation offloading problem in mobile-edge computing (MEC), in which multiple mobile vehicles select nearby MEC servers to offload their computing tasks. We propose a multiagent deep reinforcement learning (DRL)-based computation offloading scheme, in which the uncertainty of a multivehicle environment is considered so that the vehicles can make offloading decisions to achieve an optimal long-term reward. First, we formalize a formula for the computation offloading problem. The goal of this article is to determine the optimal offloading decision to the MEC server under each observed system state, so as to minimize the total task processing delay in a long-term period. Then, we use a multiagent DRL algorithm to learn an effective solution to the vehicular task offloading problem. To evaluate the performance of the proposed offloading scheme, a large number of simulations are carried out. The simulation results verify the effectiveness and superiority of the proposed scheme.

Mobile crowdsensing is an emerging paradigm that selects users to complete sensing tasks. Recently, mobile vehicles are adopted to perform sensing data collection tasks in the urban city due to their ubiquity and mobility. In this article, we study how mobile vehicles can be optimally selected in order to collect maximum data from the urban environment in a future period of tens of minutes. We formulate the recruitment of vehicles as a maximum data limited budget problem. The application scenario is generalized to a realistic online setting where vehicles are continuously moving in real-time and the data center decides to recruit a set of vehicles immediately. A deep learning-based scheme through mobile vehicles (DLMV) is proposed to collect sensing data in the urban environment. We first propose a deep learning-based offline algorithm to predict vehicle mobility in a future time period. Furthermore, we propose a greedy online algorithm to recruit a subset of vehicles with a limited budget for the NP-Complete problem. Extensive experimental evaluations are conducted on the real mobility dataset in Rome. The results have not only verified the efficiency of our proposed solution but also validated that DLMV can improve the quantity of collected sensing data compared with other algorithms.

QoS information is often used as the basic evidence for Web services optimization. In order to collect QoS information of Web services, we propose a lightweight Server-Monitor framework. This proposal contains a WS-Monitor Model related to metamodel of QoS index information and an extending WSDL algorithm that adds QoS collect strategy into WSDL. Experimental results show that Server-Monitor obtains QoS information effectively and causes tiny negative effect on Web service.