Learning from a large corpus of data, pre-trained models have achieved impressive progress nowadays. As a popular generative pre-training method, diffusion models stand out by capturing both low-level visual knowledge and high-level semantic relations. In this paper, we propose to exploit such knowledgeable pre-trained diffusion models for mainstream discriminative tasks such as annotation-free saliency segmentation. However, a notable structural discrepancy between generative and discriminative models poses a significant challenge to diffusion models’ direct application. Furthermore, the absence of explicit manually labeled data is a substantial barrier in annotation-free settings. To tackle these issues, we introduce FreeSegDiff, one novel synthesis-exploitation framework containing two-stage strategies. In the first synthesis stage, to alleviate data insufficiency, we synthesize abundant images, and propose a novel training-free DiffusionCut to produce masks. In the second exploitation stage, to bridge the structural gap, we employ the inversion technique to convert given images back to diffusion features. These features seamlessly integrate with downstream architectures. Extensive experiments and ablation studies demonstrate the superiority of adapting diffusion for annotation-free saliency segmentation.

Temporal sentence grounding aims to detect event timestamps described by the natural language query from given untrimmed videos. The existing fully-supervised setting achieves great results but requires expensive annotation costs; while the weakly-supervised setting adopts cheap labels but performs poorly. To pursue high performance with less annotation costs, this paper introduces an intermediate partially-supervised setting, i.e., only short-clip is available during training. To make full use of partial labels, we specially design one contrast-unity framework, with the two-stage goal of implicit-explicit progressive grounding. In the implicit stage, we align event-query representations at fine granularity using comprehensive quadruple contrastive learning: event-query gather, event-background separation, intra-cluster compactness and inter-cluster separability. Then, high-quality representations bring acceptable grounding pseudo-labels. In the explicit stage, to explicitly optimize grounding objectives, we train one fully-supervised model using obtained pseudo-labels for grounding refinement and denoising. Extensive experiments and thoroughly ablations on Charades-STA and ActivityNet Captions demonstrate the significance of partial supervision, as well as our superior performance.

Video understanding is a complex challenge that requires effective modeling of spatial-temporal dynamics. With the success of image foundation models (IFMs) in image understanding, recent approaches have explored parameter-efficient fine-tuning (PEFT) to adapt IFMs for video. However, most of these methods tend to process spatial and temporal information separately, which may fail to capture the full intricacy of video dynamics. In this paper, we propose MoMa, an efficient adapter framework that achieves full spatial-temporal modeling by integrating Mamba's selective state space modeling into IFMs. We propose a novel SeqMod operation to inject spatial-temporal information into pre-trained IFMs, without disrupting their original features. By incorporating SeqMod into a Divide-and-Modulate architecture, MoMa enhances video understanding while maintaining computational efficiency. Extensive experiments on multiple video benchmarks demonstrate the effectiveness of MoMa, achieving superior performance with reduced computational cost. Codes will be released upon publication.

Referring Image Segmentation (RIS) aims to segment the target object in an image given a natural language expression. While recent methods leverage pre-trained vision backbones and more training corpus to achieve impressive results, they predominantly focus on simple expressions—short, clear noun phrases like “red car” or “left girl”. This simplification often reduces RIS to a key word/concept matching problem, limiting the model’s ability to handle referential ambiguity in expressions. In this work, we identify two challenging real-world scenarios: object-distracting expressions, which involve multiple entities with contextual cues, and category-implicit expressions, where the object class is not explicitly stated. To address the challenges, we propose a novel framework, SaFiRe, which mimics the human two-phase cognitive process—first forming a global understanding, then refining it through detail-oriented inspection. This is naturally supported by Mamba’s scan-then-update property, which aligns with our phased design and enables efficient multi-cycle refinement with linear complexity. We further introduce aRefCOCO, a new benchmark designed to evaluate RIS models under ambiguous referring expressions. Extensive experiments on both standard and proposed datasets demonstrate the superiority of SaFiRe over state-of-the-art baselines. Project page: https://zhenjiemao.github.io/SaFiRe/.

With the development of information and communication technology, the Internet of Things (IoT) has gained significant popularity across various applications. As the number of devices surges and data generation accelerates, robust data security and accurate data retrieval have become increasingly important. Fuzzy keyword search provides an elegant way to allow retrieval over encrypted data. Privacy and accuracy are two important factors when applying fuzzy keyword search into cloud storage systems. State-of-the-art mechanisms cannot effectively balance privacy and accuracy, and they even compromise the access pattern. In this paper, we propose SeaPA, a fuzzy keyword search scheme with strong privacy and high accuracy. SeaPA is built on top of two non-colluding servers and integrated with secure multi-party computation. To construct SeaPA, we introduce a top-k document retrieval algorithm that conceals the similarity of search results from identical queries, preventing access pattern leakage. We implement a prototype system and conduct extensive experiments on the 20NewsGroups dataset. The results show that SeaPA achieves higher accuracy, with over$16\times$speedups in index encryption efficiency and over$9.3\times$speedups in search efficiency, compared to previous schemes.

The objective of Audio-Visual Segmentation (AVS) is to localise the sounding objects within visual scenes by accurately predicting pixel-wise segmentation masks. To tackle the task, it involves a comprehensive consideration of both the data and model aspects. In this paper, first, we initiate a novel pipeline for generating artificial data for the AVS task without extra manual annotations. We leverage existing image segmentation and audio datasets and match the image-mask pairs with its corresponding audio samples using category labels in segmentation datasets, that allows us to effortlessly compose (image, audio, mask) triplets for training AVS models. The pipeline is annotation-free and scalable to cover a large number of categories. Additionally, we introduce a lightweight model SAMA-AVS which adapts the pre-trained segment anything model (SAM) to the AVS task. By introducing only a small number of trainable parameters with adapters, the proposed model can effectively achieve adequate audio-visual fusion and interaction in the encoding stage with vast majority of parameters fixed. We conduct extensive experiments, and the results show our proposed model remarkably surpasses other competing methods. Moreover, by using the proposed model pretrained with our synthetic data, the performance on real AVSBench data is further improved, achieving 83.17 mIoU on S4 subset and 66.95 mIoU on MS3 set. The project page is https://jinxiang-liu.github.io/anno-free-AVS/.

Industrial Wireless Sensor Networks (IWSNs) have been widely used in industrial applications that require highly reliable and real-time wireless transmission. A lot of works have been done to optimize the Relay Node Placement (RNP), which determines the underlying topology of IWSNs and hence impacts the network performance. However, existing RNP algorithms use a fixed communication radius to compute the deployment result at once offline, while ignoring that the radio environment may vary drastically across different locations, also known as radio irregularity. To address this limitation, we propose a Voronoi diagram and Q-learning based RNP (VQRNP) method in this article. Instead of using a fixed communication radius, VQRNP employs the Q-learning algorithm to dynamically update the radio environment of measured areas, uses a Voronoi diagram based method to estimate the radio environment of unmeasured areas, and proposes a coverage extension location selection algorithm to place RNs so as to extend the coverage of the deployed network based on the results estimated by Voronoi diagram based Graph Generating (VGG). In this way, the VQRPN method can adapt itself well to the variation of radio environment and largely speed up the deployment process. Extensive simulations verify that VQRNP significantly outperforms existing RNP algorithms in terms of reliability.

This paper studies the problem of weakly open-vocabulary semantic segmentation (WOVSS), which learns to segment objects of arbitrary classes using mere image-text pairs. Existing works turn to enhance the vanilla vision transformer by introducing explicit grouping recognition, i.e., employing several group tokens/centroids to cluster the image tokens and perform the group-text alignment. Nevertheless, these methods suffer from a granularity inconsistency regarding the usage of group tokens, which are aligned in the all-to-one v.s. one-to-one manners during the training and inference phases, respectively. We argue that this discrepancy arises from the lack of elaborate supervision for each group token. To bridge this granularity gap, this paper explores explicit supervision for the group tokens from the prototypical knowledge. To this end, this paper proposes the non-learnable prototypical regularization (NPR) where non-learnable prototypes are estimated from source features to serve as supervision and enable contrastive matching of the group tokens. This regularization encourages the group tokens to segment objects with less redundancy and capture more comprehensive semantic regions, leading to increased compactness and richness. Based on NPR, we propose the prototypical guidance segmentation network (PGSeg) that incorporates multi-modal regularization by leveraging prototypical sources from both images and texts at different levels, progressively enhancing the segmentation capability with diverse prototypical patterns. Experimental results show that our proposed method achieves state-of-the-art performance on several benchmark datasets.

In smart grids, various Internet-of-Things-based (IoT-based) components are massively deployed across the power systems. However, most of these IoT-based components have their own vulnerabilities, leveraging which malicious users can launch different cyber/physical attacks to steal electricity. Economic losses caused by electricity theft amount to $96 billion in 2017. Most existing electricity theft detection techniques suffer from either a high deployment cost or a low detection accuracy. To address these concerns, we propose a novel Electricity Theft Detector based upon Convolutional Long Short Term Memory neural networks, called ETD-ConvLSTM. By installing a central observer meter in each community, we can know which communities have malicious users. For these communities, users’ time series of electricity consumptions with temporal correlations are transformed into spatio-temporal sequence data, mainly by constructing a two-dimensional matrix containing both consumptions and consumption differences among several adjacent days. This matrix is then divided into a sequence of sub-matrices, which are then fed into a ConvLSTM network consisting of multiple stacked ConvLSTM layers, with each layer formed by several temporarily concatenated ConvLSTM nodes. When capturing the periodicity in users’ consumption patterns, the ETD-ConvLSTM method considers both global and local knowledge, and hence the detection accuracy improves significantly. Simulations results show that compared with existing state-of-the-art detectors, the proposed ETD-ConvLSTM method can obtain better or comparable performance in terms of detection accuracy, false negative rates and false positive rates within much shorter detection time.

The increasingly wide utilization of Wireless Sensor Networks (WSNs) in industrial applications outstands the significance of the Delay Constrained Relay Node Placement (DCRNP) problem. Existing algorithms to the DCRNP problem are designed based on the ideal geometric disk wireless channel model, and no real-world deployments are performed to verify the effectiveness of these algorithms. However, the unreliable and unpredictable wireless links in WSNs may lead these algorithms to fail in practice. Therefore, we first conduct extensive real-world deployments under the guidance of existing algorithms to evaluate their performance and to gain some insights for designing practical deployment algorithms. The results exhibit that the WSNs built by existing algorithms have a favorable performance in end-to-end delay but a poor performance in reliability, which is mainly due to the lack of methods ensuring high-quality links. To this end, we first devise a Set-Covering-based Algorithm (SCA) which figures out the DCRNP problem while ensuring the quality of each link better than a given threshold. As our experiments also show that the fault-tolerant topology can significantly improve network reliability, we then design a k-Set-Covering-based Algorithm (kSCA) to build fault-tolerant WSNs based on the methodology of SCA. Furthermore, the elaborate analysis proves that both SCA and kSCA are polynomial-time algorithms, and their approximation ratios are both O(ln n), where n is the number of sensor nodes. Finally, extensive experiments are performed under the guidance of SCA and kSCA to demonstrate the effectiveness of these two algorithms.

Interactive segmentation has recently been explored to effectively and efficiently harvest high-quality segmentation masks by iteratively incorporating user hints. While iterative in nature, most existing interactive segmentation methods tend to ignore the dynamics of successive interactions and take each interaction independently. We here propose to model iterative interactive image segmentation with a Markov decision process (MDP) and solve it with reinforcement learning (RL) where each voxel is treated as an agent. Considering the large exploration space for voxel-wise prediction and the dependence among neighboring voxels for the segmentation tasks, multi-agent reinforcement learning is adopted, where the voxel-level policy is shared among agents. Considering that boundary voxels are more important for segmentation, we further introduce a boundary-aware reward, which consists of a global reward in the form of relative cross-entropy gain, to update the policy in a constrained direction, and a boundary reward in the form of relative weight, to emphasize the correctness of boundary predictions. To combine the advantages of different types of interactions, i. e., simple and efficient for point-clicking, and stable and robust for scribbles, we propose a supervoxel-clicking based interaction design. Experimental results on four benchmark datasets have shown that the proposed method significantly outperforms the state-of-the-arts, with the advantage of fewer interactions, higher accuracy, and enhanced robustness.

The centralized computing model in traditional Wireless Sensor/Actuator Networks (WSANs) can lead to large delays and unbalances, which severely restricts the adoption of WSANs in applications requiring high network performance. To address this limitation, the in-network computation model has been proposed, in which the computation capability is distributed among wireless nodes in WSANs, i.e., wireless nodes perform not only data communication but also data processing. Node placement is a primary step to build the underlaying topologies of WSANs. Nevertheless, the problem of node placement to design underlaying network topologies supporting in-network computation is still unexplored. To this end, we propose an In-network-oriented Node Placement Algorithm (INPA) to build WSANs supporting in-network computation. Moreover, we investigate the time complexity of INPA and verify the efficiency of INPA through extensive simulations.

In the era of industrial 4.0, industrial Internet of Things (IIoT) has brought essential changes to human society. For IIoT, communication in network can be defined as the basic condition for further development and integrated information exchange. In this way, cached-enabled heterogeneous industrial network is necessary to be optimized. In this paper, we consider the optimal geographical placement of contents in cache-enabled heterogeneous networks to minimize the total missing probability. And the probability represents that typical user cannot find requested file in the nearby base stations (BSs). In contract to existing works which only concern content placement, we jointly optimize content placement at BSs and activation densities of BSs of different tiers subject to the cache size limits and the constraint on the BSs energy consumption cost. In addition, the user distribution in this work is modeled by a homogeneous Poisson Point Process. We prove that the original optimization problem can be transformed to a convex problem. The convexity of the optimization problem allows us to apply the KKT conditions to derive useful analytical results of the optimal solution. Based on this, we propose a low-complexity near-optimal algorithm to find the approximated content placement probabilities. We further extend the optimization to heterogeneous networks with the user distribution modeled by the modified Cluster Process. Extensive simulation results show the superior performance of joint optimization of content placement and BSs activation densities compared to only optimizing content placement.

Wireless Sensor Networks (WSNs) are gradually employed in many applications requiring real-time data transmission. As hop count is an important factor affecting end-to-end delay, in this paper, we investigate the Hop Constrained Relay Node Placement (HCRNP) problem where at least one path fulfilling the hop constraint is built between each Sensor Node (SN) and the sink. To address this problem, we present a Cover-based Relay Node Placement (CRNP) algorithm which places Relay Nodes (RNs) from SNs to the sink. Through formulating the deployment of RNs in each iteration as a cover problem (the set cover problem for arbitrary settings or the discrete unit disk cover problem for special settings) with respect to hop constraint, the CRNP algorithm iteratively deploys RNs adjacent to the SNs or the previously placed RNs so as to gradually connect SNs to the sink. Through rigorous analysis, we show that the CRNP algorithm has an approximation ratio better than existing algorithms for the HCRNP problem (i.e., O(1) for special settings and O(ln n) for arbitrary settings, where n is the number of SNs). Finally, we conduct extensive simulations to verify the effectiveness of the proposed algorithm.

The lifetime of Wireless Sensor Networks (WSNs) is significantly shortened by the energy hole problem that is caused by the many-to-one communication pattern adopted by most WSNs. Various approaches have been designed to solve the energy hole problem, and this paper considers improving the energy efficiency by deploying additional relays, which is called the Lifetime Constrained Relay Node Placement (LCRNP) problem. To address the NP-hardness of the LCRNP problem, this paper proposes a Cluster-based Approximation Algorithm (CAA) that first groups the sensors into different clusters in which the lifetime constraint can be ignored and sensors are close to each other, and then builds network connectivity for each cluster. Next, the Augmented CAA is designed based on the CAA to further improve network lifetime by building addition paths for the relays prone to suffer heavy traffic loads. Unlike existing works, we prove that the proposed algorithms can guarantee polynomial time complexities and explicit approximation ratios. Finally, the efficiency of the proposed algorithms is verified through extensive simulations.

As Wireless Sensor Networks (WSNs) are widely used in time-critical applications, e.g., factory automation and smart grid, the importance of Delay Constrained Relay Node Placement (DCRNP) problem is becoming increasingly noticeable. This paper proposes a Set-Covering-based Approximation (SCA) algorithm to solve the DCRNP problem. The SCA deploys relay nodes by levels from the sink to sensor nodes. To avoid the limitation suffering by existing algorithms and ensure a polynomial time complexity, SCA employs a novel approach to formulate the deployment of relay nodes at each level as the set covering problem subject to delay constraints, and based on the classic greedy-set-covering algorithm, a set of relay nodes are placed to connect the nodes (sensor nodes and relay nodes) that are already connected to the sink. Since delay constraints are met at each level, all the sensor nodes will be connected to the sink via feasible paths fulfilling delay constraints. In addition, the elaborated analysis of the time complexity and the approximation ratio of the SCA algorithm is given out. Extensive simulations show that SCA can significantly save deployed relay nodes in comparison to existing algorithms.

In two-tiered Wireless Sensor Networks (WSNs) relay node placement considering resource constraints and high overhead of the relay nodes plays a key role in extending the network lifetime. Therefore, approaches that support fewer relay nodes are desired to cover the WSNs. In this paper, we formulate the relay node placement problem as a Geometric Disc Covering (GDC) problem, and propose a novel local search approximation algorithm (LSAA) to solve the GDC problem. In the proposed LSAA, the sensor nodes are allocated into independent groups and then a Set Cover (SC) for each group is performed. The set of the SC for each group constitutes a SC of the GDC problem. LSAA is extensively investigated and analyzed by rigorous proof and the simulation results presented in this paper clearly demonstrate that the proposed LSAA outperform the approaches reported in literature in the reduction in deployed relay nodes.