Salient object detection (SOD) and camouflaged object detection (COD) are related but distinct binary mapping tasks, each involving multiple modalities that share commonalities while maintaining unique characteristics. Existing approaches often rely on complex, task-specific architectures, leading to redundancy and limited generalization. Our previous work, VSCode, introduced a generalist model that effectively handles four SOD tasks and two COD tasks. VSCode leveraged VST as its foundation model and incorporated 2D prompts within an encoder-decoder framework to capture domain and task-specific knowledge, utilizing a prompt discrimination loss to optimize the model. Building upon the proven effectiveness of our previous work VSCode, we identify opportunities to further strengthen generalization capabilities through focused modifications in model design and optimization strategy. To unlock this potential, we propose VSCode-v2, an extension that introduces a Mixture of Prompt Experts (MoPE) layer to generate adaptive prompts. We also redesign the training process into a two-stage approach: first learning shared features across tasks, then capturing specific characteristics. To preserve knowledge during this process, we incorporate distillation from our conference version model. Furthermore, we propose a contrastive learning mechanism with data augmentation to strengthen the relationships between prompts and feature representations. VSCode-v2 demonstrates balanced performance improvements across six SOD and COD tasks. Moreover, VSCode-v2 effectively handles various multimodal inputs and exhibits zero-shot generalization capability to novel tasks, such as RGB-D Video SOD.

Audio-Visual Segmentation (AVS) faces a fundamental challenge of effectively aligning audio and visual modalities. While recent approaches leverage foundation models to address data scarcity, they often rely on single-modality knowledge or combine foundation models in an off-the-shelf manner, failing to address the cross-modal alignment challenge. In this paper, we present TAViS, a novel framework that \textbf{couples} the knowledge of multimodal foundation models (ImageBind) for cross-modal alignment and a segmentation foundation model (SAM2) for precise segmentation. However, effectively combining these models poses two key challenges: the difficulty in transferring the knowledge between SAM2 and ImageBind due to their different feature spaces, and the insufficiency of using only segmentation loss for supervision. To address these challenges, we introduce a text-bridged design with two key components: (1) a text-bridged hybrid prompting mechanism where pseudo text provides class prototype information while retaining modality-specific details from both audio and visual inputs, and (2) an alignment supervision strategy that leverages text as a bridge to align shared semantic concepts within audio-visual modalities. Our approach achieves superior performance on single-source, multi-source, semantic datasets, and excels in zero-shot settings.

In order to improve the performance of target localization based on multi-sensor system, this paper proposes a novel sparse acoustic vector sensor (AVS) array with nested moving sampling scheme, for underdetermined two-dimensional (2D) direction of arrival (DOA) estimation of coherent target signals. By exploiting the dense sampling of the nested sampling scheme, a synthetic array composed of the original and shifted sparse arrays, is achieved with increased degrees of freedom (DOF) and extended aperture. Based on it, the sparse sampling in the sampling scheme is applied to decorrelate the coherent signals, and then eigenstructure based methods are used for high-accuracy 2D DOA estimation in underdetermined scenario. The simulation results are given to demonstrate the effectiveness of the proposed method over the state-of-the-art techniques.

Salient object detection (SOD) and camouflaged object detection (COD) are related yet distinct binary mapping tasks. These tasks involve multiple modalities, sharing commonalities and unique cues. Existing research often employs intricate task-specific specialist models, potentially leading to redundancy and suboptimal results. We introduce VS-Code, a generalist model with novel 2D prompt learning, to jointly address four SOD tasks and three COD tasks. We utilize VST as the foundation model and introduce 2D prompts within the encoder-decoder architecture to learn domain and task-specific knowledge on two separate dimensions. A prompt discrimination loss helps disentangle peculiarities to benefit model optimization. VSCode outperforms state-of-the-art methods across six tasks on 26 datasets and exhibits zero-shot generalization to unseen tasks by combining 2D prompts, such as RGB-D COD. Source code has been available at https://github.com/Sssssuperior/VSCode.

Fine-grained remote sensing object detection aims at precisely locating objects and determining the fine-level categories. This task is exceptionally challenging due to the substantial interclass similarity, presenting difficulties in capturing discriminative features. We attribute this to the absence of essential information that can serve as supervision for the learning. This involves comprehensive visual patterns of objects and intrinsic relationships of multigranular features. In this article, we propose a novel scheme dubbed as digging into the multigranular archetype (DIMA) for fine-grained remote sensing object detection. In detail, we first design a simple yet effective frequency-aware representation supplement (FARS) mechanism learning from original images and their auxiliary frequency counterparts simultaneously. The FARS introduces high- and low-frequency representations to reinforce a range of visual cues, such as particular regions associated with the former and contours of objects related to the latter. Then, we further devise a module named hierarchical classification paradigm (HCP), which constructs the interhierarchy relationships between coarse and fine-level representations and then exploits them to guide fine-grained feature enhancement. HCP eventually selects and boosts samples that are hard to discriminate by keeping consistency in multilevels. Our method can be easily integrated into prevailing oriented object detectors and brings consistent performance improvements across these detectors. Notably, our method combined with oriented RCNN (ORCNN) achieves 44.44% (+3.62%) on the FAIR1M and 91.0% (+6.9%) on the MAR20. Moreover, thoughtful discussions about qualitative results and rich visualizations are provided to intuitively underscore the superiority of our approach. The source code is available athttps://github.com/chengjc2019/DIMA.

Joint application of radio-frequency identification (RFID) and ultrawideband (UWB) technologies in the intelligent warehousing management system is proposed. In this system, we regard forklift as infrastructure, and both the UWB mobile terminal (MT) and the RFID reader are mounted on the forklift. The RFID reader is used not only to read the information of goods but also to determine the goods' status of loading and unloading. The UWB MT is used to locate the forklift. The goods or pallets are labeled with RFID tags. Utilizing the integration of these two technologies, the dual goals of goods information and goods location perception are achieved. An M/N-K sliding window method is proposed to determine the loading and unloading of goods in this article. Our experiments reveal that this novel method can quickly, accurately, and efficiently determine the states of the goods on the forklift. For the indoor localization, an algorithm is proposed based on RSS residual weighting (RRW). Experiments show that RRW can mitigate the nonline-of-sight error substantially compared with the conventional Taylor algorithm and recent Convex approximation algorithm. Finally, a real working practice in a warehouse of a company is introduced, and it illustrates the feasibility of the system.

More recently, due to the enormous potential of long short‐term memory (LSTM) neural network in various fields, some efficient training algorithms have been developed, including the extended Kalman filter (EKF)‐based training algorithm and particle filter (PF)‐based training algorithm. However, it should be noted that if the system is highly non‐linear, the linearisation employed in the EKF may cause instability. Moreover, the PF usually suffers from the particle degeneracy. Therefore, the PF‐based training algorithm may only find a poor local optimum. To solve these problems, an unscented Kalman filter (UKF)‐based training algorithm is proposed. The UKF employs a deterministic sampling method; hence, there is no linearisation in it and it does not have the degeneracy problem. Moreover, the computational complexity of the UKF is the same order as that of the EKF. To further reduce the computational complexity, the authors propose a minimum norm UKF (MN‐UKF) to obtain a good trade‐off between performance and complexity. To the best of the authors’ knowledge, this is the first reported solution to this problem. Simulations using both benchmark synthetic signal and real‐world signal illustrate the potential of the algorithms developed.

We describe a spatially scalable video coding framework in which motion correspondences between successive video frames are exploited in the wavelet transform domain. The basic motivation for our coder is that motion fields are typically smooth and, therefore, can be efficiently captured through a multiresolutional framework. A wavelet decomposition is applied to each video frame and the coefficients at each level are predicted from the coarser level through backward motion compensation. To remove the aliasing effects caused by downsampling in the transform, a special interpolation filter is designed with the weighted aliasing energy as part of the optimization goal, and motion estimation is carried out with low pass filtering and interpolation in the estimation loop. Further, to achieve robust motion estimation against quantization noise, we propose a novel backward/forward hybrid motion compensation scheme, and a tree structured dynamic programming algorithm to optimize the backward/forward mode choices. A novel adaptive quantization scheme is applied to code the motion predicted residue wavelet coefficients, Experimental results reveal 0.3-2-dB increase in coded PSNR at low bit rates over the state-of-the-art H.263 standard with all enhancement modes enabled, and similar improvements over MPEG-2 at high bit rates, with a considerable improvement in subjective reconstruction quality, while simultaneously supporting a scalable representation.

Multiple description coding refers to the encoding of an information source into multiple bit streams, in a manner that each bit stream independently represents a "coarse" description of the source, while multiple descriptions jointly convey a higher fidelity source representation. Thus multiple description coding allows enhanced resilience to loss of individual descriptions, and can be viewed as a form of source-based diversity coding. This paper addresses the design of subband filter banks to optimize the rate-distortion performance of multiple description subband coding. The problem is formulated in a Lagrangian optimization framework, and is solved directly in the frequency domain to produce the optimal filter spectral responses. By varying the Lagrangian parameter, our design obtains all points on the redundancy-distortion curve for Gaussian wide-sense stationary input processes. We consider the cases of both orthonormal and more general biorthogonal filter banks. We also analyze the connections between these two solutions, and quantify the gap in their optimal performance. Application of the proposed methods to the popular first-order autoregressive (AR(1)) model yields very interesting and insightful results. In the two extreme cases of maximum and minimum redundancy, our solutions degenerate to previously known results. Finally, comparisons with popularly deployed filter banks reveal large gaps from the theoretical performance bound.

We introduce a novel region-based video compression framework based on morphology to efficiently capture motion correspondences between consecutive frames in an image sequence. Our coder is built on the observation that the motion field associated with typical image sequences can be segmented into component motion subfield "clusters" associated with distinct objects or regions in the scene, and further, that these clusters can be efficiently captured using morphological operators in a "backward" framework that avoids the need to send region shape information. Region segmentation is performed directly on the motion field by introducing a small "core" for each cluster that captures the essential features of the cluster and reliably represents its motion behavior. Cluster matching is used in lieu of the conventional block matching methods of standard video coders to define a cluster motion representation paradigm. Furthermore, a region-based pel-recursive approach is applied to find the refinement motion field for each cluster and the cluster motion prediction error image is coded by a novel adaptive scalar quantization method. Experimental results reveal a 10-20% reduction in prediction error energy and 1-3 dB gain in the final reconstructed peak signal-to-noise ratio (PSNR) over the standard MPEG-1 coder at typical bit rates of 500 Kb/s to 1 Mb/s on standard test sequences, while also requiring lower computational complexity.

Multiple description coding refers to encoding an information source into multiple bitstreams, in a way that various levels of tradeoff between decoding quality and channel protection can be supported. This paper addresses the design of two-channel orthonormal filter banks to optimize the redundancy rate-distortion performance of multiple description subband coding. The problem is formulated in a Lagrangian optimization framework, and is solved directly in the frequency domain to produce the optimal filter responses. By varying the Lagrangian parameter, our design obtains all points on the information redundancy rate-distortion curve for Gaussian wide sense stationary input processes. Application of the proposed algorithm to the popular AR(1) model yields very interesting and insightful results. Comparisons with popularly deployed filter banks reveal large gaps from the theoretical performance bound.

We address practical issues which arise in the implementation of optimized cache replenishment algorithms within a "soft" caching framework. We study the algorithms that have been proposed for optimized soft caching and simulate them using actual proxy traces. Our objective is to determine what compromises have to be made in order to approximate the desired optimal performance while maintaining a complexity level sufficiently low to enable a real-time implementation.

We describe a spatially scalable video coding framework in which motion correspondences between successive video frames are exploited in the wavelet transform domain in a purely backward fashion. The basic motivation for our coder is that motion fields are typically smooth and therefore can be efficiently captured through a multiresolutional framework. A wavelet decomposition is applied on each video frame and the coefficients at each level are predicted from the coarser level through backward motion compensation. To remove the aliasing effects caused by downsampling in the transform, a special interpolation filter is designed with the weighted aliasing energy as part of the optimization goal, and motion estimation is carried out with lowpass filtering and interpolation in the estimation loop. A novel adaptive quantization scheme is then applied to code the motion predicted residue wavelet coefficients in each band. Experimental results reveal typically a 0.5-2 dB increase on coded PSNR at 24-48 kb/s over the state-of-the-art H.263 standard, with also a considerable improvement in subjective reconstruction quality.

We introduce a novel region-based video compression framework that uses a morphological operation to efficiently capture motion correspondences between consecutive frames in an image sequence. Our coder is built on the observation that the motion field associated with typical image sequences can be segmented into component motion subfield "clusters" associated with distinct objects or regions in the scene, and further that these clusters can be efficiently captured using morphological operators in a "backward" framework that avoids needing to explicitly send object boundaries. Cluster matching is used in lieu of the conventional block matching methods of standard video codecs to define a cluster motion representation paradigm. Experimental coding results show about 10-20% reduction in prediction error energy and 0.3-1 db (average of about 0.44 dB) reduction in the final residue-coded peak signal to noise ratio (PSNR) using our proposed motion compensation framework on the football sequence over standard block motion compensation methods like MPEG, while also requiring less computational complexity.

We introduce a novel region-based video compression framework based on morphology to efficiently capture motion correspondences between consecutive frames in an image sequence. Our coder is built on the observation that the motion field associated with typical image sequences can be segmented into component motion subfield "clusters" associated with distinct objects or regions in the scene, and further that these clusters can be efficiently captured using morphological operators in a "backward" framework that avoids the need to send region shape information. The cluster motion field is obtained using a novel "cluster matching" approach which is computationally efficient due to the use of only a small cluster "core" that serves as a reliable representative of the entire cluster. A novel morphological dilation operation is used to expand the cluster based on the motion information, and quantization of the cluster motion prediction error is directly incorporated in the region-growing process. Experimental results reveal a 10-30% reduction in coding bit rate over the standard MPEG-1 coder at typical bit rates of 500 kbits/sec to 1 Mb/sec on standard test sequences, while also requiring much lower computational complexity.