The application of methods based on Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3D GS) have steadily gained popularity in the field of 3D object segmentation in static scenes. These approaches demonstrate efficacy in a range of 3D scene understanding and editing tasks. Nevertheless, the 4D object segmentation of dynamic scenes remains an underexplored field due to the absence of a sufficiently extensive and accurately labelled multi-view video dataset. In this paper, we present MUVOD, a new multi-view video dataset for training and evaluating object segmentation in reconstructed real-world scenarios. The 17 selected scenes, describing various indoor or outdoor activities, are collected from different sources of datasets originating from various types of camera rigs. Each scene contains a minimum of 9 views and a maximum of 46 views. We provide 7830 RGB images (30 frames per video) with their corresponding segmentation mask in 4D motion, meaning that any object of interest in the scene could be tracked across temporal frames of a given view or across different views belonging to the same camera rig. This dataset, which contains 459 instances of 73 categories, is intended as a basic benchmark for the evaluation of multi-view video segmentation methods. We also present an evaluation metric and a baseline segmentation approach to encourage and evaluate progress in this evolving field. Additionally, we propose a new benchmark for 3D object segmentation task with a subset of annotated multi-view images selected from our MUVOD dataset. This subset contains 50 objects of different conditions in different scenarios, providing a more comprehensive analysis of state-of-the-art 3D object segmentation methods. Our proposed MUVOD dataset is available athttps://volumetric-repository.labs.b-com.com/#/muvod.

Cross-modal retrieval has drawn much attention in both computer vision and natural language processing domains. With the development of convolutional and recurrent neural networks, the bottleneck of retrieval across image-text modalities is no longer the extraction of image and text features but an efficient loss function learning in embedding space. Many loss functions try to closer pairwise features from heterogeneous modalities. This paper proposes a method for learning joint embedding of images and texts using an intra-modal constraint loss function to reduce the violation of negative pairs from the same homogeneous modality. Experimental results show that our approach outperforms state-of-the-art bi-directional image-text retrieval methods on Flickr30K and Microsoft COCO datasets. Our code is publicly available1

Unlike the conventional facial expressions, micro-expressions are involuntary and transient facial expressions capable of revealing the genuine emotions that people attempt to hide. Therefore, they can provide important information in a broad range of applications such as lie detection, criminal detection, etc. Since micro-expressions are transient and of low intensity, however, their detection and recognition is difficult and relies heavily on expert experiences. Due to its intrinsic particularity and complexity, video-based micro-expression analysis is attractive but challenging, and has recently become an active area of research. Although there have been numerous developments in this area, thus far there has been no comprehensive survey that provides researchers with a systematic overview of these developments with a unified evaluation. Accordingly, in this survey paper, we first highlight the key differences between macro- and micro-expressions, then use these differences to guide our research survey of video-based micro-expression analysis in a cascaded structure, encompassing the neuropsychological basis, datasets, features, spotting algorithms, recognition algorithms, applications and evaluation of state-of-the-art approaches. For each aspect, the basic techniques, advanced developments and major challenges are addressed and discussed. Furthermore, after considering the limitations of existing micro-expression datasets, we present and release a new dataset — calledmicro-and-macro expression warehouse(MMEW) — containing more video samples and more labeled emotion types. We then perform a unified comparison of representative methods on CAS(ME)$^2$for spotting, and on MMEW and SAMM for recognition, respectively. Finally, some potential future research directions are explored and outlined.

Though there are many computational models proposed for saliency detection, few of them take object boundary information into account. This paper presents a hierarchical saliency detection model incorporating probabilistic object boundaries, which is based on the observation that salient objects are generally surrounded by explicit boundaries and show contrast with their surroundings. We perform adaptive thresholding operation on ultrametric contour map, which leads to hierarchical image segmentations, and compute the saliency map for each layer based on the proposed robust center bias, border bias, color dissimilarity and spatial coherence measures. After a linear weighted combination of multi-layer saliency maps, and Bayesian enhancement procedure, the final saliency map is obtained. Extensive experimental results on three challenging benchmark datasets demonstrate that the proposed model outperforms eight state-of-the-art saliency detection models.

A recently developed sparse representation algorithm, has been proved to be useful for multi‐object tracking and this study is a proposal for developing its parallelisation. An online dictionary learning is used for object recognition. After detection, each moving object is represented by a descriptor containing its appearance features and its position feature. Any detected object is classified and indexed according to the sparse solution obtained by an orthogonal matching pursuit (OMP) algorithm. For a real‐time tracking, the visual information needs to be processed very fast without reducing the results accuracy. However, both the large size of the descriptor and the growth of the dictionary after each detection, slow down the system process. In this work, a novel accelerating OMP algorithm implementation on a graphics processing unit is proposed. Experimental results demonstrate the efficiency of the parallel implementation of the used algorithm by significantly reducing the computation time.

This letter studies the sensing performance of random Bernoulli matrices with column size$n$much larger than row size$m$. It is observed that as the compression ratio$n/m$increases, this kind of matrices tends to present a performance floor regarding the guaranteed signal sparsity. The performance floor is effectively estimated with the formula${1 \over 2}(\sqrt {\pi m/2} + 1)$. To the best of our knowledge, it is the first time in compressed sensing, a theoretical estimation is successfully proposed to reflect the real performance.

This paper presents a novel unsupervised algorithm to detect salient regions and to segment out foreground objects from background. In contrast to previous unidirectional saliency-based object segmentation methods, in which only the detected saliency map is used to guide the object segmentation, our algorithm mutually exploits detection/segmentation cues from each other. To achieve this goal, an initial saliency map is generated by the proposed segmentation driven low-rank matrix recovery model. Such a saliency map is exploited to initialize object segmentation model, which is formulated as energy minimization of Markov random field. Mutually, the quality of saliency map is further improved by the segmentation result, and serves as a new guidance for the object segmentation. The optimal saliency map and the final segmentation are achieved by jointly optimizing the defined objective functions. Extensive evaluations on MSRA-B and PASCAL-1500 datasets demonstrate that the proposed algorithm achieves the state-of-the-art performance for both the salient region detection and the object segmentation.

This paper addresses the problem of automatic figure-ground segmentation, which aims at automatically segmenting out all foreground objects from background. The underlying idea of this approach is to transfer segmentation masks of globally and locally (glocally) similar exemplars into the query image. For this purpose, we propose a novel high-level image representation method named as object-oriented descriptor. Using this descriptor, a set of exemplar images glocally similar to the query image is retrieved. Then, using over-segmented regions of these retrieved exemplars, a discriminative classifier is learned on-the-fly and subsequently used to predict foreground probability for the query image. Finally, the optimal segmentation is obtained by combining the online prediction with typical energy optimization of Markov random field. The proposed approach has been extensively evaluated on three datasets, including Pascal VOC 2010, VOC 2011 segmentation challenges, and iCoseg dataset. Experiments show that the proposed approach outperforms state-of-the-art methods and has the potential to segment large-scale images containing unknown objects, which never appear in the exemplar images.

Low-rank matrix recovery (LRMR) model, aiming at decomposing a matrix into a low-rank matrix and a sparse one, has shown the potential to address the problem of saliency detection, where the decomposed low-rank matrix naturally corresponds to the background, and the sparse one captures salient objects. This is under the assumption that the background is consistent and objects are obviously distinctive. Unfortunately, in real images, the background may be cluttered and may have low contrast with objects. Thus directly applying the LRMR model to the saliency detection has limited robustness. This paper proposes a novel approach that exploits bottom-up segmentation as a guidance cue of the matrix recovery. This method is fully unsupervised, yet obtains higher performance than the supervised LRMR model. A new challenging dataset PASCAL-1500 is also introduced to validate the saliency detection performance. Extensive evaluations on the widely used MSRA-1000 dataset and also on the new PASCAL-1500 dataset demonstrate that the proposed saliency model outperforms the state-of-the-art models.

Recently sparse representation has been successfully applied to single object tracking by observing the reconstruction error of candidate object with sparse representation. In practice, sparse representation also shows competitive performance on multi-class classification, and thus is potential for multi-object tracking. In this paper we explore this technique for on-line multi-object tracking through a simple tracking-by-detection scheme, with background subtraction for object detection and sparse representation for object recognition. Final experiments demonstrate that the proposed approach only combining color histogram and 2-dimensional coordinates as features, achieves favorable performance over state-of-the-art work in persistent identity tracking.

We present the sparse binary matrix defined by low-density parity-check (LDPC) codes as measurement matrix in compressed sensing. This kind of matrix owns much stronger orthogonality than current other main measurement matrices. In practice, for a matrix given size, the optimal column degree for high orthogonality can also be determined by progressive edge-growth (PEG) algorithm.

The purpose of this paper is segmenting objects in an image and assigning a predefined semantic label to each object. There are two contributions in this paper. On one hand, semantic segmentation is guided by hierarchical regions instead of by single-level regions or multi-scale regions generated by multiple segmentations. On the other hand, sparse coding is introduced as high level description of the regions, which contributes to reduction of quantization error compared to traditional bag-of-visual-words method. Experiments on the challenging Microsoft Research Cambridge dataset (MSRC 21) show that our algorithm achieves state-of-the-art performance.

In IKONOS imagery, both multispectral (MS) and panchromatic data are provided, with different spatial resolutions. IKONOS images can be fused or pan-sharpened for the visual interpretation of large-area-scale applications. To date, many image fusion techniques have been developed. In particular, the intensity-hue-saturation (IHS)-based methods are well known in quick image pan-sharpening. However, most of these processes produce color distortion due to the unnatural spectral response of IKONOS sensors, particularly in vegetated areas. Hence, many recent methods make use of vegetation enhancement in order to correct the unnatural color appearance. Generally, this enhancement is accomplished using vegetation indexes (VIs). A new fusion approach that produces images with natural colors is proposed. Moreover, in this technique, a high-resolution normalized difference VI is also proposed and used in delineating the vegetation. The procedure is performed in two steps: MS fusion using the IHS technique and vegetation enhancement. Vegetation enhancement is a correction step, and it depends on the considered application. The new approach provides very good results in terms of objective quality measures. In addition, visual analysis proves that the concept of the proposed approach is promising, and it improves well the fusion quality by enhancing the vegetated zones.

In this paper, we present some preliminary results obtained with the MSGPR method in the context of pattern recognition: this new method is based on the analysis of closed contour of planar objects. The contour C/sub 0/ of a planar object is parameterised by its curvilinear abscissa u and separated into two signals x(u) and y(u) corresponding to its coordinates x and y. These signals are then filtered with a Gaussian low-pass filter. The size of the sub-consequent contour C/sub /spl sigma// is then adjusted so that it has the same size as the original contour C/sub 0/. Then the contour C/sub GC/(/spl sigma/) thus obtained and the original contour C/sub 0/ yield intersection points that give intersection points map (IPM) which is used to characterise the pattern. By construction, this method is invariant under translation. The results of analysis have shown its invariance under rotation and its resistance against scale change for a large range of scaling. It is a fast and simple algorithm that can be successfully applied in the context of pattern recognition dedicated to planar non-occluded objects. The application of this method in the research of a query pattern in a database has given very encouraging results.

This paper deals with a major problem encountered in the area of remote sensing consisting of the registration of multitemporal and/or multisensor images. In general, such images have different gray-level characteristics, and simple techniques such as those based on correlation cannot be applied directly. In this work, a new automatic satellite image registration approach is proposed. This technique exploits the invariant relations between regions of a reference and a sensed image, respectively. It involves an edge-based selection of the most distinctive control points (CPs) in the reference image. The search for the corresponding CPs in the sensed image is based on local similarity detection by means of template matching according to a combined invariants-based similarity measure. The final warping of the images according to the selected CPs is performed by using the thin-plate spline interpolation. The procedure is fully automatic and computationally efficient. The proposed algorithm for this technique has been successfully applied to register multitemporal SPOT and synthetic aperture radar images from urban and agricultural areas. The experimental results demonstrate the robustness, efficiency and accuracy of the algorithm.

In this work we present a method for multiresolution texture image segmentation via supervised Bayesian classification. This method takes into account a function called classification index CI (or Indice de Classification IC3). This function measures the accuracy rate of a pixel classification after it has been classified. The CI measure is based on the location of the pixel attribute vector in the observation space relative to the classes to which it must be classified. To reduce the time used in the classification process, we propose a new method (ASH: Algorithme de Segmentation Hierarchique3) that segments hierarchically the multiresolution images.