Automatic sign language translation has gained particular interest in the computer vision and computational linguistics communities in recent years. Given each sign language country’s particularities, machine translation requires local data to develop new techniques and adapt existing ones. This work presents iLSU-T, an open dataset of interpreted Uruguayan Sign Language RGB videos with audio and text transcriptions. This type of multimodal and curated data is paramount for developing novel approaches to understand or generate tools for sign language processing. iLSU-T comprises more than 185 hours of interpreted sign language videos from public TV broadcasting. It covers diverse topics and includes the participation of 18 professional interpreters of sign language. A series of experiments using three state-of-the-art translation algorithms is presented. The aim is to establish a baseline for this dataset and evaluate its usefulness and the proposed pipeline for data processing. The experiments highlight the need for more localized datasets for sign language translation and understanding, which are critical for developing novel tools to improve accessibility and inclusion of all individuals. Our data and code can be accessed at https://github.com/ariel-e-stassi/iLSU-T.

An automated method to analyze personal record cards generated by Organismo Coordinador de Operaciones Antisubversivas (OCOA) during the civic-military dictatorship in Uruguay between 1973 and 1985 is presented. These personal records are part of Archivo Berrutti, a collection of digitized documents, partially processed by the Uruguayan government. The main goal of this study is to extract the maximum amount of information from the personal record cards to ease the analysis by specialized teams. To achieve the goal, a methodology which combines image processing and pattern recognition techniques has been developed. This methodology takes advantage of the known geometric structure of the cards to straighten them, identify and extract relevant pieces, classify them, extract relevant fields, and transcribe crucial information such as names and identification numbers.

Multi-view stereo reconstruction of scenes from satellite images is traditionally performed with a pair-wise stereovision approach: (1) multiple views are grouped into pairs, (2) each pair is processed by two-view stereo methods producing an elevation model or point cloud, lastly (3) the pairwise reconstructions are integrated and filtered to obtain a final result. These steps are organized in a pipeline and the end-to-end performance of reconstructions depends on the behavior of these steps. This work introduces two changes that increase the performance of the reconstructions: a new pair selection approach and a new integration method are presented. The new pair selection replaces commonly used heuristics with a principled criterion that predicts the completeness of a pair based on offline simulations. The presented integration method is based on an iterated bilateral filter. Experiments show that these changes yield a systematic improvement on the performance of the pipeline.

Different methods can be applied to satellite images to derive an altitude map from a set of images. In this article we evaluate a set of representative methods from different approaches. We consider true multi-view stereo methods as well as pair-wise ones, classic methods and deep learning based ones, methods already in use on satellite images and others that were originally devised for close range imaging and are adapted to satellite imagery. While deep learning (DL) methods have taken over multi-view stereo reconstruction in the last years, this tendency has not fully reached satellite stereo pipelines that still largely rely on pair-wise classic algorithms. For the comparison, we set-up a framework that allows to interface a DL-based stereo method taken from the computer vision literature with a satellite stereo pipeline. For multi-view stereo algorithms we build on a recently proposed framework originally devised to apply Colmap method to satellite images. Methods are compared on several datasets that include sets of images taken within a few days and sets of images taken months apart. Results show that DL methods have, in general, a good generalization power. In particular, the use of the GANet DL method as the matching step in a pair-wise stereo pipeline is promising as it already performs better than the classic counterpart, even without a specific training.

This work explores the connections between the minimum description length (MDL) principle as developed by Rissanen, and the a-contrario framework for structure detection proposed by Desolneux, Moisan, and Morel. The MDL principle focuses on the best interpretation for the whole data while the a-contrario approach concentrates on detecting parts of the data with anomalous statistics. Although framed in different theoretical formalisms, we show that both methodologies share many common concepts and tools in their machinery and yield very similar formulations in a number of interesting scenarios ranging from simple toy examples to practical applications such as polygonal approximation of curves and line segment detection in images. We also formulate the conditions under which both approaches are formally equivalent.

In spite of many interesting attempts, the problem of automatically finding alignments in a 2D set of points seems to be still open. The difficulty of the problem is illustrated here by very simple examples. We then propose an elaborate solution. We show that a correct alignment detection depends on not less than four interlaced criteria, namely the amount of masking in texture, the relative bilateral local density of the alignment, its internal regularity, and finally a redundancy reduction step. Extending tools of the a contrario detection theory, we show that all of these detection criteria can be naturally embedded in a single probabilistic a contrario model with a single user parameter, the number of false alarms. Our contribution to the a contrario theory is the use of sophisticated conditional events on random point sets, for which expectation we nevertheless find easy bounds. By these bounds the mathematical consistency of our detection model receives a simple proof. Our final algorithm also includes a new formulation of the exclusion principle in Gestalt theory to avoid redundant detections. Aiming at reproducibility, a source code and an online demo open to any data point set are provided. The method is carefully compared to three state-of-the-art algorithms and an application to real data is discussed. Limitations of the final method are also illustrated and explained.

We present a novel method for automatic vanishing point detection based on primal and dual point alignment detection. The very same point alignment detection algorithm is used twice: First in the image domain to group line segment endpoints into more precise lines. Second, it is used in the dual domain where converging lines become aligned points. The use of the recently introduced PClines dual spaces and a robust point alignment detector leads to a very accurate algorithm. Experimental results on two public standard datasets show that our method significantly advances the state-of-the-art in the Manhattan world scenario, while producing state-of-the-art performances in non-Manhattan scenes.

We will consider the problem of detecting configurations of points regularly spaced and lying on a smooth curve. This corresponds to the notion of good continuation introduced in the Gestalt theory. We present a robust algorithm for clustering points along such curves, whilst at the same time discarding noisy samples. Based on the a contrario methodology, the detector builds upon a simple, symmetric primitive for a triplet of points, and finds statistically meaningful chains of such triplets. An efficient implementation is proposed using the Floyd-Warshall algorithm. Experiments on synthetic and real data show that the method is able to identify the perceptually relevant configuration of points in good continuation.

We present a novel segmentation algorithm based on a hierarchical representation of images. The main contribution of this work is to explore the capabilities of the a contrario reasoning when applied to the segmentation problem and to overcome the limitations of current algorithms within that framework. This exploratory approach has three main goals. Our first goal is to extend the search space of greedy merging algorithms to the set of all partitions spanned by a certain hierarchy and to cast the segmentation as a selection problem within this space. In this way we increase the number of tested partitions, and thus we potentially improve the segmentation results. In addition, this space is considerably smaller than the space of all possible partitions, and thus we still keep the complexity controlled. Our second goal aims to improve the locality of region merging algorithms, which usually merge pairs of neighboring regions. In this work, we overcome this limitation by introducing a validation procedure for complete partitions rather than for pairs of regions. The third goal is to perform an exhaustive experimental evaluation methodology in order to provide reproducible results. Finally, we embed the selection process on a statistical a contrario framework which allows us to have only one free parameter related to the desired scale.

We propose a linear-time line segment detector that gives accurate results, a controlled number of false detections, and requires no parameter tuning. This algorithm is tested and compared to state-of-the-art algorithms on a wide set of natural images.

Hierarchies are a powerful tool for image segmentation, they produce a multiscale representation which allows to design robust algorithms and can be stored in tree-like structures which provide an efficient implementation. These hierarchies are usually constructed explicitly or implicitly by means of region merging algorithms. These algorithms obtain the segmentation from the hierarchy by either using a greedy merging order or by cutting the hierarchy at a fixed scale. Our main contribution is to enlarge the search space of these algorithms to the set of all possible partitions spanned by a certain hierarchy, and to cast the segmentation as a selection problem within this space. The importance of this is two-fold. First, we are enlarging the search space of classic greedy algorithms and thus potentially improving the segmentation results. Second, this space is considerably smaller than the space of all possible partitions, thus we are reducing the complexity. In addition, we embed the selection process on a statistical a contrario framework which allows us to reduce the number of free parameters of our algorithm to only one.

Kernel methods have been widely studied in the field of pattern recognition. These methods implicitly map, "the kernel trick," the data into a space which is more appropriate for analysis. Many manifold learning and dimensionality reduction techniques are simply kernel methods for which the mapping is explicitly computed. In such cases, two problems related with the mapping arise: The out-of-sample extension and the pre-image computation. In this paper we propose a new pre-image method based on the Nystrom formulation for the out-of-sample extension, showing the connections between both problems. We also address the importance of normalization in the feature space, which has been ignored by standard pre-image algorithms. As an example, we apply these ideas to the Gaussian kernel, and relate our approach to other popular pre-image methods. Finally, we show the application of these techniques in the study of dynamic shapes.

A framework for the regularized estimation of nonuniform dimensionality and density in high dimensional data is introduced in this work. This leads to learning stratifications, that is, mixture of manifolds representing different characteristics and complexities in the data set. The basic idea relies on modeling the high dimensional sample points as a process of Poisson mixtures, with regularizing restrictions and spatial continuity constraints. Theoretical asymptotic results for the model are presented as well. The presentation of the framework is complemented with artificial and real examples showing the importance of regularized stratification learning in computer vision applications.

In this paper we propose a new method for detecting straight line segments in digital images. It improves upon existing methods by giving precise results while controlling the number of false detections and can be applied to any digital image without parameter setting. The method is a nontrivial extension of the approach presented by Desolneux et al. (2000). The core of the method is an algorithm to cut a binary sequences into what we call a multisegment: a set of collinear and disjoint segments. We shall define a functional that measures the so called meaningfulness of a multisegment. This functional allows us to validate detections against an a contrario non-structured model and to select the best ones. The result is a global interpretation, line by line, of the image in terms of straight segments which gives back its geometry with high accuracy. Comparisons with state of the art methods are presented (more examples are available on-line).

Automatic ultrasound (US) image segmentation is a difficult task due to the quantity of noise present in the images and the lack of information in several zones produced by the acquisition conditions. In this paper, we propose a method that combines shape priors and image information to achieve this task. In particular, we introduce knowledge about the rib-eye shape using a set of images manually segmented by experts. A method is proposed for the automatic segmentation of new samples in which a closed curve is fitted taking into account both the US image information and the geodesic distance between the evolving curve and the estimated mean rib-eye shape in a shape space. This method can be used to solve similar problems that arise when dealing with US images in other fields. The method was successfully tested over a database composed of 610 US images, for which we have the manual segmentations of two experts.

The tree of shapes is a powerful tool for image representation which holds many interesting properties. Many works in the literature use it for image segmentation, but most of them use only boundary information along the level lines. In many real images this is not enough to achieve a good segmentation, and region information must be introduced. In this work we present a novel region-based segmentation algorithm using the tree of shapes. The approach taken consists in the selection of relevant level-lines according to region based descriptors computed from their interior. We describe a region using the histogram of its features and we select interesting regions by identifying parts of the tree with an homogeneous histogram. The main contribution of this work is the joint use of histograms and suitable metrics between them, with the powerful representation of the tree of shapes. This allows us to handle complex region models and thus improves on previous works which were only able to deal with piecewise constant models. We validate our approach with real images and we obtain results which are favorably compared with some well known related approaches.

The study of point cloud data sampled from a stratification, a collection of manifolds with possible different dimensions, is pursued in this paper. We present a technique for simultaneously soft clustering and estimating the mixed dimensionality and density of such structures. The framework is based on a maximum likelihood estimation of a Poisson mixture model. The presentation of the approach is completed with artificial and real examples demonstrating the importance of extending manifold learning to stratification learning.

Some of the most successful algorithms for the automated segmentation of images use an active regions approach, where a curve is evolved so as to maximize the disparity of its interior and exterior. But these techniques require the manual selection of several parameters, which make impractical the work with long image sequences or with a very dissimilar set of sequences. Unfortunately this is precisely the case with 3D biological image sequences. In this work we improve on previous active regions algorithms in two aspects: by introducing a way to compute and update the optimum weights for the different channels involved (color, texture, etc.) and by estimating if the moving curve has lost any object so as to launch a re-initialization step. Our method is shown to outperform previous approaches. Several examples of biological image sequences, quite long and different among themselves, are presented.

A method is presented for segmentation of anatomical structures that incorporates prior information about shape. The method iteratively applies steps, which find object's border considering its properties independently from shape. The boundary is regularized taking in account the shape being extracted. Detection is not directly performed in the image but in a "shape space" referred to the shape in each step. The problem is reduced to work in this new coordinate system where the border is approximately a horizontal line. Shape information is introduced through a higher dimensional map similar to a distance map of a mean shape. Segmentation results are demonstrated on ultrasound imagery to measure meat quality of bovine and ovine livestock.

A method for deforming curves in a given image to a desired position in a second image is introduced. The algorithm is based on deforming the first image toward the second one via a partial differential equation (PDE), while tracking the deformation of the curves of interest in the first image with an additional, coupled PDE; both the images and the curves on the frame/slices of interest are used for tracking. The technique can be applied to object tracking and sequential segmentation. The topology of the deforming curve can change without any special topology handling procedures added to the scheme. This permits, for example, the automatic tracking of scenes where, due to occlusions, the topology of the objects of interest changes from frame to frame. In addition, this work introduces the concept of projecting velocities to obtain systems of coupled PDEs for image analysis applications. We show examples for object tracking and segmentation of electronic microscopy.

Since the work by Osher and Sethian on level-sets algorithms for numerical shape evolutions, this technique has been used for a large number of applications in numerous fields. In medical imaging, this numerical technique has been successfully used, for example, in segmentation and cortex unfolding algorithms. The migration from a Lagrangian implementation to a Eulerian one via implicit representations or level-sets brought some of the main advantages of the technique, i.e., topology independence and stability. This migration means also that the evolution is parametrization free. Therefore, we do not know exactly how each part of the shape is deforming and the point-wise correspondence is lost. In this note we present a technique to numerically track regions on surfaces that are being deformed using the level-sets method. The basic idea is to represent the region of interest as the intersection of two implicit surfaces and then track its deformation from the deformation of these surfaces. This technique then solves one of the main shortcomings of the very useful level-sets approach. Applications include lesion localization in medical images, region tracking in functional MRI (fMRI) visualization, and geometric surface mapping.

In this article, we describe some of the algorithms for depth and motion analysis which have been developed within ESPRIT project 940. Specifically we discuss edge detection, token tracking in sequences of images, and trinocular stereo. These processes have been implemented in hardware to form the core of the Depth and Motion Analysis (DMA) machine which has been developed to provide sophisticated real time vision capabilities for a large variety of robotics tasks.