Deep learning has shown impressive results, obtained at the cost of training huge neural networks.However, the larger the architecture, the higher the computational, financial, and environmental costs during training and inference.We aim at reducing both training and inference durations.We focus on Neural Architecture Growth, which can increase the size of a small model when needed, directly during training using information from the backpropagation.We expand existing work and freely grow neural networks in the form of any Directed Acyclic Graph.We design strategies that reduce excessive computations and steer network growth toward more parameter-efficient architectures.

MOTIVATION: We present dnadna, a flexible python-based software for deep learning inference in population genetics. It is task-agnostic and aims at facilitating the development, reproducibility, dissemination and re-usability of neural networks designed for population genetic data. RESULTS: dnadna defines multiple user-friendly workflows. First, users can implement new architectures and tasks, while benefiting from dnadna utility functions, training procedure and test environment, which saves time and decreases the likelihood of bugs. Second, the implemented networks can be re-optimized based on user-specified training sets and/or tasks. Newly implemented architectures and pre-trained networks are easily shareable with the community for further benchmarking or other applications. Finally, users can apply pre-trained networks in order to predict evolutionary history from alternative real or simulated genetic datasets, without requiring extensive knowledge in deep learning or coding in general. dnadna comes with a peer-reviewed, exchangeable neural network, allowing demographic inference from SNP data, that can be used directly or retrained to solve other tasks. Toy networks are also available to ease the exploration of the software, and we expect that the range of available architectures will keep expanding thanks to community contributions. AVAILABILITY AND IMPLEMENTATION: dnadna is a Python (≥3.7) package, its repository is available at gitlab.com/mlgenetics/dnadna and its associated documentation at mlgenetics.gitlab.io/dnadna/.

Applications of generative models for genomic data have gained significant momentum in the past few years, with scopes ranging from data characterization to generation of genomic segments and functional sequences. In our previous study, we demonstrated that generative adversarial networks (GANs) and restricted Boltzmann machines (RBMs) can be used to create novel high-quality artificial genomes (AGs) which can preserve the complex characteristics of real genomes such as population structure, linkage disequilibrium and selection signals. However, a major drawback of these models is scalability, since the large feature space of genome-wide data increases computational complexity vastly. To address this issue, we implemented a novel convolutional Wasserstein GAN (WGAN) model along with a novel conditional RBM (CRBM) framework for generating AGs with high SNP number. These networks implicitly learn the varying landscape of haplotypic structure in order to capture complex correlation patterns along the genome and generate a wide diversity of plausible haplotypes. We performed comparative analyses to assess both the quality of these generated haplotypes and the amount of possible privacy leakage from the training data. As the importance of genetic privacy becomes more prevalent, the need for effective privacy protection measures for genomic data increases. We used generative neural networks to create large artificial genome segments which possess many characteristics of real genomes without substantial privacy leakage from the training dataset. In the near future, with further improvements in haplotype quality and privacy preservation, large-scale artificial genome databases can be assembled to provide easily accessible surrogates of real databases, allowing researchers to conduct studies with diverse genomic data within a safe ethical framework in terms of donor privacy.

International audience

In machine learning the best performance on a certain task is achieved by fully supervised methods when perfect ground truth labels are available. However, labels are often noisy, especially in remote sensing where manually curated public datasets are rare. We study the multi-modal cadaster map alignment problem for which available annotations are misaligned polygons, resulting in noisy supervision. We subsequently set up a multiple-rounds training scheme which corrects the ground truth annotations at each round to better train the model at the next round. We show that it is possible to reduce the noise of the dataset by iteratively training a better alignment model to correct the annotation alignment.

In this work, we propose a novel multi-task framework, to learn satellite image pansharpening and segmentation jointly. Our framework is based on the encoder-decoder architecture, where both tasks share the same encoder but each one has its own decoder. We compare our framework against single-task models with different architectures. Results show that our framework outperforms all other approaches in both tasks.

Given a trained neural network, we aim at understanding how similar it considers any two samples. For this, we express a proper definition of similarity from the neural network perspective (i.e. we quantify how undissociable two inputs A and B are), by taking a machine learning viewpoint: how much a parameter variation designed to change the output for A would impact the output for B as well? We study the mathematical properties of this similarity measure, and show how to estimate sample density with it, in low complexity, enabling new types of statistical analysis for neural networks. We also propose to use it during training, to enforce that examples known to be similar should also be seen as similar by the network. We then study the self-denoising phenomenon encountered in regression tasks when training neural networks on datasets with noisy labels. We exhibit a multimodal image registration task where almost perfect accuracy is reached, far beyond label noise variance. Such an impressive self-denoising phenomenon can be explained as a noise averaging effect over the labels of similar examples. We analyze data by retrieving samples perceived as similar by the network, and are able to quantify the denoising effect without requiring true labels.

The ultimate goal of land mapping from remote sensing image classification is to produce polygonal representations of Earth's objects, to be included in geographic information systems. This is most commonly performed by running a pixelwise image classifier and then polygonizing the connected components in the classification map. We here propose a novel polygonization algorithm, which uses a labeled triangular mesh to approximate the input classification maps. The mesh is optimized in terms of an l1norm with respect to the classifiers's output. We use a rich set of optimization operators, which includes a vertex relocator, and add a topology preservation strategy. The method outperforms current approaches, yielding better accuracy with fewer vertices.

New challenges in remote sensing impose the necessity of designing pixel classification methods that, once trained on a certain dataset, generalize to other areas of the earth. This may include regions where the appearance of the same type of objects is significantly different. In the literature it is common to use a single image and split it into training and test sets to train a classifier and assess its performance, respectively. However, this does not prove the generalization capabilities to other inputs. In this paper, we propose an aerial image labeling dataset that covers a wide range of urban settlement appearances, from different geographic locations. Moreover, the cities included in the test set are different from those of the training set. We also experiment with convolutional neural networks on our dataset.

We address the pixelwise classification of high-resolution aerial imagery. While convolutional neural networks (CNNs) are gaining increasing attention in image analysis, it is still challenging to adapt them to produce fine-grained classification maps. This is due to a well-known trade-off between recognition and localization: the impressive capability of CNNs to recognize meaningful objects comes at the price of losing spatial precision. We here propose an architecture that addresses this issue. It learns features at different levels of detail and also learns a function to combine them. By integrating local and global information in an efficient and flexible manner, it outperforms previous techniques.

While initially devised for image categorization, convolutional neural networks (CNNs) are being increasingly used for the pixelwise semantic labeling of images. However, the proper nature of the most common CNN architectures makes them good at recognizing but poor at localizing objects precisely. This problem is magnified in the context of aerial and satellite image labeling, where a spatially fine object outlining is of paramount importance. Different iterative enhancement algorithms have been presented in the literature to progressively improve the coarse CNN outputs, seeking to sharpen object boundaries around real image edges. However, one must carefully design, choose, and tune such algorithms. Instead, our goal is to directly learn the iterative process itself. For this, we formulate a generic iterative enhancement process inspired from partial differential equations, and observe that it can be expressed as a recurrent neural network (RNN). Consequently, we train such a network from manually labeled data for our enhancement task. In a series of experiments, we show that our RNN effectively learns an iterative process that significantly improves the quality of satellite image classification maps.

We propose an end-to-end framework for the dense, pixelwise classification of satellite imagery with convolutional neural networks (CNNs). In our framework, CNNs are directly trained to produce classification maps out of the input images. We first devise a fully convolutional architecture and demonstrate its relevance to the dense classification problem. We then address the issue of imperfect training data through a two-step training approach: CNNs are first initialized by using a large amount of possibly inaccurate reference data, and then refined on a small amount of accurately labeled data. To complete our framework, we design a multiscale neuron module that alleviates the common tradeoff between recognition and precise localization. A series of experiments show that our networks consider a large amount of context to provide fine-grained classification maps.

The problem of dense semantic labeling consists in assigning semantic labels to every pixel in an image. In the context of aerial image analysis, it is particularly important to yield high-resolution outputs. In order to use convolutional neural networks (CNNs) for this task, it is required to design new specific architectures to provide fine-grained classification maps. Many dense semantic labeling CNNs have been recently proposed. Our first contribution is an in-depth analysis of these architectures. We establish the desired properties of an ideal semantic labeling CNN, and assess how those methods stand with regard to these properties. We observe that even though they provide competitive results, these CNNs often underexploit properties of semantic labeling that could lead to more effective and efficient architectures. Out of these observations, we then derive a CNN framework specifically adapted to the semantic labeling problem. In addition to learning features at different resolutions, it learns how to combine these features. By integrating local and global information in an efficient and flexible manner, it outperforms previous techniques. We evaluate the proposed framework and compare it with state-of-the-art architectures on public benchmarks of high-resolution aerial image labeling.

We propose a convolutional neural network (CNN) model for remote sensing image classification. Using CNNs provides us with a means of learning contextual features for large-scale image labeling. Our network consists of four stacked convolutional layers that downsample the image and extract relevant features. On top of these, a deconvolutional layer upsamples the data back to the initial resolution, producing a final dense image labeling. Contrary to previous frameworks, our network contains only convolution and deconvolution operations. Experiments on aerial images show that our network produces more accurate classifications in lower computational time.

A partition tree is a hierarchical representation of an image. Once constructed, it can be repeatedly processed to extract information. Multi-object multi-class image segmentation with shape priors is one of the tasks that can be efficiently done upon an available tree. The traditional construction approach is a greedy clustering based on color similarities. However, not considering higher level cues during the construction phase leads to trees that might not accurately represent the underlying objects in the scene, inducing mistakes in the later segmentation. We propose a method to optimize a tree based both on color distributions and shape priors. It consists in pruning and regrafting tree branches in order to minimize the energy of the best segmentation that can be extracted from the tree. Theoretical guarantees help reducing the search space and make the optimization efficient. Our experiments show that we succeed in incorporating shape information to restructure a tree, which in turn enables to extract from it good quality multi-object segmentations with shape priors.

We propose a new binary partition tree (BPT)-based framework for multi-class segmentation of remote sensing images. In the literature, BPTs are typically computed in a bottom-up manner based on spectral similarities, then analyzed to extract image objects. When image objects exhibit a considerable internal spectral variability, it often happens that such objects are composed of several disjoint regions in the BPT, yielding errors in object extraction. We pose the multi-class segmentation problem as an energy minimization task and solve it by using BPTs. Our main contribution consists in introducing a new dissimilarity function for the tree construction, which combines both spectral discrepancies and supervised class-specific information to take into account the within-class spectral variability. The experimental validation proved that the proposed method constitutes a competitive alternative for object-based image classification.

We propose a new representation of videos, as spatio-temporal fibers. These fibers are clusters of trajectories that are meshed spatially in the image domain. They form a hierarchical partition of the video into regions that are coherent in time and space. They can be seen as dense, spatially-organized, long-term optical flow. Their robustness to noise and ambiguities is ensured by taking into account the reliability of each source of information. As fibers allow users to handle easily moving objects in videos, they prove useful for video editing, as demonstrated in a video inpainting example.

We propose a new method for joint segmentation of monotonously growing or shrinking shapes in a time sequence of noisy images. The task of segmenting the image time series is expressed as an optimization problem using the spatio-temporal graph of pixels, in which we are able to impose the constraint of shape growth or of shrinkage by introducing monodirectional infinite links connecting pixels at the same spatial locations in successive image frames. The globally optimal solution is computed with a graph cut. The performance of the proposed method is validated on three applications: segmentation of melting sea ice floes and of growing burned areas from time series of 2D satellite images, and segmentation of a growing brain tumor from sequences of 3D medical scans. In the latter application, we impose an additional intersequences inclusion constraint by adding directed infinite links between pixels of dependent image structures.

We propose a new method based on graph cuts for joint segmentation of monotonously growing or shrinking shapes in time series of noisy images. By introducing directed infinite links connecting pixels at the same spatial locations in successive image frames, we impose shape growth/shrinkage constraint in graph cuts. Minimization of energy computed on the resulting graph of the image sequence yields globally optimal segmentation. We validate the proposed approach on two applications: segmentation of melting sea ice floes from a time series of multimodal satellite images and segmentation of a growing brain tumor from sequences of 3D multimodal medical scans. In the latter application, we impose an additional inter-sequences inclusion constraint by adding directed infinite links between pixels of dependent image structures.

We propose a new method based on graph cuts for the segmentation of burned areas in time series of satellite images. The method consists in rewriting a segmentation problem as a (s, t)-min-cut on the spatio-temporal image graph and computing this minimal cut. As burned areas grow in time, we introduce growth constraint in graph cuts by using directed infinite links connecting pixels at the same spatial locations in successive image frames. This method guarantees to find the globally optimal segmentation satisfying the growth constraint in small time complexity. Experimental results on a set of MODIS measurements over the Northern Australia demonstrated that the new approach succeeded in combining both spatial and temporal information for accurate segmentation of burned areas.

Graph cuts are widely used in many fields of computer vision in order to minimize in small polynomial time complexity certain classes of energies. These specific classes depend on the way chosen to build the graphs representing the problems to solve. We study here all possible ways of building graphs and the associated energies minimized, leading to the exhaustive family of energies minimizable exactly by a graph cut. To do this, we consider the issue of coding pixel labels as states of the graph, i.e. the choice of state interpretations. The family obtained comprises many new classes, in particular energies that do not satisfy the submodularity condition, including energies that are even not permuted-submodular. A generating subfamily is studied in details, in particular we propose a canonical form to represent Markov random fields, which proves useful to recognize energies in this subfamily in linear complexity almost surely, and then to build the associated graph in quasilinear time. A few experiments are performed, to illustrate the new possibilities offered.

In this paper, we present a faster version of the newly proposed Multiple Birth and Cut (MBC) algorithm. MBC is an optimization method applied to the energy minimization of an object based model, defined by a marked point process. We show that, by proposing good candidates in the birth step of this algorithm, the speed of convergence is increased. The algorithm starts by generating a dense configuration in a special organization, the best candidates are selected using the belief propagation algorithm. Next, this candidate configuration is combined with the current configuration using binary graph cuts as presented in the original version of the MBC algorithm. We tested the performance of our algorithm on the particular problem of counting flamingos in a colony, and show that it is much faster with the modified birth step.

We propose a new approach to compute non-linear, intrinsic shape statistics and to incorporate them into a shape prior for an image segmentation task. Given a sample set of contours, we first define their mean shape as the one which is simultaneously closest to all samples up to rigid motions, and compute it in a gradient descent framework. We consider here a differentiable approximation of the Hausdorff distance between shapes. Statistics on the instantaneous deformation fields that the mean shape should undergo to move towards each sample lead to sensible characteristic modes of deformation that convey the shape variability. Contour statistics are turned into a shape prior which is rigid-motion invariant. Image segmentation results show the improvement gained by the shape prior.

This article deals with statistics on sets of shapes. The approach is based on the Hausdorff distance between shapes, The choice of the Hausdorff distance between shapes is itself not fundamental since the same framework could be applied with another distance. We first define a smooth approximation of the Hausdorff distance and build non-supervised warpings between shapes by a gradient descent of the approximation. Local minima can be avoided by changing the scalar product in the tangent space of the shape being warped. When non-supervised warping fails, we present a way to guide the evolution with a small number of landmarks. Thanks to the warping fields, we can define the mean of a set of shapes and express statistics on them. Finally, we come back to the initial distance between shapes and use it to represent a set of shapes by a graph, which with the technique of graph Laplacian leads to a way of projecting shapes onto a low dimensional space

We propose a new approach to deal with the first and second order statistics of a set of images. These statistics take into account the images characteristic deformations and their variations in intensity. The central algorithm is based on nonsupervised diffeomorphic image matching (without landmarks or human intervention). As they convey the notion of the mean shape and colors of an object and the one of its common variations, such statistics of sets of images may be relevant in the context of object recognition, both in the segmentation of any of its representations and in the classification of them. The proposed approach has been tested on a small database of face images to compute a mean face and second order statistics. The results are very encouraging since, whereas the algorithm does not need any human intervention and is not specific to face image databases, the mean image looks like a real face and the characteristic modes of variation (deformation and intensity changes) are sensible.

This paper tackles an important aspect of the variational problems involving active contours, which has been largely overlooked so far: the optimization by gradient flows. Classically, the definition of a gradient depends directly on the choice of an inner product structure. This consideration is largely absent from the active contours literature. Most authors, overtly or covertly, assume that the space of admissible deformations is ruled by the canonical L2inner product. The classical gradient flows reported in the literature are relative to this particular choice. In this paper, we investigate the relevance of using other inner products, yielding other gradient descents, and some other minimizing flows not deriving from any inner product. In particular we, show how to induce different degrees of spatial coherence into the minimizing flow, in order to decrease the probability of getting trapped into irrelevant local minima. We show with some numerical experiments that the sensitivity of the active contours method to initial conditions, which seriously limits its applicability and its efficiency, is alleviated by our application-specific spatially coherent minimizing flows

Approximations of shape metrics, such as the Hausdorff distance, to define similarity measures between shapes are proposed. Our approximations being continuous and differentiable, they provide an obvious way to warp a shape onto another by solving a partial differential equation (PDE), in effect a curve flow, obtained from their first order variation. This first order variation defines a normal deformation field for a given curve. We use the normal deformation fields induced by several sample shape examples to define their mean, their covariance "operator", and the principal modes of variation. Our theory, which can be seen as a nonlinear generalization of the linear approaches proposed by several authors, is illustrated with numerous examples. Our approach being based upon the use of distance functions is characterized by the fact that it is intrinsic, i.e. independent of the shape parametrization.