Deep neural networks (DNNs) trained for image denoising are able to generate high-quality samples with score-based reverse diffusion algorithms. These impressive capabilities seem to imply an escape from the curse of dimensionality, but recent reports of memorization of the training set raise the question of whether these networks are learning the "true" continuous density of the data. Here, we show that two DNNs trained on non-overlapping subsets of a dataset learn nearly the same score function, and thus the same density, when the number of training images is large enough. In this regime of strong generalization, diffusion-generated images are distinct from the training set, and are of high visual quality, suggesting that the inductive biases of the DNNs are well-aligned with the data density. We analyze the learned denoising functions and show that the inductive biases give rise to a shrinkage operation in a basis adapted to the underlying image. Examination of these bases reveals oscillating harmonic structures along contours and in homogeneous regions. We demonstrate that trained denoisers are inductively biased towards these geometry-adaptive harmonic bases since they arise not only when the network is trained on photographic images, but also when it is trained on image classes supported on low-dimensional manifolds for which the harmonic basis is suboptimal. Finally, we show that when trained on regular image classes for which the optimal basis is known to be geometry-adaptive and harmonic, the denoising performance of the networks is near-optimal.

A central question in deep learning is to understand the functions learned by deep networks. What is their approximation class? Do the learned weights and representations depend on initialization? Previous empirical work has evidenced that kernels defined by network activations are similar across initializations. For shallow networks, this has been theoretically studied with random feature models, but an extension to deep networks has remained elusive. Here, we provide a deep extension of such random feature models, which we call the rainbow model. We prove that rainbow networks define deterministic (hierarchical) kernels in the infinite-width limit. The resulting functions thus belong to a data-dependent RKHS which does not depend on the weight randomness. We also verify numerically our modeling assumptions on deep CNNs trained on image classification tasks, and show that the trained networks approximately satisfy the rainbow hypothesis. In particular, rainbow networks sampled from the corresponding random feature model achieve similar performance as the trained networks. Our results highlight the central role played by the covariances of network weights at each layer, which are observed to be low-rank as a result of feature learning.

There is a growing gap between the impressive results of deep image generative models and classical algorithms that offer theoretical guarantees. The former suffer from mode collapse or memorization issues, limiting their application to scientific data. The latter require restrictive assumptions such as log-concavity to escape the curse of dimensionality. We partially bridge this gap by introducing conditionally strongly log-concave (CSLC) models, which factorize the data distribution into a product of conditional probability distributions that are strongly log-concave. This factorization is obtained with orthogonal projectors adapted to the data distribution. It leads to efficient parameter estimation and sampling algorithms, with theoretical guarantees, although the data distribution is not globally log-concave. We show that several challenging multiscale processes are conditionally log-concave using wavelet packet orthogonal projectors. Numerical results are shown for physical fields such as the $\varphi^4$ model and weak lensing convergence maps with higher resolution than in previous works.

Score-based generative models (SGMs) synthesize new data samples from Gaussian white noise by running a time-reversed Stochastic Differential Equation (SDE) whose drift coefficient depends on some probabilistic score. The discretization of such SDEs typically requires a large number of time steps and hence a high computational cost. This is because of ill-conditioning properties of the score that we analyze mathematically. Previous approaches have relied on multiscale generation to considerably accelerate SGMs. We explain how this acceleration results from an implicit factorization of the data distribution into a product of conditional probabilities of wavelet coefficients across scales. The resulting Wavelet Score-based Generative Model (WSGM) synthesizes wavelet coefficients with the same number of time steps at all scales, and its time complexity therefore grows linearly with the image size. This is proved mathematically for Gaussian distributions, and shown numerically for physical processes at phase transition and natural image datasets.

The wavelet scattering transform is an invariant and stable signal representation suitable for many signal processing and machine learning applications. We present the Kymatio software package, an easy-to-use, high-performance Python implementation of the scattering transform in 1D, 2D, and 3D that is compatible with modern deep learning frameworks, including PyTorch and TensorFlow/Keras. The transforms are implemented on both CPUs and GPUs, the latter offering a significant speedup over the former. The package also has a small memory footprint. Source code, documentation, and examples are available under a BSD license at https://www.kymat.io.

Modeling time series with complex statistical properties such as heavy-tails, long-range dependence, and temporal asymmetries remains an open problem. In particular, financial time series exhibit such properties. Existing models suffer from serious limitations and often rely on high-order moments. We introduce a wavelet-based maximum entropy model for such random processes, based on new scattering and phase-harmonic moments. We analyze the model's performance with a synthetic multifractal random process and real-world financial time series. We show that scattering moments capture heavy tails and multifractal properties without estimating high-order moments. Further, we show that additional phase-harmonic terms capture temporal asymmetries.

Motivated by the prediction of cell loads in cellular networks, we formulate the following new, fundamental problem of statistical learning of geometric marks of point processes: An unknown marking function, depending on the geometry of point patterns, produces characteristics (marks) of the points. One aims at learning this function from the examples of marked point patterns in order to predict the marks of new point patterns. To approximate (interpolate) the marking function, in our baseline approach, we build a statistical regression model of the marks with respect to some local point distance representation. In a more advanced approach, we use a global data representation via the scattering moments of random measures, which build informative and stable to deformations data representation, already proven useful in image analysis and related application domains. In this case, the regression of the scattering moments of the marked point patterns with respect to the non-marked ones is combined with the numerical solution of the inverse problem, where the marks are recovered from the estimated scattering moments. Considering some simple, generic marks, often appearing in the modeling of wireless networks, such as the shot-noise values, nearest neighbour distance, and some characteristics of the Voronoi cells, we show that the scattering moments can capture similar geometry information as the baseline approach, and can reach even better performance, especially for non-local marking functions. Our results motivate further development of statistical learning tools for stochastic geometry and analysis of wireless networks, in particular to predict cell loads in cellular networks from the locations of base stations and traffic demand.

We introduce a solid harmonic wavelet scattering representation, invariant to rigid motion and stable to deformations, for regression and classification of 2D and 3D signals. Solid harmonic wavelets are computed by multiplying solid harmonic functions with Gaussian windows dilated at different scales. Invariant scattering coefficients are obtained by cascading such wavelet transforms with the complex modulus nonlinearity. We study an application of solid harmonic scattering invariants to the estimation of quantum molecular energies, which are also invariant to rigid motion and stable with respect to deformations. A multilinear regression over scattering invariants provides close to state of the art results over small and large databases of organic molecules.

Dictionary learning algorithms or supervised deep convolution networks have considerably improved the efficiency of predefined feature representations such as SIFT. We introduce a deep scattering convolution network, with complex wavelet filters over spatial and angular variables. This representation brings an important improvement to results previously obtained with predefined features over object image databases such as Caltech and CIFAR. The resulting accuracy is comparable to results obtained with unsupervised deep learning and dictionary based representations. This shows that refining image representations by using geometric priors is a promising direction to improve image classification and its understanding.

An affine invariant representation is constructed with a cascade of invariants, which preserves information for classification. A joint translation and rotation invariant representation of image patches is calculated with a scattering transform. It is implemented with a deep convolution network, which computes successive wavelet transforms and modulus non-linearities. Invariants to scaling, shearing and small deformations are calculated with linear operators in the scattering domain. State-of-the-art classification results are obtained over texture databases with uncontrolled viewing conditions.

Environmental sounds are an interesting subject of study for machine audition because of their wide variety of acoustical characteristics and their central presence in our everyday life. They are perceived effortlessly in the human auditory system whereas state-of-the-art computational systems are far from reaching the same efficiency. In this paper we propose a novel representation of such sounds based on the scattering transform which has the property of stability to time-warping deformations and invariance to time-shift useful for classifications tasks. This representation is compared to several state-of-the-art approaches for the task of quantifying similarity between environmental sounds.

A wavelet scattering network computes a translation invariant image representation which is stable to deformations and preserves high-frequency information for classification. It cascades wavelet transform convolutions with nonlinear modulus and averaging operators. The first network layer outputs SIFT-type descriptors, whereas the next layers provide complementary invariant information that improves classification. The mathematical analysis of wavelet scattering networks explains important properties of deep convolution networks for classification. A scattering representation of stationary processes incorporates higher order moments and can thus discriminate textures having the same Fourier power spectrum. State-of-the-art classification results are obtained for handwritten digits and texture discrimination, with a Gaussian kernel SVM and a generative PCA classifier.

A general framework for solving image inverse problems with piecewise linear estimations is introduced in this paper. The approach is based on Gaussian mixture models, which are estimated via a maximum a posteriori expectation-maximization algorithm. A dual mathematical interpretation of the proposed framework with a structured sparse estimation is described, which shows that the resulting piecewise linear estimate stabilizes the estimation when compared with traditional sparse inverse problem techniques. We demonstrate that, in a number of image inverse problems, including interpolation, zooming, and deblurring of narrow kernels, the same simple and computationally efficient algorithm yields results in the same ballpark as that of the state of the art.

A scattering vector is a local descriptor including multiscale and multi-direction co-occurrence information. It is computed with a cascade of wavelet decompositions and complex modulus. This scattering representation is locally translation invariant and linearizes deformations. A supervised classification algorithm is computed with a PCA model selection on scattering vectors. State of the art results are obtained for handwritten digit recognition and texture classification.

An image representation framework based on structured sparse model selection is introduced in this work. The corresponding modeling dictionary is comprised of a family of learned orthogonal bases. For an image patch, a model is first selected from this dictionary through linear approximation in a best basis, and the signal estimation is then calculated with the selected model. The model selection leads to a guaranteed near optimal denoising estimator. The degree of freedom in the model selection is equal to the number of the bases, typically about 10 for natural images, and is significantly lower than with traditional overcomplete dictionary approaches, stabilizing the representation. For an image patch of size √N × √N, the computational complexity of the proposed framework is O (N2), typically 2 to 3 orders of magnitude faster than estimation in an overcomplete dictionary. The orthogonal bases are adapted to the image of interest and are computed with a simple and fast procedure. State-of-the-art results are shown in image denoising, deblurring, and inpainting.

We introduce a class of inverse problem estimators computed by mixing adaptively a family of linear estimators corresponding to different priors. Sparse mixing weights are calculated over blocks of coefficients in a frame providing a sparse signal representation. They minimize an l1 norm taking into account the signal regularity in each block. Adaptive directional image interpolations are computed over a wavelet frame with an O(N logN) algorithm, providing state-of-the-art numerical results.

Super-resolution image zooming is possible when the image has some geometric regularity. We introduce a general class of non-linear inverse estimators, which combines linear estimators with mixing weights in a frame providing a sparse representation. Mixing weights are computed with a block decomposition, which minimizes a Tikhonov energy penalized by an 11norm of the mixing weights. A fast orthogonal matching pursuit algorithm computes the mixing weights. Adaptive directional image interpolations are calculated with mixing weights in a wavelet frame.

We investigate a new audio denoising algorithm. Complex wavelets protect phase of signals and are thus preferred in audio signal processing to real wavelets. The block attenuation eliminates the residual noise artifacts in reconstructed signals and provides a good approximation of the attenuation with oracle. A connection between the block attenuation and the decision-directed a priori SNR estimator of Ephraim and Malah is studied. Finally we introduce an adaptive block technique based on the dyadic CART algorithm. The experiments show that not only the proposed method does eliminate the residual noise artifacts, but it also preserves transients of signals better than short-time Fourier based methods do.

This paper describes the construction of second generation bandelet orthogonal bases. The decomposition on a bandelet basis is computed using a wavelet filter bank followed by adaptive geometric orthogonal filters, that require O(N) operations. The resulting geometry is multiscale and calculated with a fast procedure that minimizes a Lagrangian cost at each scale. Image compression with the resulting bandelet transform code gives significantly better results than a wavelet transform code.

This paper introduces a new class of bases, called bandelet bases, which decompose the image along multiscale vectors that are elongated in the direction of a geometric flow. This geometric flow indicates directions in which the image gray levels have regular variations. The image decomposition in a bandelet basis is implemented with a fast subband-filtering algorithm. Bandelet bases lead to optimal approximation rates for geometrically regular images. For image compression and noise removal applications, the geometric flow is optimized with fast algorithms so that the resulting bandelet basis produces minimum distortion. Comparisons are made with wavelet image compression and noise-removal algorithms.

This paper describes the construction of second generation bandelet bases and their application to 3D geometry compression. This new coding scheme is orthogonal and the corresponding basis functions are regular. In our method, surfaces are decomposed in a bandelet basis with a fast bandeletization algorithm that removes the geometric redundancy of orthogonal wavelet coefficients. The resulting transform coding scheme has an error decay that is asymptotically optimal for geometrically regular surfaces. We then use these bandelet bases to perform geometry image and normal map compression. Numerical tests show that for complex surfaces bandelets bring an improvement of 1.5dB to 2dB over state of the art compression schemes.

The deconvolution of signals is studied with thresholding estimators that decompose signals in an orthonormal basis and threshold the resulting coefficients. A general criterion is established to choose the orthonormal basis in order to minimize the estimation risk. Wavelet bases are highly sub-optimal to restore signals and images blurred by a low-pass filter whose transfer function vanishes at high frequencies. A new orthonormal basis called mirror wavelet basis is constructed to minimize the risk for such deconvolutions. An application to the restoration of satellite images is shown.

Studies the recovery of shape from texture under perspective projection. We regard shape from texture as a statistical estimation problem, the texture being the realization of a stochastic process. We introduce warplets, which generalize wavelets over the 2D affine group. At fine scales, the warpogram of the image obeys a transport equation, called texture gradient equation. In order to recover the 3D shape of the surface, one must estimate the deformation gradient, which measures metric changes in the image. This is made possible by imposing a notion of homogeneity for the original texture, according to which the deformation gradient is equal to the velocity of the texture gradient equation. By measuring the warplet transform of the image at different scales, we obtain a deformation gradient estimator.

Summary form only given, as follows. To improve image representations, it is necessary to take advantage of the geometrical regularity of singularities along edges. Bandelets are orthogonal families, that can be adapted to capture singularities that evolve regularly along smooth geometrical contours, with few non-zero coefficients. They are constructed from one-dimensional foveal wavelets, that are orthogonal one-dimensional functions that approximate signals with a strategy similar to that of the retina. Images are partly represented with bandelet coefficients along edges, plus a residual which is decomposed in a regular two-dimensional wavelet basis. The edge curves are chosen to minimize the error for a given number of nonzero bandelet and wavelet coefficients. They are represented in a one-dimensional wavelet basis. An application to image compression has been compared with JPEG2000.

We introduce a sparse image representation that takes advantage of the geometrical regularity of edges in images. A new class of one-dimensional wavelet orthonormal bases, called foveal wavelets, are introduced to detect and reconstruct singularities. Foveal wavelets are extended in two dimensions, to follow the geometry of arbitrary curves. The resulting two dimensional "bandelets" define orthonormal families that can restore close approximations of regular edges with few non-zero coefficients. A double layer image coding algorithm is described. Edges are coded with quantized bandelet coefficients, and a smooth residual image is coded in a standard two-dimensional wavelet basis.

This paper is a contribution to the recovery of shape from texture under perspective projection. We regard shape from texture as a statistical estimation problem, the texture being the realization of a stochastic process. There are two minimal conditions in order for the problem to be solvable: the first is a stationarity condition on the texture with respect to the surface; the second is the regularity of the surface. Information about the surface is obtained by estimating a deformation map. We prove that at a fine scale, the wavelet decomposition of the image obeys a transport PDE, the coefficients of which can be estimated, and related to the deformation map. We show how the global surface shape can then be integrated.

We propose a best basis algorithm for signal enhancement in white Gaussian noise. The best basis search is performed in families of orthonormal bases constructed with wavelet packets or local cosine bases. We base our search for the "best" basis on a criterion of minimal reconstruction error of the underlying signal. This approach is intuitively appealing, because the enhanced or estimated signal has an associated measure of performance, namely, the resulting mean-square error. Previous approaches in this framework have focused on obtaining the most "compact" signal representations, which consequently contribute to effective denoising. These approaches, however, do not possess the inherent measure of performance which our algorithm provides. We first propose an estimator of the mean-square error, based on a heuristic argument and subsequently compare the reconstruction performance based upon it to that based on the Stein (1981) unbiased risk estimator. We compare the two proposed estimators by providing both qualitative and quantitative analyses of the bias term. Having two estimators of the mean-square error, we incorporate these cost functions into the search for the "best" basis, and subsequently provide a substantiating example to demonstrate their performance.

Deconvolution in presence of additive noise is an inverse problem that often occurs in image processing. We introduce a restoration algorithm which is regularized with a thresholding technique, in an optimally designed mirror wavelet basis. We prove the asymptotic optimality and the superiority of this procedure over linear methods in the set of signals with bounded variations. Besides, this restoration procedure is fast, provides excellent metric and perceptual results and has been chosen as the best method by satellite images photointerpreters from the French space agency (CNES), among several different competing algorithms.

This paper deals with low bit rate transform coding of images. We point out why the hypotheses used for the characterization of transform coders at high bit rate does not hold at low bit rate, and derive a new analytical formula for the distortion rate function. This formula is tested with JPEG, wavelet and embedded coders.

Early on, computer vision researchers have realized that multiscale transforms are important to analyze the information content of images. The wavelet theory gives a stable mathematical foundation to understand the properties of such multiscale algorithms. This tutorial describes major applications to multiresolution search, multiscale edge detection, and texture discrimination.

The joint detection of all users in a multiple access (MA) communication system in which user transmissions are correlated has been shown in recent literature to enhance the system performance relative to that achieved without joint detection. Over the past several years the area of low complexity joint detectors has received much attention. This paper explains the problem of multiple access joint detection in geometrical terms. Geometric interpretation leads to the proposal of an alternating projection joint detection algorithm (APJD). Due to some similarities between our APJD and the multistage joint detector (MJD) of Varansi and Aazhang (1990), the MJD is also discussed. The APJD is guaranteed to converge and a proof is given. The geometric interpretation of the MA joint detection problem allows for the exploration of determining, a priori, the error probability of a joint detector and user waveform set in the absence of noise. Simulations offer empirical characterization of the error behavior of both detectors.

We propose a best basis algorithm for signal enhancement in white Gaussian noise. We base our search of best basis on a criterion of minimal reconstruction error of the underlying signal. We subsequently compare our simple error criterion to the Stein (1981) unbiased risk estimator, and provide a substantiating example to demonstrate its performance. A review is also given of noise removal by thresholding and of wavepacket orthonormal bases.

A crucial problem in image analysis is to construct efficient low-level representations of an image, providing precise characterization of features which compose it, such as edges and texture components. An image usually contains very different types of features, which have been successfully modelled by the very redundant family of 2D Gabor oriented wavelets, describing the local properties of the image: localization, scale, preferred orientation, amplitude and phase of the discontinuity. However, this model generates representations of very large size. Instead of decomposing a given image over this whole set of Gabor functions, we use an adaptive algorithm (called matching pursuit) to select the Gabor elements which approximate at best the image, corresponding to the main features of the image. This produces compact representation in terms of few features that reveal the local image properties. Results proved that the elements are precisely localized on the edges of the images, and give a local decomposition as linear combinations of "textons" in the textured regions. We introduce a fast algorithm to compute the matching pursuit decomposition.

In this work, a novel method of multiscale geometric feature extraction and object recognition is developed. In particular, the new representation should have the following characteristics. First, the coarse scale features should have a geometric interpretation so that the overall geometry of the object is discernible from just these features. Second, the presence of fine scale detail should not change the coarse scale representation. These two goals are not achieved by current techniques which are based on error as measured by the L/sup 2/ norm. Two methods to accomplish these goals are presented. In the first, morphological filtering and wavelet networks are used. In the second, the correlation criteria of the matching pursuit algorithm of Mallat and Zhang (1993) is modified to obtain a variable, high resolution matching pursuit.

One can detect and characterize the singularities of a signal from the evolution of the wavelet transform coefficients across scales. The authors discriminate signal information from noise by using some prior knowledge of the properties of singularities. The wavelet transform of the signal is processed in order to remove the singularities created by the noise. The authors restore a sharp signal where part of the noise has been suppressed. Examples in one and two dimensions are shown.>

A multiscale Canny edge detection is equivalent to finding the local maxima of a wavelet transform. The authors study the properties of multiscale edges through the wavelet theory. For pattern recognition, one often needs to discriminate different types of edges. They show that the evolution of wavelet local maxima across scales characterize the local shape of irregular structures. Numerical descriptors of edge types are derived. The completeness of a multiscale edge representation is also studied. The authors describe an algorithm that reconstructs a close approximation of 1-D and 2-D signals from their multiscale edges. For images, the reconstruction errors are below visual sensitivity. As an application, a compact image coding algorithm that selects important edges and compresses the image data by factors over 30 has been implemented.>

The mathematical characterization of singularities with Lipschitz exponents is reviewed. Theorems that estimate local Lipschitz exponents of functions from the evolution across scales of their wavelet transform are reviewed. It is then proven that the local maxima of the wavelet transform modulus detect the locations of irregular structures and provide numerical procedures to compute their Lipschitz exponents. The wavelet transform of singularities with fast oscillations has a particular behavior that is studied separately. The local frequency of such oscillations is measured from the wavelet transform modulus maxima. It has been shown numerically that one- and two-dimensional signals can be reconstructed, with a good approximation, from the local maxima of their wavelet transform modulus. As an application, an algorithm is developed that removes white noises from signals by analyzing the evolution of the wavelet transform maxima across scales. In two dimensions, the wavelet transform maxima indicate the location of edges in images.>

A second generation image coding algorithm based on multiscale edges is described. Edges are detected from the local maxima of the image wavelet transform. The original image can be reconstructed from the corresponding multiscale edge map. For coding purposes, the edges that are important for the image visualization are selected, and an efficient encoding of the corresponding data is described. High compression ratio can be obtained for images where the texture information is not important. Examples with compression ratios of around 30 are given.>

The completeness, stability, and application to pattern recognition of a multiscale representation based on zero-crossings is discussed. An alternative projection algorithm is described that reconstructs a signal from a zero-crossing representation, which is stabilized by keeping the value of the wavelet transform integral between each pair of consecutive zero-crossings. The reconstruction algorithm has a fast convergence and each iteration requires O(N log/sup 2/ (N)) computation for a signal of N samples. The zero-crossings of a wavelet transform define a representation which is particularly well adapted for solving pattern recognition problems. As an example, the implementation and results of a coarse-to-fine stereo-matching algorithm are described.>

The edges of an image can be detected at different scales from the local maxima of its wavelet transform. an algorithm is described that reconstructs images from their edges at dyadic scales. The wavelet maxima representation is a novel reorganization of the image information that makes it possible to develop algorithms uniquely based on edges for solving image processing and computer vision problems. The evolution of the wavelet maxima across scales gives a precise characterization of the edge type which can be used for pattern recognition. A coding algorithm is described that selects the most important image edges in order to obtain a compact representation.>

An algorithm that reconstructs one-dimensional signals and images from their sharper variation points at dyadic scales is described. This algorithm exactly reconstructs images from their multiscale edges. It is proved that the evolution across scales of the wavelet maxima characterizes the local shape of the sharp variations of the signal. One can thus not only detect edges but also classify them. The wavelet maxima representation is a new reorganization of the image information that makes it possible to develop algorithms uniquely based on edges for solving image processing problems.>

Multiresolution representations are effective for analyzing the information content of images. The properties of the operator which approximates a signal at a given resolution were studied. It is shown that the difference of information between the approximation of a signal at the resolutions 2/sup j+1/ and 2/sup j/ (where j is an integer) can be extracted by decomposing this signal on a wavelet orthonormal basis of L/sup 2/(R/sup n/), the vector space of measurable, square-integrable n-dimensional functions. In L/sup 2/(R), a wavelet orthonormal basis is a family of functions which is built by dilating and translating a unique function psi (x). This decomposition defines an orthogonal multiresolution representation called a wavelet representation. It is computed with a pyramidal algorithm based on convolutions with quadrature mirror filters. Wavelet representation lies between the spatial and Fourier domains. For images, the wavelet representation differentiates several spatial orientations. The application of this representation to data compression in image coding, texture discrimination and fractal analysis is discussed.>