This paper introduces MNE-RT, a Python package designed for real-time neural feature extraction from magne-toencephalography (MEG) and electroencephalography (EEG) signals in Brain-Computer Interface (BCI) systems. The package incorporates efficient algorithms spanning traditional univariate metrics, such as frequency band power and entropy, to advanced bivariate connectivity measures. It is compatible with various recording systems, enabling the extraction of neural targets from brain signals in real time, with potential applications in enhancing neurofeedback efficacy.

In the past years, many signal processing operations have been successfully adapted to the graph setting. One elegant and effective approach is to exploit the eigendecomposition of a graph shift operator (GSO), such as the adjacency or Laplacian operator, to define a graph Fourier transform for projecting graph signals on the corresponding basis. However, the extension of this scheme to directed graphs is challenging since the associated GSO is non-symmetric and, in general, not diagonalizable. Here, we build upon a recent framework that adds a minimal number of edges to allow diagonalization of the GSO and thus provide a proper graph Fourier transform. Furthermore, we show that such minimal addition of edges creates a cycle cover and that it is essential for the phase analysis of a signal throughout the graph. Concurrently, we propose a generalization of the Hilbert transform interpreted over the newfound cycle cover, which re-establishes intuitions from traditional Hilbert transform, equivalent to the generalized Hilbert transform on a single cycle. This generalization leads to a number of simple and elegant recipes to effectively exploit the phase information of graph signals provided by the graph Fourier transform. The feasibility of the approach is demonstrated on several examples.

Linking time-varying functional brain activity with underlying neural architecture remains a complex and challenging endeavor. A recent framework for this undertaking is graph signal processing (GSP), where functional activity patterns are treated as signals living on a graph that is characterized by structural connectivity. Then graph spectral filtering can be used to obtain the parts of functional activity that are more or less smooth on the graph; i.e., more coupled or decoupled from brain structure, respectively. Given the time-varying behavior of functional magnetic resonance imaging (fMRI) networks, structure- function coupling may also change over time. Here, we leverage the GSP framework in a sliding-window setting to investigate the dynamics of brain structure-function coupling during resting-state at the node- and edge-wise levels. We conclude that dynamics are captured by both node- and edge-wise metrics of structure-function coupling and we identify principal patterns of dynamic functional connectivity respectively coupled and decoupled from structure.

indicators. The outstanding spatial extent and resolution of this type of data open unique opportunities for understanding the complex organization of neuronal circuits across the brain. However, the analysis of this data remains challenging because the observed variations in fluorescence are, in fact, noisy indirect measures of the neuronal activity. Moreover, measuring over large field-of-view negatively impact temporal resolution and signal-to-noise ratio, which further impedes conventional spike inference. Here we argue that meaningful information can be extracted from large-scale functional imaging data by deconvolving with the calcium response and by modeling moments of sustained neuronal activity instead of individual spikes. Specifically, we characterize the calcium response by a linear system of which the inverse is a differential operator. This operator is then included in a regularization term promoting sparsity of activity transients through generalized total variation. Our results illustrate the numerical performance of the algorithm on simulated signals; i.e., we show the firing rate phase transition at which our model outperforms spike inference. Finally, we apply the proposed algorithm to experimental data from zebrafish larvæ. In particular, we show that, when applied to a specific group of neurons, the algorithm retrieves neural activation that matches the locomotor behavior unknown to the method.

Functional magnetic resonance imaging is a non-invasive tomographic imaging modality that has provided insights into system-level brain function. New analysis methods are emerging to study the dynamic behavior of brain activity. The innovation-driven co-activation pattern (iCAP) approach is one such approach that relies on the detection of timepoints with a significant transient activity to subsequently retrieve spatially and temporally overlapping large-scale brain networks. To recover temporal profiles of the iCAPs for further time-resolved analysis, spatial patterns are fitted back to the activity-inducing signals. In this crucial step, spatial dependences can hinder the recovery of temporal overlapping activity. To overcome this effect, we propose a novel back-projection method that optimally fits activity-inducing signals given a set of transient timepoints and spatial maps of iCAPs, thus taking into account both spatial and temporal constraints. Validation on simulated data shows that transient-based constraints improve the quality of fitted time courses. Further evaluation on experimental data demonstrates that overfitting and underfitting are prevented by the use of optimized spatio-temporal constraints. Spatial and temporal properties of resulting iCAPs support that brain activity is characterized by the recurrent co-activation and co-deactivation of spatially overlapping large-scale brain networks. This new approach opens new avenues to explore the brain's dynamic core.

Modern neuroimaging techniques offer disctinct views on brain structure and function. Data acquired using these techniques can be analyzed in terms of its network structure to identify organizing principles at the systems level. Graph representations are flexible frameworks where nodes are related to brain regions and edges to structural or functional links. Most research to date has focused on analyzing these graphs reflecting structure or function. Graph signal processing (GSP) is an emerging area of research where signals at the nodes are studied atop the underlying graph structure. Here, we review GSP tools for brain imaging data and discuss their potential to integrate brain structure with function. We discuss how brain activity can be meaningfully filtered. We also derive surrogate data as a null model to test significance for graph signals. We review that individuals with less concentration on graph high frequency could switch attention faster.

Modern neuroimaging techniques provide us with unique views on brain structure and function; i.e., how the brain is wired, and where and when activity takes place. Data acquired using these techniques can be analyzed in terms of its network structure to reveal organizing principles at the systems level. Graph representations are versatile models where nodes are associated to brain regions and edges to structural or functional connections. Structural graphs model neural pathways in white matter, which are the anatomical backbone between regions. Functional graphs are built based on functional connectivity, which is a pairwise measure of statistical interdependency between pairs of regional activity traces. Therefore, most research to date has focused on analyzing these graphs reflecting structure or function. Graph signal processing (GSP) is an emerging area of research where signals recorded at the nodes of the graph are studied atop the underlying graph structure. An increasing number of fundamental operations have been generalized to the graph setting, allowing to analyze the signals from a new viewpoint. Here, we review GSP for brain imaging data and discuss their potential to integrate brain structure, contained in the graph itself, with brain function, residing in the graph signals. We review how brain activity can be meaningfully filtered based on concepts of spectral modes derived from brain structure. We also derive other operations such as surrogate data generation or decompositions informed by cognitive systems. In sum, GSP offers a novel framework for the analysis of brain imaging data.

Functional magnetic resonance imaging (fMRI) provides a window on the human brain at work. Spontaneous brain activity measured during resting-state has already provided many insights into brain function. In particular, recent interest in dynamic interactions between brain regions has increased the need for more advanced modeling tools. Here, we deploy a recent fMRI deconvolution technique to express resting-state temporal fluctuations as a combination of large-scale functional network activity profiles. Then, building upon a novel sparse coupled hidden Markov model (SCHMM) framework, we parameterised their temporal evolution as a mix between intrinsic dynamics, and a restricted set of cross-network modulatory couplings extracted in data-driven manner. We demonstrate and validate the method on simulated data, for which we observed that the SCHMM could accurately estimate network dynamics, revealing more precise insights about direct network-to-network modulatory influences than with conventional correlational methods. On experimental resting-state fMRI data, we unraveled a set of reproducible cross-network couplings across two independent datasets. Our framework opens new perspectives for capturing complex temporal dynamics and their changes in health and disease.

The study of complex systems greatly benefits from graph models and their analysis. In particular, the eigendecomposition of the graph Laplacian lets emerge properties of global organization from local interactions; e.g., the Fiedler vector has the smallest nonzero eigenvalue and plays a key role for graph clustering. Graph signal processing focuses on the analysis of signals that are attributed to the graph nodes. Again, the eigendecomposition of the graph Laplacian is important to define the graph Fourier transform and extend conventional signal-processing operations to graphs. Here, we introduce the design of Slepian graph signals by maximizing energy concentration in a predefined subgraph given a graph spectral bandlimit. We establish a novel link with classical Laplacian embedding and graph clustering, which provides a meaning to localized graph frequencies.

The increasing availability of network data is leading to a growing interest in processing of signals on graphs. One notable tool for extending conventional signal-processing operations to networks is the graph Fourier transform that can be obtained as the eigendecomposition of the graph Laplacian. In this letter, we used the graph Fourier transform to define a new method for generating surrogate graph signals. The approach is based on sign-randomization of the graph Fourier coefficients and, therefore, the correlation structure of the surrogate graph signals (i.e., smoothness on the graph topology) is imposed by the measured data. The proposed method of surrogate data generation can be widely applied for nonparametric statistical hypothesis testing. Here, we showed a proof-of-concept with a high-density electroencephalography dataset.

We propose a texture learning approach that exploits local organizations of scales and directions. First, linear combinations of Riesz wavelets are learned using kernel support vector machines. The resulting texture signatures are modeling optimal class-wise discriminatory properties. The visualization of the obtained signatures allows verifying the visual relevance of the learned concepts. Second, the local orientations of the signatures are optimized to maximize their responses, which is carried out analytically and can still be expressed as a linear combination of the initial steerable Riesz templates. The global process is iteratively repeated to obtain final rotation-covariant texture signatures. Rapid convergence of class-wise signatures is observed, which demonstrates that the instances are projected into a feature space that leverages the local organizations of scales and directions. Experimental evaluation reveals average classification accuracies in the range of 97% to 98% for the Outex_TC_00010, the Outex_TC_00012, and the Contrib_TC_00000 suites for even orders of the Riesz transform, and suggests high robustness to changes in images orientation and illumination. The proposed framework requires no arbitrary choices of scales and directions and is expected to perform well in a large range of computer vision applications.

We discuss a novel sparsity prior for compressive imaging in the context of the theory of compressed sensing with coherent redundant dictionaries, based on the observation that natural images exhibit strong average sparsity over multiple coherent frames. We test our prior and the associated algorithm, based on an analysis reweighted formulation, through extensive numerical simulations on natural images for spread spectrum and random Gaussian acquisition schemes. Our results show that average sparsity outperforms state-of-the-art priors that promote sparsity in a single orthonormal basis or redundant frame, or that promote gradient sparsity. Code and test data are available at https://github.com/basp-group/sopt.

We study the impact of sampling theorems on the fidelity of sparse image reconstruction on the sphere. We discuss how a reduction in the number of samples required to represent all information content of a band-limited signal acts to improve the fidelity of sparse image reconstruction, through both the dimensionality and sparsity of signals. To demonstrate this result, we consider a simple inpainting problem on the sphere and consider images sparse in the magnitude of their gradient. We develop a framework for total variation inpainting on the sphere, including fast methods to render the inpainting problem computationally feasible at high resolution. Recently a new sampling theorem on the sphere was developed, reducing the required number of samples by a factor of two for equiangular sampling schemes. Through numerical simulations, we verify the enhanced fidelity of sparse image reconstruction due to the more efficient sampling of the sphere provided by the new sampling theorem.

Magnetic resonance spectroscopic imaging (MRSI) is a powerful tool capable of providing spatially localized maps of metabolite concentrations. Its utility, however, is often depreciated by spectral leakage artifacts resulting from low spatial resolution measurements through an effort to reduce acquisition times. Though model-based techniques can help circumvent these drawbacks, they require strong prior knowledge, and can introduce additional artifacts when the underlying models are inaccurate. We introduce a novel scheme in which a generative model is estimated from the raw MRSI data via a regularized variational framework that minimizes the model approximation error within a measurement-prescribed subspace. As additional a priori information, our approach relies only upon a measured field inhomogeneity map at high spatial resolution. We demonstrate the feasibility of our approach on both synthetic and experimental data.

The nine papers in this special section represent the state-of-art in the area of multimodal biomedical imaging algorithms and applications.

We propose near-affine-invariant texture descriptors derived from isotropic wavelet frames for the characterization of lung tissue patterns in high-resolution computed tomography (HRCT) imaging. Affine invariance is desirable to enable learning of nondeterministic textures without a priori localizations, orientations, or sizes. When combined with complementary gray-level histograms, the proposed method allows a global classification accuracy of 76.9% with balanced precision among five classes of lung tissue using a leave-one-patient-out cross validation, in accordance with clinical practice.

We propose a novel compressed sensing technique to accelerate the magnetic resonance imaging (MRI) acquisition process. The method, coined spread spectrum MRI or simply s(2)MRI, consists of premodulating the signal of interest by a linear chirp before random k-space under-sampling, and then reconstructing the signal with nonlinear algorithms that promote sparsity. The effectiveness of the procedure is theoretically underpinned by the optimization of the coherence between the sparsity and sensing bases. The proposed technique is thoroughly studied by means of numerical simulations, as well as phantom and in vivo experiments on a 7T scanner. Our results suggest that s(2)MRI performs better than state-of-the-art variable density k-space under-sampling approaches.

We propose a new framework to extract the activity-related component in the BOLD functional magnetic resonance imaging (fMRI) signal. As opposed to traditional fMRI signal analysis techniques, we do not impose any prior knowledge of the event timing. Instead, our basic assumption is that the activation pattern is a sequence of short and sparsely distributed stimuli, as is the case in slow event-related fMRI. We introduce new wavelet bases, termed “activelets”, which sparsify the activity-related BOLD signal. These wavelets mimic the behavior of the differential operator underlying the hemodynamic system. To recover the sparse representation, we deploy a sparse-solution search algorithm. The feasibility of the method is evaluated using both synthetic and experimental fMRI data. The importance of the activelet basis and the non-linear sparse recovery algorithm is demonstrated by comparison against classical B-spline wavelets and linear regularization, respectively.

We present a functional framework for the design of tight steerable wavelet frames in any number of dimensions. The 2-D version of the method can be viewed as a generalization of Simoncelli's steerable pyramid that gives access to a larger palette of steerable wavelets via a suitable parametrization. The backbone of our construction is a primal isotropic wavelet frame that provides the multiresolution decomposition of the signal. The steerable wavelets are obtained by applying a one-to-many mapping (Nth-order generalized Riesz transform) to the primal ones. The shaping of the steerable wavelets is controlled by an M×M unitary matrix (where M is the number of wavelet channels) that can be selected arbitrarily; this allows for a much wider range of solutions than the traditional equiangular configuration (steerable pyramid). We give a complete functional description of these generalized wavelet transforms and derive their steering equations. We describe some concrete examples of transforms, including some built around a Mallat-type multiresolution analysis of L(2)(R(d)), and provide a fast Fourier transform-based decomposition algorithm. We also propose a principal-component-based method for signal-adapted wavelet design. Finally, we present some illustrative examples together with a comparison of the denoising performance of various brands of steerable transforms. The results are in favor of an optimized wavelet design (equalized principal component analysis), which consistently performs best.

Nonlocal means (NLM) is an effective denoising method that applies adaptive averaging based on similarity between neighborhoods in the image. An attractive way to both improve and speed-up NLM is by first performing a linear projection of the neighborhood. One particular example is to use principal components analysis (PCA) to perform dimensionality reduction. Here, we derive Stein's unbiased risk estimate (SURE) for NLM with linear projection of the neighborhoods. The SURE can then be used to optimize the parameters by a search algorithm or we can consider a linear expansion of multiple NLMs, each with a fixed parameter set, for which the optimal weights can be found by solving a linear system of equations. The experimental results demonstrate the accuracy of the SURE and its successful application to tune the parameters for NLM.

We consider images probed through incomplete and noisy Fourier coverages, both in the context of radio interferometry (RI) and of magnetic resonance imaging (MRI). We show that the quality of signal reconstruction can be significantly enhanced by the introduction of a linear chirp modulation, which induces a spread spectrum phenomenon.

Simple weighted undirected graphs with a fixed number of vertices and fixed vertex orderings can be used to represent data and patterns in a wide variety of scientific and engineering domains. Classification of such graphs by existing graph matching methods perform rather poorly because they do not exploit their specificity. As an alternative, methods relying on vector-space embedding hold promising potential. We propose two such techniques that can be deployed as a front-end for any pattern recognition classifiers: one has low computational cost but generates high-dimensional spaces, while the other is more computationally demanding but can yield relatively low-dimensional vector space representations. We show experimental results on an fMRI brain state decoding task and discuss the shortfalls of graph edit distance for the type of graph under consideration.

Our main goal in this paper is to set the foundations of a general continuous-domain framework for designing steerable, reversible signal transformations (a.k.a. frames) in multiple dimensions ( d >or= 2). To that end, we introduce a self-reversible, Nth-order extension of the Riesz transform. We prove that this generalized transform has the following remarkable properties: shift-invariance, scale-invariance, inner-product preservation, and steerability. The pleasing consequence is that the transform maps any primary wavelet frame (or basis) of [Formula: see text] into another "steerable" wavelet frame, while preserving the frame bounds. The concept provides a functional counterpart to Simoncelli's steerable pyramid whose construction was primarily based on filterbank design. The proposed mechanism allows for the specification of wavelets with any order of steerability in any number of dimensions; it also yields a perfect reconstruction filterbank algorithm. We illustrate the method with the design of a novel family of multidimensional Riesz-Laplace wavelets that essentially behave like the N th-order partial derivatives of an isotropic Gaussian kernel.

We introduce an Nth-order extension of the Riesz transform in d dimensions. We prove that this generalized transform has the following remarkable properties: shift-invariance, scale-invariance, inner-product preservation, and steerability. The pleasing consequence is that the transform maps any primary wavelet frame (or basis) of L2(¿d) into another ¿steerable¿ wavelet frame, while preserving the frame bounds. The concept provides a rigorous functional counterpart to Simoncelli's steerable pyramid whose construction was entirely based on digital filter design. The proposed mechanism allows for the specification of wavelets with any order of steerability in any number of dimensions; it also yields a perfect reconstruction filterbank algorithm. We illustrate the method using a Mexican-hat-like polyharmonic spline wavelet transform as our primary frame.

Abstract Within the context of emission tomography, we study volumetric reconstruction methods based on the Expectation Maximization (EM) algorithm. We show, for the first time, the equivalence of the standard implementation of the EM‐based reconstruction with an implementation based on hardware‐accelerated volume rendering for nearest‐neighbor (NN) interpolation. This equivalence suggests that higher‐order kernels should be used with caution and do not necessarily lead to better performance. We also show that the EM algorithm can easily be adapted for different lattices, the body‐centered cubic (BCC) one in particular. For validation purposes, we use the 3D version of the Shepp‐Logan synthetic phantom, for which we derive closed‐form analytical expressions of the projection data. The experimental results show the theoretically‐predicted optimality of NN interpolation in combination with the EM algorithm, for both the noiseless and the noisy case. Moreover, reconstruction on the BCC lattice leads to superior accuracy, more compact data representation, and better noise reduction compared to the Cartesian one. Finally, we show the usefulness of the proposed method for optical projection tomography of a mouse embryo.

Non-local means (NLM) provides a powerful framework for denoising. However, there are a few parameters of the algorithm-most notably, the width of the smoothing kernel-that are data-dependent and difficult to tune. Here, we propose to use Stein's unbiased risk estimate (SURE) to monitor the mean square error (MSE) of the NLM algorithm for restoration of an image corrupted by additive white Gaussian noise. The SURE principle allows to assess the MSE without knowledge of the noise-free signal. We derive an explicit analytical expression for SURE in the setting of NLM that can be incorporated in the implementation at low computational cost. Finally, we present experimental results that confirm the optimality of the proposed parameter selection.

In this contribution, we study the notion of affine invariance (specifically, invariance to the shifting, scaling, and rotation of the coordinate system) as a starting point for the development of mathematical tools and approaches useful in the characterization and analysis of multivariate fractional Brownian motion (fBm) fields. In particular, using a rigorous and powerful distribution theoretic formulation, we extend previous results of Blu and Unser (2006) to the multivariate case, showing that polyharmonic splines and fBm processes can be seen as the (deterministic vs stochastic) solutions to an identical fractional partial differential equation that involves a fractional Laplacian operator. We then show that wavelets derived from polyharmonic splines have a behavior similar to the fractional Laplacian, which also turns out to be the whitening operator for fBm fields. This fact allows us to study the probabilistic properties of the wavelet transform coefficients of fBm-like processes, leading for instance to ways of estimating the Hurst exponent of a multiparameter process from its wavelet transform coefficients. We provide theoretical and experimental verification of these results. To complement the toolbox available for multiresolution processing of stochastic fractals, we also introduce an extended family of multidimensional multiresolution spaces for a large class of (separable and nonseparable) lattices of arbitrary dimensionality.

The monogenic signal is the natural 2-D counterpart of the 1-D analytic signal. We propose to transpose the concept to the wavelet domain by considering a complexified version of the Riesz transform which has the remarkable property of mapping a real-valued (primary) wavelet basis of L(2) (R(2)) into a complex one. The Riesz operator is also steerable in the sense that it give access to the Hilbert transform of the signal along any orientation. Having set those foundations, we specify a primary polyharmonic spline wavelet basis of L(2) (R(2)) that involves a single Mexican-hat-like mother wavelet (Laplacian of a B-spline). The important point is that our primary wavelets are quasi-isotropic: they behave like multiscale versions of the fractional Laplace operator from which they are derived, which ensures steerability. We propose to pair these real-valued basis functions with their complex Riesz counterparts to specify a multiresolution monogenic signal analysis. This yields a representation where each wavelet index is associated with a local orientation, an amplitude and a phase. We give a corresponding wavelet-domain method for estimating the underlying instantaneous frequency. We also provide a mechanism for improving the shift and rotation-invariance of the wavelet decomposition and show how to implement the transform efficiently using perfect-reconstruction filterbanks. We illustrate the specific feature-extraction capabilities of the representation and present novel examples of wavelet-domain processing; in particular, a robust, tensor-based analysis of directional image patterns, the demodulation of interferograms, and the reconstruction of digital holograms.

In this work, we propose a new image rotation algorithm. The main feature of our approach is the symmetric reversibility, which means that when using the same algorithm for the converse operation, then the initial data is recovered exactly. To that purpose, we decompose the lattice conversion process into three successive shear operations. The translations along the shear directions are implemented by 1-D convolutions, with new appropriate fractional delay filters. Also, the method is fast and provides high-quality resampled images.

We propose new discrete-to-continuous interpolation models for hexagonally sampled data, that generalize two families of splines developed in the literature for the hexagonal lattice, to say the hex-splines and three directional box-splines. This extension is inspired by the construction of MOMS functions in 1-D, that generalize and outperform classical 1-D B-splines [1]. Our new generators have optimal approximation theoretic performances, for exactly the same computation cost as their spline counterparts.

We introduce a new semi-orthogonal complex wavelet basis of L2(R2). The basis functions are associated to the complex gradient-Laplace operator, which plays a central role in image processing. We define analytically a single-generator wavelet that is shifted on the coset positions of the subsampling matrix. Next, we propose the "wavelet Marr pyramid" for an extension of the new basis that achieves near shift-invariance and steerability (using a Gaussian-like smoothing kernel), for a mild redundancy factor only. This new wavelet pyramid decomposition closely mimicks the basic operations of Marx's framework for early vision. The pyramid is implemented by a fast filterbank algorithm using the FFT.

A fast computational method is given for the Fourier transform of the polyharmonic B-spline autocorrelation sequence in d dimensions. The approximation error is exponentially decaying with the number of terms taken into account. The algorithm improves speed upon a simple truncated-sum approach. Moreover, it is virtually independent of the spline's order. The autocorrelation filter directly serves for various tasks related to polyharmonic splines, such as interpolation, orthonormalization, and wavelet basis design.

Due to the limited depth of field of brightfield microscopes, it is usually impossible to image thick specimens entirely in focus. By optically sectioning the specimen, the in-focus information at the specimen's surface can be acquired over a range of images. Commonly based on a high-pass criterion, extended-depth-of-field methods aim at combining the in-focus information from these images into a single image of the texture on the specimen's surface. The topography provided by such methods is usually limited to a map of selected in-focus pixel positions and is inherently discretized along the axial direction, which limits its use for quantitative evaluation. In this paper, we propose a method that jointly estimates the texture and topography of a specimen from a series of brightfield optical sections; it is based on an image formation model that is described by the convolution of a thick specimen model with the microscope's point spread function. The problem is stated as a least-squares minimization where the texture and topography are updated alternately. This method also acts as a deconvolution when the in-focus PSF has a blurring effect, or when the true in-focus position falls in between two optical sections. Comparisons to state-of-the-art algorithms and experimental results demonstrate the potential of the proposed approach.

A new grid conversion method is proposed to resample between two 2-D periodic lattices with the same sampling density. The main feature of our approach is the symmetric reversibility, which means that when using the same algorithm for the converse operation, then the initial data is recovered exactly. To that purpose, we decompose the lattice conversion process into (at most) three successive shear operations. The translations along the shear directions are implemented by 1-D fractional delay operators, which revert to simple 1-D convolutions, with appropriate filters that yield the property of symmetric reversibility. We show that the method is fast and provides high-quality resampled images. Applications of our approach can be found in various settings, such as grid conversion between the hexagonal and the Cartesian lattice, or fast implementation of affine transformations such as rotations.

A distinction is usually made between wavelet bases and wavelet frames. The former are associated with a one-to-one representation of signals, which is somewhat constrained but most efficient computationally. The latter are over-complete, but they offer advantages in terms of flexibility (shape of the basis functions) and shift-invariance. In this paper, we propose a framework for improved wavelet analysis based on an appropriate pairing of a wavelet basis with a mildly redundant version of itself (frame). The processing is accomplished in four steps: 1) redundant wavelet analysis, 2) wavelet-domain processing, 3) projection of the results onto the wavelet basis, and 4) reconstruction of the signal from its nonredundant wavelet expansion. The wavelet analysis is pyramid-like and is obtained by simple modification of Mallat's filterbank algorithm (e.g., suppression of the down-sampling in the wavelet channels only). The key component of the method is the subband regression filter (Step 3) which computes a wavelet expansion that is maximally consistent in the least squares sense with the redundant wavelet analysis. We demonstrate that this approach significantly improves the performance of soft-threshold wavelet denoising with a moderate increase in computational cost. We also show that the analysis filters in the proposed framework can be adjusted for improved feature detection; in particular, a new quincunx Mexican-hat-like wavelet transform that is fully reversible and essentially behaves the (gamma/2)th Laplacian of a Gaussian.

Our aim in this paper is to tighten the link between wavelets, some classical image-processing operators, and David Marr's theory of early vision. The cornerstone of our approach is a new complex wavelet basis that behaves like a smoothed version of the Gradient-Laplace operator. Starting from first principles, we show that a single-generator wavelet can be defined analytically and that it yields a semi-orthogonal complex basis of L2 (R2), irrespective of the dilation matrix used. We also provide an efficient FFT-based filterbank implementation. We then propose a slightly redundant version of the transform that is nearly translation-invariant and that is optimized for better steerability (Gaussian-like smoothing kernel).We call it the Marr-like wavelet pyramid because it essentially replicates the processing steps in Marr's theory of early vision.We use it to derive a primal wavelet sketch which is a compact description of the image by a multiscale, subsampled edge map. Finally, we provide an efficient iterative algorithm for the reconstruction of an image from its primal wavelet sketch.

Tomographic reconstruction from positron emission tomography (PET) data is an ill-posed problem that requires regularization. An attractive approach is to impose an l(1) -regularization constraint, which favors sparse solutions in the wavelet domain. This can be achieved quite efficiently thanks to the iterative algorithm developed by Daubechies et al., 2004. In this paper, we apply this technique and extend it for the reconstruction of dynamic (spatio-temporal) PET data. Moreover, instead of using classical wavelets in the temporal dimension, we introduce exponential-spline wavelets (E-spline wavelets) that are specially tailored to model time activity curves (TACs) in PET. We show that the exponential-spline wavelets naturally arise from the compartmental description of the dynamics of the tracer distribution. We address the issue of the selection of the "optimal" E-spline parameters (poles and zeros) and we investigate their effect on reconstruction quality. We demonstrate the usefulness of spatio-temporal regularization and the superior performance of E-spline wavelets over conventional Battle-LemariE wavelets in a series of experiments: the 1-D fitting of TACs, and the tomographic reconstruction of both simulated and clinical data. We find that the E-spline wavelets outperform the conventional wavelets in terms of the reconstructed signal-to-noise ratio (SNR) and the sparsity of the wavelet coefficients. Based on our simulations, we conclude that replacing the conventional wavelets with E-spline wavelets leads to equal reconstruction quality for a 40% reduction of detected coincidences, meaning an improved image quality for the same number of counts or equivalently a reduced exposure to the patient for the same image quality.

We introduce a family of box splines for efficient, accurate and smooth reconstruction of volumetric data sampled on the Body Centered Cubic (BCC) lattice, which is the favorable volumetric sampling pattern due to its optimal spectral sphere packing property. First, we construct a box spline based on the four principal directions of the BCC lattice that allows for a linear C(0) reconstruction. Then, the design is extended for higher degrees of continuity. We derive the explicit piecewise polynomial representation of the C(0) and C(2) box splines that are useful for practical reconstruction applications. We further demonstrate that approximation in the shift-invariant space---generated by BCC-lattice shifts of these box splines---is {twice} as efficient as using the tensor-product B-spline solutions on the Cartesian lattice (with comparable smoothness and approximation order, and with the same sampling density). Practical evidence is provided demonstrating that not only the BCC lattice is generally a more accurate sampling pattern, but also allows for extremely efficient reconstructions that outperform tensor-product Cartesian reconstructions.

In this work, we propose a new grid conversion algorithm between the hexagonal lattice and the orthogonal (a.k.a. Cartesian) lattice. The conversion process, named H2O, is easy to implement and is perfectlyreversibleusing the same algorithm to return from one lattice to the other. The key observation of our approach is a decomposition of the lattice conversion as a sequence of shearing operations along three well-chosen directions. Hence, only 1-D fractional sample delay operators are required, which can be implemented by simple convolutions. The proposed algorithm combines reversibility and fast 1-D operations, together with high-quality resampled images.

The reconstruction of a continuous-domain representation from sampled data is an essential element of many image processing tasks, in particular, image resampling. Until today, most image data have been available on Cartesian lattices, despite the many theoretical advantages of hexagonal sampling. In this paper, we propose new reconstruction methods for hexagonally sampled data that use the intrinsically 2-D nature of the lattice, and that at the same time remain practical and efficient. To that aim, we deploy box-spline and hex-spline models, which are notably well adapted to hexagonal lattices. We also rely on the quasi-interpolation paradigm to design compelling prefilters; that is, the optimal filter for a prescribed design is found using recent results from approximation theory. The feasibility and efficiency of the proposed methods are illustrated and compared for a hexagonal to Cartesian grid conversion problem.

Magnetic resonance spectroscopy imaging (MRSI) is an attractive tool for medical imaging. However, its practical use is often limited by the intrinsic low spatial resolution and long acquisition time. Spectral localization by imaging (SLIM) has been proposed as a non-Fourier reconstruction algorithm that incorporates spatial a priori information about spectroscopically uniform compartments. Unfortunately, the influence of the magnetic field inhomogeneity--in particular, the susceptibility effects at tissues' boundaries--undermines the validity of the compartmental model. Therefore, we propose BSLIM as an extension of SLIM with field inhomogeneity compensation. A B0-field inhomogeneity map, which can be acquired rapidly and at high resolution, is used by the new algorithm as additional a priori information. We show that the proposed method is distinct from the generalized SLIM (GSLIM) framework. Experimental results of a two-compartment phantom demonstrate the feasibility of the method and the importance of inhomogeneity compensation.

The Matern class is a parametric family of autocorrelation functions that is commonly used in geostatistics. We argue that a generalized, anisotropic version of this model is suitable for capturing the correlation structure of a variety of natural images. We specify the optimal space for the MMSE reconstruction of stochastic Matern signals from their uniformly-sampled noisy measurements (generalized sampling problem). We prove that the optimal reconstruction space is generated by the multi-integer shifts of a Matern function which form a Riesz basis. Based on this representation, we propose a practical filter-based reconstruction method that relies on the prior identification of the Matern parameters from the measured data. We present experimental results to justify the use of the Matern model and to demonstrate the performance of our signal-adapted reconstruction technique

Many experimental paradigms in biology aim at studying the response to coordinated stimuli. In dynamic imaging experiments, the observed data is often not straightforward to interpret and not directly measurable in a quantitative fashion. Consequently, the data is typically preprocessed in an ad hoc fashion and the results subjected to a statistical inference at the level of a population. We propose a new framework for analyzing time-lapse images that exploits some a priori knowledge on the type of temporal response and takes advantage of the spatial correlation of the data. This is achieved by processing the data in the wavelet domain and expressing the time course of each wavelet coefficient by a linear model. We end up with a statistical map in the spatial domain for the contrast of interest (i.e., the stimulus response). The feasibility of the method is demonstrated by an example of intrinsic microscopy imaging of mice's brains during coordinated sensory stimulation

Recently, we have proposed a new framework for detecting brain activity from fMRI data, which is based on the spatial discrete wavelet transform. The standard wavelet-based approach performs a statistical test in the wavelet domain, and therefore fails to provide a rigorous statistical interpretation in the spatial domain. The new framework provides an "integrated" approach: the data is processed in the wavelet domain (by thresholding wavelet coefficients), and a suitable statistical testing procedure is applied afterwards in the spatial domain. This method is based on conservative assumptions only and has a strong type-I error control by construction. At the same time, it has a sensitivity comparable to that of SPM. Here, we discuss the extension of our algorithm to the redundant discrete wavelet transform, which provides a shift-invariant detection scheme. The key features of our technique are illustrated with experimental results. An implementation of our framework is available as a toolbox (WSPM) for the SPM2 software

Three-directional box-splines are particularly well-suited to interpolate and approximate hexagonally sampled data. In this paper, we propose a computationally efficient end-to-end reconstruction process. First, we introduce a prefiltering step that is based on a quasi-interpolation scheme using low-complexity finite-impulse-response (FIR) filters. Second, we derive a closed analytical expression for three-directional box-splines of any order that leads to a fast evaluation of the spline surface. All operations act locally on the data, and thus are well adapted to applications dealing with large images. To demonstrate the feasibility of our method, we implemented the complete procedure and we present experimental results.

We propose a new characterization of three-directional box-splines, which are well adapted for interpolation and approximation on hexagonal lattices. Inspired by a construction already applied with success for exponential splines and hex-splines, we characterize a box-spline as a convolution of a generating function, which is a Green function of the spline's associated differential operator, and a discrete filter that plays the role of a localization operator. This process leads to an elegant analytical expression of three-directional box-splines. It also brings along a particularly efficient implementation

Motivated by the fractal-like behavior of natural images, we develop a smoothing technique that uses a regularization functional which is a fractional iterate of the Laplacian. This type of functional was initially introduced by Duchon for the approximation of nonuniformily sampled, multidimensional data. He proved that the general solution is a smoothing spline that is represented by a linear combination of radial basis functions (RBFs). Unfortunately, this is tedious to implement for images because of the poor conditioning of RBFs and their lack of decay. Here, we present a much more efficient method for the special case of a uniform grid. The key idea is to express Duchon's solution in a fractional polyharmonic B-spline basis that spans the same space as the RBFs. This allows us to derive an algorithm where the smoothing is performed by filtering in the Fourier domain. Next, we prove that the above smoothing spline can be optimally tuned to provide the MMSE estimation of a fractional Brownian field corrupted by white noise. This is a strong result that not only yields the best linear filter (Wiener solution), but also the optimal interpolation space, which is not bandlimited. It also suggests a way of using the noisy data to identify the optimal parameters (order of the spline and smoothing strength), which yields a fully automatic smoothing procedure. We evaluate the performance of our algorithm by comparing it against an oracle Wiener filter, which requires the knowledge of the true noiseless power spectrum of the signal. We find that our approach performs almost as well as the oracle solution over a wide range of conditions.

We provide a new comparison between hexagonal and orthogonal lattices, based on approximation theory. For each of the lattices, we select the "natural" spline basis function as generator for a shift-invariant function space; i.e., the tensor-product B-splines for the orthogonal lattice and the non-separable hex-splines for the hexagonal lattice. For a given order of approximation, we compare the asymptotic constants of the error kernels, which give a very good indication of the approximation quality. We find that the approximation quality on the hexagonal lattice is consistently better, when choosing lattices with the same sampling density. The area sampling gain related to these asymptotic constants quickly converges when the order of approximation of the basis functions increases. Surprisingly, nearest-neighbor interpolation does not allow to profit from the hexagonal grid. For practical purposes, the second-order hex-spline (i.e., constituted by linear patches) appears as a particularly useful candidate to exploit the advantages of hexagonal lattices when representing images on them.

Shannon's sampling theory and its variants provide effective solutions to the problem of reconstructing a signal from its samples in some "shift-invariant" space, which may or may not be bandlimited. In this paper, we present some further justification for this type of representation, while addressing the issue of the specification of the best reconstruction space. We consider a realistic setting where a multidimensional signal is prefiltered prior to sampling and the samples corrupted by additive noise. We consider two formulations of the reconstruction problem. In the first deterministic approach, we determine the continuous-space function that minimizes a variational, Tikhonov-like criterion that includes a discrete data term and a suitable continuous-space regularization functional. In the second formulation, we seek the minimum mean square error (MMSE) estimation of the signal assuming that the input signal is a realization of a stationary random process. Interestingly, both approaches yield a solution included in some optimal shift-invariant space that is generally not bandlimited. The solutions can be made equivalent by choosing a regularization operator that corresponds to the whitening filter of the process. We present some practical examples that demonstrate the optimality of the approach.

The dilation matrix associated with the three-dimensional (3-D) face-centered cubic (FCC) sublattice is often considered to be the natural 3-D extension of the two-dimensional (2-D) quincunx dilation matrix. However, we demonstrate that both dilation matrices are of different nature: while the 2-D quincunx matrix is a similarity transform, the 3-D FCC matrix is not. More generally, we show that is impossible to obtain a dilation matrix that is a similarity transform and performs downsampling of the Cartesian lattice by a factor of two in more than two dimensions. Furthermore, we observe that the popular 3-D FCC subsampling scheme alternates between three different lattices: Cartesian, FCC, and quincunx. The latter one provides a less isotropic sampling density, a property that should be taken into account to properly orient 3-D data before processing using such a subsampling matrix.

We present a new family of two-dimensional and three-dimensional orthogonal wavelets which uses quincunx sampling. The orthogonal refinement filters have a simple analytical expression in the Fourier domain as a function of the order lamda, which may be noninteger. We can also prove that they yield wavelet bases of L2(R2) for any lambda > 0. The wavelets are fractional in the sense that the approximation error at a given scale a decays like O(a(lamda)); they also essentially behave like fractional derivative operators. To make our construction practical, we propose a fast Fourier transform-based implementation that turns out to be surprisingly fast. In fact, our method is almost as efficient as the standard Mallat algorithm for separable wavelets.

In this paper, we use polyharmonic B-splines to build multidimensional wavelet bases. These functions are nonseparable, multidimensional basis functions that are localized versions of radial basis functions. We show that Rabut's elementary polyharmonic B-splines do not converge to a Gaussian as the order parameter increases, as opposed to their separable B-spline counterparts. Therefore, we introduce a more isotropic localization operator that guarantees this convergence, resulting into the isotropic polyharmonic B-splines. Next, we focus on the two-dimensional quincunx subsampling scheme. This configuration is of particular interest for image processing because it yields a finer scale progression than the standard dyadic approach. However, up until now, the design of appropriate filters for the quincunx scheme has mainly been done using the McClellan transform. In our approach, we start from the scaling functions, which are the polyharmonic B-splines and, as such, explicitly known, and we derive a family of polyharmonic spline wavelets corresponding to different flavors of the semi-orthogonal wavelet transform; e.g., orthonormal, B-spline, and dual. The filters are automatically specified by the scaling relations satisfied by these functions. We prove that the isotropic polyharmonic B-spline wavelet converges to a combination of four Gabor atoms, which are well separated in the frequency domain. We also show that these wavelets are nearly isotropic and that they behave as an iterated Laplacian operator at low frequencies. We describe an efficient fast Fourier transform-based implementation of the discrete wavelet transform based on polyharmonic B-splines.

Motivated by the fractal-like behavior of natural images, we propose a new smoothing technique that uses a regularization functional which is a fractional iterate of the Laplacian. This type of functional has previously been introduced by Duchon in the context of radial basis functions (RBFs) for the approximation of non-uniform data. Here, we introduce a new solution to Duchon's smoothing problem in multiple dimensions using non-separable fractional polyharmonic B-splines. The smoothing is performed in the Fourier domain by filtering, thereby making the algorithm fast enough for most multi-dimensional real-time applications.

We propose the use of polyharmonic B-splines to build non-separable two-dimensional wavelet bases. The central idea is to base our design on isotropic-polyharmonic B-splines, a new type of polyharmonic B-splines that converge to a Gaussian as the order increases. We opt for the quincunx subsampling scheme which allows us to characterize the wavelet spaces with a single wavelet, the isotropic-polyharmonic B-spline wavelet. Interestingly, this wavelet converges to a combination of four Gabor atoms, which are well separated in the frequency domain. We also briefly discuss our Fourier-based implementation and present some experimental results.

Image-scrambling schemes are designed to render the image content unintelligible. Wyner has proposed an elegant one-dimensional (1-D) scrambling scheme without bandwidth expansion, making use of the discrete prolate spheroidal sequences (DPSS). The DPSS are optimal regarding their energy concentration in a given frequency subband. In this paper, we propose the two-dimensional (2-D) extension and application of this algorithm. We discuss new possibilities introduced by the 2-D approach. We also include experimental results.

This paper proposes a new family of bivariate, nonseparable splines, called hex-splines, especially designed for hexagonal lattices. The starting point of the construction is the indicator function of the Voronoi cell, which is used to define in a natural way the first-order hex-spline. Higher order hex-splines are obtained by successive convolutions. A mathematical analysis of this new bivariate spline family is presented. In particular, we derive a closed form for a hex-spline of arbitrary order. We also discuss important properties, such as their Fourier transform and the fact they form a Riesz basis. We also highlight the approximation order. For conventional rectangular lattices, hex-splines revert to classical separable tensor-product B-splines. Finally, some prototypical applications and experimental results demonstrate the usefulness of hex-splines for handling hexagonally sampled data.

Hex-splines are a novel family of bivariate splines which are well suited to handle hexagonally sampled data. Similar to classical 1D B-splines, the spline coefficients need to be computed by a prefilter. Unfortunately, the elegant implementation of this prefilter by causal and anti-causal recursive filtering is not applicable for the (non-separable) hex-splines. Therefore, in this paper we introduce a novel approach from the viewpoint of approximation theory. We propose three different recursive filters and optimize their parameters such that a desired order of approximation is obtained. The results for third and fourth order hex-splines are discussed. Although the proposed solutions provide only quasi-interpolation, they tend to be very close to the interpolation prefilter.

A new fuzzy filter is presented for the noise reduction of images corrupted with additive noise. The filter consists of two stages. The first stage computes a fuzzy derivative for eight different directions. The second stage uses these fuzzy derivatives to perform fuzzy smoothing by weighting the contributions of neighboring pixel values. Both stages are based on fuzzy rules which make use of membership functions. The filter can be applied iteratively to effectively reduce heavy noise. In particular, the shape of the membership functions is adapted according to the remaining noise level after each iteration, making use of the distribution of the homogeneity in the image. A statistical model for the noise distribution can be incorporated to relate the homogeneity to the adaptation scheme of the membership functions. Experimental results are obtained to show the feasibility of the proposed approach. These results are also compared to other filters by numerical measures and visual inspection.

Resampling techniques are commonly required in digital image processing systems. Many times the classical interpolation functions are used, i.e., nearest-neighbour interpolation and bilinear interpolation, which are prone to the introduction of undesirable artifacts due to aliasing such as moire patterns. This paper presents a novel approach which minimizes the loss of information, in a least-squares sense, while resampling between orthogonal and hexagonal lattices. Making use of an extension of 2D splines to hexagonal lattices, the proper reconstruction function is derived. Experimental results for a printing application demonstrate the feasibility of the proposed method and are compared against the classical techniques.

We describe the validation process of a functional metadata document encapsulated in an MPEG-21 digital item. The validation starts with an MPEG-21 digital item containing the functional metadata and ends with a valid functional metadata document or with a list of error codes. Since the W3C schema language does not suffice to validate all relationships within a functional metadata document, a set of extra rules is defined. These rules are implemented through the use of transformations on functional metadata documents. These transformations are implemented using XSLT.

The optimal one-dimensional (1-D) window, an index-limited sequence with maximum energy concentration in a finite frequency interval, is related to a particular discrete prolate spheroidal sequence. This letter presents the N-dimensional (N-D) extension, i.e., the N-D window with limited support and maximum energy concentration in a general nonseparable N-D passband. These windows can be applied to multidimensional filter design and the design of optimal convolution functions. We show the three-dimensional (3-D) optimal window based on a rhombic dodecahedron as a passband region.

In this paper we give an overview of classical and fuzzy-classical filters for image noise reduction. Together with our overview (2001) of fuzzy filters, this paper can be seen as a preparation to our comparative study of classical and fuzzy filters for image noise reduction.

In this paper we give an overview of existing fuzzy filters for image noise reduction. The paper is a sequel to our overview of classical and fuzzy-classical filters (2001), and can be seen as a preparation to our comparative study of classical and fuzzy filters for image noise reduction. We discuss the ideas behind and the construction of the following fuzzy filters: the "fuzzy inference ruled by else-action" filters, the "fuzzy control based" filters, and the GOA filter. Our goal is to give a consistent overview of these fuzzy filters, and to clearly show the mutual differences between them.

In this paper we present the results of a comparative study of classical and fuzzy filters for image noise reduction. The discussed fuzzy filters are classified, and their performance is compared with classical filters and evaluated by numerical and visual experiments.

The interlaced video scan format suffers from major flaws such as visual artifacts and unsuitability for devices such as LCD displays, video printers, computers which require or prefer the progressive scan format. De-interlacing algorithms convert a video signal from the interlaced scan format to the progressive scan format. Next to simple spatial interpolation, two categories of de-interlacing techniques are available: motion adaptive and motion compensating methods. The latter have more potential to produce better results but are also more complex since they require accurate subpixel motion estimation. This paper presents a motion compensating de-interlacing technique based on the Irani and Peleg (1991) superresolution algorithm. Appropriate weighting terms are introduced to take into account the interlaced scanning format. Two operational modes are derived: a single iteration mode for real time video de-interlacing, and a multiple iteration mode for enhanced still image generation.

Deinterlacing algorithms are used to convert an interlaced video sequence to the progressive scan format. Interlaced video exposes artifacts like line flicker and line crawling and is unsuitable for progressive media. Picture quality can be improved significantly when a proper deinterfacing algorithm is applied. The paper presents a motion adaptive technique based on a fuzzy motion detector. Preliminary experiments show results using this approach.