Cryo-electron microscopy provides 2D projection images of the 3D electron scattering intensity of many instances of the particle under study (e.g., a virus). Both symmetry (rotational point groups) and heterogeneity are important aspects of biological particles and both aspects can be combined by describing the electron scattering intensity of the particle as a stochastic process with a symmetric probability law and, therefore, symmetric moments. A maximum likelihood estimator implemented by an expectation-maximization algorithm is described, which estimates the unknown statistics of the electron scattering intensity stochastic process from the images of instances of the particle. The algorithm is demonstrated on the bacteriophage HK97 and the virus [Formula: see text]. The results are contrasted with the existing algorithms, which assume that each instance of the particle has the symmetry rather than the less restrictive assumption that the probability law has the symmetry.

Compositionality of semantic concepts in image synthesis and analysis is appealing as it can help in decomposing known and generatively recomposing unknown data. For instance, we may learn concepts of changing illumination, geometry or albedo of a scene, and try to recombine them to generate physically meaningful, but unseen data for training and testing. In practice however we often do not have samples from the joint concept space available: We may have data on illumination change in one data set and on geometric change in another one without complete overlap. We pose the following question: How can we learn two or more concepts jointly from different data sets with mutual consistency where we do not have samples from the full joint space? We present a novel answer in this paper based on cyclic consistency over multiple concepts, represented individually by generative adversarial networks (GANs). Our method, ConceptGAN, can be understood as a drop in for data augmentation to improve resilience for real world applications. Qualitative and quantitative evaluations demonstrate its efficacy in generating semantically meaningful images, as well as one shot face verification as an example application.

Cryo electron microscopy records essentially projection images of each of many instances of a nano bio object. This data allows reconstruction of a stochastic model of the object, i.e., the mean and covariance functions of the electron scattering intensity of the object. Understanding the covariance function, which characterizes the heterogeneity of the instances of the object, is challenging because the covariance function is not wide-sense stationary but instead depends separately on the two three-dimensional positions. This paper describes a method, qualitatively motivated by fluctuation-dissipation theory, for estimating a spring-and-mass mechanical model of the object. Then, from the model, a wide range of properties of the object can be understood, such as normal modes of vibration.

Various modes of transmission electron microscopy can provide unique structural information on nanometre‐scale biological machines such as viruses and ribosomes. A maximum‐likelihood statistical reconstruction algorithm using information from two such modalities, single‐particle cryo‐electron microscopy and computed electron tomography, is described for the problem where the machine is nearly symmetrical, but the localised regions of asymmetry are important for the functioning of the machine. The algorithm is demonstrated on experimental bacteriophage Lambda procapsid data. Key features of the algorithm are the use of probability density functions (pdfs) derived from normalised correlation rather than Gaussian pdfs and the solution of a constrained classification problem in which 11 out of 12 sites of asymmetry belong to one class while the 12th site belongs to a second class.

Cryo electron microscopy leads to 3-D image reconstruction problems. The data is one projection of each of many different instances of the object. When the object is heterogeneous, it is natural to describe the object as stochastic with unknown statistics and seek to estimate the statistics from the image data. The result is not a unique reconstruction but rather a statistical description of an ensemble of reconstructions where the mean of the ensemble is a natural choice if a unique reconstruction is desired. This paper describes maximum likelihood methods for estimating the statistics via an expectation-maximization algorithm and multiple algorithmic optimizations to speed up the calculations.

Cryo electron microscopy leads to 3-D image reconstruction problems. The image is one projection of each of many different instances of the object. Symmetry of the object is important because then the image shows the motif in several projections with known relative orientations even though the projections of the different motifs overlap. This paper derives image formation models for helical objects based on arbitrary motifs and where the object may, in fact, not be helical but just have a rotational and translational shift between successive motifs. This image formation model potentially has computational advantages when the number of motif repetitions per helical period is large and also allows the description of imperfect helices by stochastic variation in the shifts, which may be useful in stochastic reconstruction problems for image data sets that cannot currently be successfully processed.

Cryo electron microscopy leads to 3-D image reconstruction problems. The data is one projection of each of many different instances of the object. When the object has symmetry and is also heterogeneous, it is natural to describe the object as stochastic with symmetrical statistics. This paper describes methods for incorporating the symmetry into expectation-maximization algorithms for the maximum likelihood solution of the reconstruction problem.

Resting-state functional MRI (rs fMRI) is widely used to non-invasively study human brain networks. Network functional connectivity is estimated by calculating the standard correlation between blood-oxygen-level dependent (BOLD) signals in specific regions of interests (ROIs). However, standard correlation fails to characterize the causality and the direction of information flow between regions, which are important factors in characterizing a network. Here, we use causal linear time-invariant models, with the impulse response duration estimated by Information Criteria, to describe the effective connectivity between ROIs. To do so, we replace the standard correlation between BOLD signals with a correlation between a BOLD signal and a prediction via the model of that BOLD signal. Prediction correlation is then used in a network analysis similar to that used with standard correlation. Our results include the causality information, the direction of information flow, and the possibility of delays in information flow. This approach replicates the local and distributed network architecture of the human brain previously observed with standard correlations, as well as providing novel insight into the directed interactivity of regions comprising these networks.

Many micro- to nano-scale 3-D biological objects have a helical symmetry. Cryo electron microscopy provides 2-D projection images where, however, the images have low SNR and unknown projection directions. The object is described as a helical array of identical motifs, where both the parameters of the helical symmetry and the motif are unknown. Using a detailed image formation model, a maximum-likelihood estimator for the parameters of the symmetry and the 3-D motif based on images of many objects and algorithms for computing the estimate are described. The possibility that the objects are not identical but rather come from a small set of homogeneous classes is included. The first example is based on 316 128 × 100 pixel experimental images of Tobacco Mosaic Virus, has one class, and achieves 12.40-Å spatial resolution in the reconstruction. The second example is based on 400 128 × 128 pixel synthetic images of helical objects constructed from NaK ion channel pore macromolecular complexes, has two classes differing in helical symmetry, and achieves 7.84- and 7.90-Å spatial resolution in the reconstructions for the two classes.

A statistical approach for computing 3-D reconstructions of virus particles from cryo electron microscope images and minimal prior information has been developed which can solve a range of specific problems. The statistical approach causes high computation and storage complexity in the software implementation. A parallel software toolkit is described which allows the construction of software targeted at commodity PC clusters which is modular, reusable, and user-transparent while at the same time delivering nearly linear speedup on practical problems.

In cryo-electron microscopy, the data is comprised of noisy 2-D projection images of the 3-D electron scattering intensity of the object where the orientation of the projections is unknown. Often, the images show randomly selected objects from a mixture of different types of objects. Objects of different type may be unrelated, e.g., different species of virus, or related, e.g., different conformations of the same species of virus. Due to the low SNR and the 2-D nature of the data, it is challenging to determine the type of the object shown in an individual image. A statistical model and maximum likelihood estimator that computes simultaneous 3-D reconstruction and labels using an expectation maximization algorithm exists but requires extensive computation due to the numerical evaluation of 3-D or 5-D integrations of a square matrix of dimension equal to the number of degrees of freedom in the 3-D reconstruction. By exploiting the geometry of rotations in 3-D, the estimation problem can be transformed so that the inner-most numerical integral has a scalar rather than a matrix integrand. This leads to a dramatic reduction in computation, especially as the number of degrees of freedom in the 3-D reconstruction increases. Numerical examples of the 3-D reconstructions are provided based on synthetic and experimental images where the objects are small spherical viruses.

A 3-D reconstruction problem for virus particles based on cryo electron microscope images is described in which a spatially localized deviation from total-particle symmetry is present. In particular, we describe a statistical algorithm for the reconstruction of tailed bacteriophages where the symmetries of the capsid and tail are incompatible and therefore the tail is a deviation from symmetry. For reasons of computational complexity and reduced storage, we propose a two-step approach to reconstruction. Mathematical models and numerical results computed from synthetic data are presented.

The 3D reconstruction of the electron scattering intensity of a virus from cryo electron microscopy is essentially a 3D tomography problem in which the orientation of the 2D projections is unknown. Many biological problems concern mixtures of different types of virus particles or mixtures of different maturation states of the same type of virus particle. For a variety of reasons, especially low SNR, it can be very challenging to label the type or state shown in an individual image. Algorithms capable of computing multiple reconstructions, one for each type or state, based on images which are not labeled according to type or state, are described and demonstrated on experimental images.

A statistical method for computing 3-D reconstructions of virus particles from cryo electron microscope images and minimal prior information-particle symmetry and radii-is described and demonstrated numerically focusing on the robustness of the approach.

A statistical method for computing 3-D reconstructions of virus particles from cryo electron microscope images and minimal prior information - particle symmetry and radii - is described. Two different approaches for parallel implementation of the algorithm are described based on parallel processing of different images and based on parallel numerical integration and numerical results showing nearly linear speedup are presented.

A recursive restoration algorithm based on a Markov random field model driven by Cauchy noise is described and demonstrated to provide better edge preservation than similar algorithms using Gaussian or Laplacian noises without increased computational cost.

A statistical model for the object and the complete image formation process in the cryo electron microscopy of viruses is presented. Using this model, maximum-likelihood reconstructions of the three-dimensional (3-D) structure of viruses are computed using the expectation maximization algorithm, and alternative experimental designs are evaluated based on Cramer-Rao bounds. Numerical examples of the reconstructions and experimental design evaluations are provided based on Cowpea mosaic virus.

An interpolation problem that is important in viral X-ray crystallography is considered. The problem requires new methods because (1) the function is known to have icosahedral symmetry, (2) the data is corrupted by experimental errors and therefore lacks the symmetry, (3) the problem is 3D, (4) the measurements are irregularly spaced, and (5) the number of measurements is large (10**4). A least-squares approach is taken using two sets of basis functions: the functions implied by a minimum-energy band-limited exact interpolation problem and a complete orthonormal set of band-limited functions. A numerical example on Cowpea Mosaic virus is described.

Cryo electron microscopy of viruses provides 2D projections of the scattering intensity of the viral particle but the orientation of the projections is not known. We describe an approach to reconstructing the 3D scattering intensity in spite of the unknown projection orientations using nonlinear least squares ideas where the reconstruction is guaranteed to have the icosahedral symmetry known to be present in the viral particle.

An object model and estimation procedure for three-dimensional (3-D) reconstruction of objects from measurements of the spherically averaged Fourier transform magnitudes is described. The motivating application is the 3-D reconstruction of viruses based on solution X-ray scattering data. The object model includes symmetry, positivity and support constraints and has the form of a truncated orthonormal expansion and the parameters are estimated by maximum likelihood methods. Successful 3-D reconstructions based on synthetic and experimental measurements from Cowpea mosaic virus are described.

We compare two approaches, DESA-1 based on Teager's (1980) energy operator and MBDA based on statistical nonlinear filtering, for extracting the modulating signals from jointly amplitude (AM) and frequency (FM) modulated waveforms. Both demonstrate the structure within a pitch period of a speech signal. The DESA-1 estimates tend to be smoother while the MBDA estimates tend to perform better in the presence of noise.

Accurate and wide-area estimates of vehicle velocity and traffic spatial and temporal densities will be essential components of future algorithms for freeway and arterial street control, for incident prediction and detection, and for optimization in route selection. Algorithms like these figure prominently in the current research and development of intelligent vehicle-highway systems (IVHS). This paper presents an approach to a class of vehicle monitoring problems which is based upon a video backbone sensor and multiple target tracking (MTT). The method allows the integration of measurements made from other sensors like inductive loops, microwave radars, and laser range profilers.

We describe a new approach to decomposing signals that are modeled as a sum of jointly amplitude and frequency modulated cosines with slowing-varying center frequencies observed in noise based on statistical nonlinear filtering ideas. We demonstrate the ideas on a formant tracking problem for the sentence "Where were you while we were away".

We describe a Bayesian restoration and segmentation algorithm based on a pixel-line Markov random field and using an efficient approximation based on locality of interactions. A medical tomography example is given.

A model-based method for reconstructing the 3-dimensional structure of icosahedrally-symmetric viruses from solution X-ray scattering is presented. The algorithm is an iterative algorithm that is a generalization of the Gerchberg-Saxton algorithm from 3 dimensions (measure the magnitude squared of an unknown complex number) to n dimensions (measure the norm squared of an unknown n-vector). An example of the reconstruction, for data from cowpea mosaic virus, is described. The major opportunity provided by solution X-ray scattering is the ability to study the dynamics of virus particles in solution, information that is not accessible to crystal X-ray diffraction experiments.

We describe a family of approximations, denoted by "cluster approximations", for the computation of the mean of a Markov random field (MRF). This is a key computation in image processing when applied to the a posteriori MRF. The approximation is to account exactly for only spatially local interactions. Application of the approximation requires the solution of a nonlinear multivariable fixed-point equation for which we prove several existence, uniqueness, and convergence-of-algorithm results. Four numerical examples are presented, including comparison with Monte Carlo calculations.>

Methods for approximately computing the marginal probability mass functions and means of a Markov random field (MRF) by approximating the lattice by a tree are described. Applied to the a posteriori MRF these methods solve Bayesian spatial pattern classification and image restoration problems. The methods are described, several theoretical results concerning fixed-point problems are proven, and four numerical examples are presented, including comparison with optimal estimators and the iterated conditional mode estimator and including two agricultural optical remote sensing problems.>

X-ray crystallography is the dominant method for measuring the relative positions in three-dimensional space of each atom in a molecule. This information has long been crucial for many scientific purposes and is increasingly important for the rational design of drugs. The X-ray scattering is the Fourier transform of the electron density. However, only the intensity and not the phase of the scattering can be measured. Therefore, the signal processing is fundamentally phase retrieval. The situation is complicated, however, by the three-dimensional nature of the problem and the periodicity of the crystal. In this paper the underlying physics is sketched, a variety of crystallographic experiments and the resulting types of multidimensional signal processing are described, and the application to drug design is outlined.>

Describes a family of approximations, denoted "Bethe tree approximations", for the computation of the marginal probability mass functions (pmfs) of a Markov random field (MRF). This is a key computation in spatial pattern classification when applied to the a posteriori MRF. The approximation modifies the graph on which the MRF is defined: the original lattice is modified into a tree. Then the marginal pmfs on the tree can be computed exactly by fast recursive algorithms. A key issue is how to terminate the tree at its leaves and 4 solutions are explored of which 3 result in the solution of nonlinear multivariable fixed-point equations for which some existence, uniqueness, and convergence-of-algorithm results can be proven. The algorithm has given excellent performance on a variety of segmentation problems (1994) and a 9-class agricultural remote-sensing example is described.>

Describes a family of approximations, denoted "cluster approximations", for the computation of the mean of a Markov random field (MRF). This is a key computation in image processing when applied to the a posteriori MRF. The approximation is a form of mean field theory and accounts exactly for only spatially local interactions. The implementation of the approximation requires the solution of a nonlinear multivariable fixed-point equation. Unlike many forms of mean field theory, the approximation is easy to apply even to nonquadratic Hamiltonians (e.g., it requires no analytical calculations), the structure of the gray level values in the original problem is retained, the resulting approximate mean field can be proven to lie in the same set which contains the true mean of the MRF, and there are existence, uniqueness, and convergence-of-algorithm results for the fixed-point equation. Two numerical examples are presented which emphasize nonlinear observation processes.>

A Bayesian binary signal reconstruction problem which includes noisy magnitude of Fourier transform measurements and a Markov random field a priori model is solved. The solution is analytical and is based on the spherical model and small noise asymptotic approximations. Parameters in the solution are used for data adaptation. The problem is motivated by the phase retrieval problem in X-ray crystallography.>