MOTIVATION: Generative Adversarial Nets (GAN) achieve impressive performance for text-guided editing of natural images. However, a comparable utility of GAN remains understudied for spatial transcriptomics (ST) technologies with matched gene expression and biomedical image data. RESULTS: We propose In Silico Spatial Transcriptomic editing that enables gene expression-guided editing of immunofluorescence images. Using cell-level spatial transcriptomics data extracted from normal and tumor tissue slides, we train the approach under the framework of GAN (Inversion). To simulate cellular state transitions, we then feed edited gene expression levels to trained models. Compared to normal cellular images (ground truth), we successfully model the transition from tumor to normal tissue samples, as measured with quantifiable and interpretable cellular features. AVAILABILITY AND IMPLEMENTATION: https://github.com/CTPLab/SST-editing.

Generative modeling over natural images is one of the most fundamental machine learning problems. However, few modern generative models, including Wasserstein Generative Adversarial Nets (WGANs), are studied on manifold-valued images that are frequently encountered in real-world applications. To fill the gap, this paper first formulates the problem of generating manifold-valued images and exploits three typical instances: hue-saturation-value (HSV) color image generation, chromaticity-brightness (CB) color image generation, and diffusion-tensor (DT) image generation. For the proposed generative modeling problem, we then introduce a theorem of optimal transport to derive a new Wasserstein distance of data distributions on complete manifolds, enabling us to achieve a tractable objective under the WGAN framework. In addition, we recommend three benchmark datasets that are CIFAR-10 HSV/CB color images, ImageNet HSV/CB color images, UCL DT image datasets. On the three datasets, we experimentally demonstrate the proposed manifold-aware WGAN model can generate more plausible manifold-valued images than its competitors.

In generative modeling, the Wasserstein distance (WD) has emerged as a useful metric to measure the discrepancy between generated and real data distributions. Unfortunately, it is challenging to approximate the WD of high-dimensional distributions. In contrast, the sliced Wasserstein distance (SWD) factorizes high-dimensional distributions into their multiple one-dimensional marginal distributions and is thus easier to approximate. In this paper, we introduce novel approximations of the primal and dual SWD. Instead of using a large number of random projections, as it is done by conventional SWD approximation methods, we propose to approximate SWDs with a small number of parameterized orthogonal projections in an end-to-end deep learning fashion. As concrete applications of our SWD approximations, we design two types of differentiable SWD blocks to equip modern generative frameworks---Auto-Encoders (AE) and Generative Adversarial Networks (GAN). In the experiments, we not only show the superiority of the proposed generative models on standard image synthesis benchmarks, but also demonstrate the state-of-the-art performance on challenging high resolution image and video generation in an unsupervised manner.

Learning representations on Grassmann manifolds is popular in quite a few visual recognition tasks. In order to enable deep learning on Grassmann manifolds, this paper proposes a deep network architecture by generalizing the Euclidean network paradigm to Grassmann manifolds. In particular, we design full rank mapping layers to transform input Grassmannian data to more desirable ones, exploit re-orthonormalization layers to normalize the resulting matrices, study projection pooling layers to reduce the model complexity in the Grassmannian context, and devise projection mapping layers to respect Grassmannian geometry and meanwhile achieve Euclidean forms for regular output layers. To train the Grassmann networks, we exploit a stochastic gradient descent setting on manifolds of the connection weights, and study a matrix generalization of backpropagation to update the structured data. The evaluations on three visual recognition tasks show that our Grassmann networks have clear advantages over existing Grassmann learning methods, and achieve results comparable with state-of-the-art approaches.

Color demosaicing is a key image processing step aiming to reconstruct the missing pixels from a recorded raw image. On the one hand, numerous interpolation methods focusing on spatial-spectral correlations have been proved very efficient, whereas they yield a poor image quality and strong visible artifacts. On the other hand, optimization strategies, such as learned simultaneous sparse coding and sparsity and adaptive principal component analysis-based algorithms, were shown to greatly improve image quality compared with that delivered by interpolation methods, but unfortunately are computationally heavy. In this paper, we propose efficient regression priors as a novel, fast post-processing algorithm that learns the regression priors offline from training data. We also propose an independent efficient demosaicing algorithm based on directional difference regression, and introduce its enhanced version based on fused regression. We achieve an image quality comparable to that of the state-of-the-art methods for three benchmarks, while being order(s) of magnitude faster.

Color demosaicing is a process of reconstructing lost pixels in an incomplete color image. By extracting spatial-spectral correlations of RGB channels various interpolation methods have been proposed with low computational complexity. Meanwhile, optimization strategies such as sparsity and adaptive PCA based algorithm (SAPCA) were developed. SAPCA outperforms many interpolation techniques by impressive margins at the cost of dramatically increasing the computational time. In this paper we propose an efficient novel post-processing algorithm based on the adjusted anchored neighborhood regression (A+) method from image super-resolution literature. We greatly improve the results of the demosaicing methods, and achieve image quality as competitive as SAPCA but orders of magnitude faster.

The collaborative representation-based classifier (CRC) is proposed as an alternative to the sparse representation based classifier (SRC) for image face recognition. CRC solves an l2-regularized least squares formulation, with algebraic solution, while SRC optimizes over an I1-regularized least squares problem. As an extension of CRC, the weighted collaborative representation-based classifier (WCRC) is further proposed. The weights in WCRC are picked intuitively, it remains unclear why such choice of weights works and how we optimize those weights. In this paper, we propose a learned collaborative representation based classifier (LCRC) and attempt to answer the above questions. Our learning technique is based on the fixed point theorem and we use a weights formulation similar to WCRC as the starting point. Through extensive experiments on face datasets we show that the learning procedure is stable and convergent, and that LCRC is able to improve in performance over CRC and WCRC, while keeping the same computational efficiency at test.

With the advancement of peer-to-peer technology, media streaming applications become more and more popular in the Internet. However, the traditional development methods for this kind of applications need developers not only to consider the application logic but also to manage the dynamics of Internet resources, thus increasing the difficulty of development and limiting the deployment of personal video distribution applications. In this paper, we design and implement a peer-to-peer streaming system in a much easier way. In this way we can concentrate on the application itself without distraction from the dynamics of Internet resources. Such simplification owes to the Internet-based Virtual Computing Environment (iVCE), which provides programming abstractions and runtime utilities that can encapsulate the complexity of managing transient resources into the platform, thus facilitating the construction of Internet applications. When we build our streaming application based on the iVCE, we only need to define the interaction protocols among distributed nodes with the Owlet programming language. Also, we implement a JavaBean, which can be used by the Owlet program, to assist the transferring and rendering of the content. Our implementation shows that peer-to-peer applications such as media streaming, can be elegantly built using the iVCE platform, and it can serve as a reference implementation for developing similar applications.

Peer-to-peer media streaming has been an important service on the internet in recent years. The Data-driven (or mesh-based) structure is adopted by most working systems,in which data scheduling is one of the important problems.However, those frequently used scheduling algorithms are often faced with such a case: A neighbor peer takes up its bandwidth to deliver the packets that other neighbors can also supply, but some packets only held by it are not delivered.These packets can not be delivered in the current scheduling cycle, even though that the other neighbors have surplus bandwidth. This is a kind of waste of bandwidth and decreases the throughput of transmission. In this paper we propose anew scheduling algorithm aiming at the optimal throughput:Bipartite-matching based Block Scheduling algorithm(BBS).We convert the original data scheduling problem to a problem of finding a maximum match on the correspond bipartite graph, then assign data packets to neighbors according to the maximum match. We evaluate the performance of BBS with extensive experiments and the results show that BBS throughput and provides better streaming quality than those frequently used scheduling algorithms.