Spatial Transcriptomics (ST) technology detects gene expression from tissue biopsies, playing an emerging role in cancer diagnosis and precision medicine. However, the high cost of ST technology limits its broader application. Recently, deep learning approaches have provided insight into predicting gene expression based on H&E-stained histopathology images. Nevertheless, the relationship between morphological features and gene expression is highly complex. To address these challenges, we propose DiffBulk, a novel two-stage framework that leverages conditional diffusion models to learn expressive image representations enriched with gene expression information. In the first stage, we introduce a gene-to-image conditional diffusion model equipped with a permutation-invariant open-embedding gene encoder, which enables unified training across diverse gene panels. In the second stage, diffusion-derived features are fused with representations from a pathology foundation model, effectively bridging the domain gap and improving downstream gene expression prediction. We evaluate DiffBulk on high-quality Xenium ST data curated from the HEST dataset and the CrunchDAO challenge, constructing tile-level pseudo-bulk datasets for training and evaluation. Extensive experiments demonstrate that DiffBulk consistently outperforms state-of-the-art baselines across all metrics for gene expression prediction. These findings highlight the potential of diffusion-based gene-image representation learning and suggest promising directions for future research.

This article introduces a novel framework named double-latent optimization for representation disentanglement (D-LORD), which is designed for motion stylization (motion style transfer and motion retargeting). The primary objective of this framework is to separate the class and content information from a given motion sequence using a data-driven latent optimization approach. Here, class refers to person-specific style, such as a particular emotion or an individual’s identity, while content relates to the style-agnostic aspect of an action, such as walking or jumping, as universally understood concepts. The key advantage of D-LORD is its ability to perform style transfer without needing paired motion data. Instead, it utilizes class and content labels during the latent optimization process. By disentangling the representation, the framework enables the transformation of one motion sequence’s style to another’s style using adaptive instance normalization. The proposed D-LORD framework is designed with a focus on generalization, allowing it to handle different class and content labels for various applications. In addition, it can generate diverse motion sequences when specific class and content labels are provided. The framework’s efficacy is demonstrated through experimentation on three datasets: 1) the CMU XIA dataset for motion style transfer; 2) the multimodal human action database dataset; and 3) the RRIS Ability dataset for motion retargeting. Notably, this article presents the first generalized framework for motion style transfer and motion retargeting, showcasing its potential contributions in this area.

In-Context Learning (ICL) has been a powerful emergent property of large language models that has attracted increasing attention in recent years. In contrast to regular gradient-based learning, ICL is highly interpretable and does not require parameter updates. In this paper, we show that, for linearized transformer networks, ICL can be made explicit and permanent through the inclusion of bias terms. We mathematically demonstrate the equivalence between a model with ICL demonstration prompts and the same model with the additional bias terms. Our algorithm (ICLCA) allows for exact conversion in an inexpensive manner. Existing methods are not exact and require expensive parameter updates. We demonstrate the efficacy of our approach through experiments that show the exact incorporation of ICL tokens into a linear transformer. We further suggest how our method can be adapted to achieve cheap approximate conversion of ICL tokens, even in regular transformer networks that are not linearized. Our experiments on GPT-2 show that, even though the conversion is only approximate, the model still gains valuable context from the included bias terms.

Histopathology is a crucial diagnostic tool in cancer and involves the analysis of gigapixel slides. Multiple instance learning (MIL) promises success in digital histopathology thanks to its ability to handle gigapixel slides and work with weak labels. MIL is a machine learning paradigm that learns the mapping between bags of instances and bag labels. It represents a slide as a bag of patches and uses the slide's weak label as the bag's label. This paper introduces distribution-based pooling filters that obtain a bag-level representation by estimating marginal distributions of instance features. We formally prove that the distribution-based pooling filters are more expressive than the classical point estimate-based counterparts, like 'max' and 'mean' pooling, in terms of the amount of information captured while obtaining bag-level representations. Moreover, we empirically show that models with distribution-based pooling filters perform equal to or better than those with point estimate-based pooling filters on distinct real-world MIL tasks defined on the CAMELYON16 lymph node metastases dataset. Our model with a distribution pooling filter achieves an area under the receiver operating characteristics curve value of 0.9325 (95% confidence interval: 0.8798 - 0.9743) in the tumor vs. normal slide classification task.

Many machine learning adversarial attacks find adversarial samples of a victim model M by following the gradient of some attack objective functions, either explicitly or implicitly. To confuse and detect such attacks, we take the proactive approach that modifies those functions with the goal of misleading the attacks to some local minima, or to some designated regions that can be easily picked up by an analyzer. To achieve this goal, we propose adding a large number of artifacts, which we called attractors, onto the otherwise smooth function. An attractor is a point in the input space, where samples in its neighborhood have gradient pointing toward it. We observe that decoders of watermarking schemes exhibit properties of attractors and give a generic method that injects attractors from a watermark decoder into the victim model M. This principled approach allows us to leverage on known watermarking schemes for scalability and robustness and provides explainability of the outcomes. Experimental studies show that our method has competitive performance. For instance, for un-targeted attacks on CIFAR-10 dataset, we can reduce the overall attack success rate of DeepFool to 1.9%, whereas known defense LID, FS and MagNet can reduce the rate to 90.8%, 98.5% and 78.5% respectively.

BACKGROUND: High resolution 2D whole slide imaging provides rich information about the tissue structure. This information can be a lot richer if these 2D images can be stacked into a 3D tissue volume. A 3D analysis, however, requires accurate reconstruction of the tissue volume from the 2D image stack. This task is not trivial due to the distortions such as tissue tearing, folding and missing at each slide. Performing registration for the whole tissue slices may be adversely affected by distorted tissue regions. Consequently, regional registration is found to be more effective. In this paper, we propose a new approach to an accurate and robust registration of regions of interest for whole slide images. We introduce the idea of multi-scale attention for registration. RESULTS: Using mean similarity index as the metric, the proposed algorithm (mean ± SD [Formula: see text]) followed by a fine registration algorithm ([Formula: see text]) outperformed the state-of-the-art linear whole tissue registration algorithm ([Formula: see text]) and the regional version of this algorithm ([Formula: see text]). The proposed algorithm also outperforms the state-of-the-art nonlinear registration algorithm (original: [Formula: see text], regional: [Formula: see text]) for whole slide images and a recently proposed patch-based registration algorithm (patch size 256: [Formula: see text] , patch size 512: [Formula: see text]) for medical images. CONCLUSION: Using multi-scale attention mechanism leads to a more robust and accurate solution to the problem of regional registration of whole slide images corrupted in some parts by major histological artifacts in the imaged tissue.

Anomaly detection is a classical problem where the aim is to detect anomalous data that do not belong to the normal data distribution. Current state-of-the-art methods for anomaly detection on complex high-dimensional data are based on the generative adversarial network (GAN). However, the traditional GAN loss is not directly aligned with the anomaly detection objective: it encourages the distribution of the generated samples to overlap with the real data and so the resulting discriminator is ineffective as an anomaly detector. In this paper, we propose modifications to the GAN loss such that the generated samples lie at the boundaries of the real data distribution. With our modified GAN loss, our anomaly detection method, called Fence GAN (FGAN), directly uses the discriminator score as an anomaly threshold. Our experimental results on the MNIST, CIFAR10 and KDD99 datasets show that FGAN yields the best anomaly classification accuracy compared to state-of-the-art methods.

BACKGROUND: Drosophila melanogaster is an important organism used in many fields of biological research such as genetics and developmental biology. Drosophila wings have been widely used to study the genetics of development, morphometrics and evolution. Therefore there is much interest in quantifying wing structures of Drosophila. Advancement in technology has increased the ease in which images of Drosophila can be acquired. However such studies have been limited by the slow and tedious process of acquiring phenotypic data. RESULTS: We have developed a system that automatically detects and measures key points and vein segments on a Drosophila wing. Key points are detected by performing image transformations and template matching on Drosophila wing images while vein segments are detected using an Active Contour algorithm. The accuracy of our key point detection was compared against key point annotations of users. We also performed key point detection using different training data sets of Drosophila wing images. We compared our software with an existing automated image analysis system for Drosophila wings and showed that our system performs better than the state of the art. Vein segments were manually measured and compared against the measurements obtained from our system. CONCLUSION: Our system was able to detect specific key points and vein segments from Drosophila wing images with high accuracy.

Computer aided diagnosis (CAD) in medical imaging is of growing interest in recent years. Our proposed CAD system aims to enhance diagnosis and prognosis of traumatic brain injury (TBI) patients with hematomas. Hematoma caused by blood vessel rupture is the major lesion in TBI cases and is usually assessed using head computed tomography (CT). In our CAD system, we segment the hematoma region from each slice of a CT series, extract features from the hematoma segments, and automatically classify the hematoma types using machine learning methods. We propose two sets of shape based features for each segmented hematoma region. The first set contains primitive features describing the overall shape of a hematoma region. The features in the second set are based on the dissimilarities of the shapes of hematoma regions measured by geodesic distances. After feature extraction, we classify the hematoma regions into three types -- epidural hematoma, sub-dural hematoma, and intracerebral hematoma, using random forest. Each tree of the random forest votes one class for each hematoma, and the random forest takes the class label with the majority votes for the hematoma. As hematomas are volumetric in nature, some hematomas are observed across several consecutive slices in the same CT series. For each class, we add the votes from each hematoma slice that comprises the volumetric hematoma in that class, then we take the class with the majority of the summed votes as the class label for that volumetric hematoma. The overall classification accuracies for hematoma region from each CT slice are 80.7%, 81.3%, and 81.1% using primitive features only, geodesic distance features only, or both sets of features, respectively. For volumetric hematoma classification, the overall accuracies are 80.9%, 81.5%, and 81.5% respectively. The results are promising to radiologists and neurosurgeons specialized in this field of research.

In a recent paper, Krinidis and Chatzis proposed a variation of fuzzy c-means algorithm for image clustering. The local spatial and gray-level information are incorporated in a fuzzy way through an energy function. The local minimizers of the designed energy function to obtain the fuzzy membership of each pixel and cluster centers are proposed. In this paper, it is shown that the local minimizers of Krinidis and Chatzis to obtain the fuzzy membership and the cluster centers in an iterative manner are not exclusively solutions for true local minimizers of their designed energy function. Thus, the local minimizers of Krinidis and Chatzis do not converge to the correct local minima of the designed energy function not because of tackling to the local minima, but because of the design of energy function.

Neural stem cells and neural progenitors are early nervous system cells that form neurospheres when propagated in vitro. We study changes in growth using brightfield images to understand the effects of drugs. The image quality is generally poor, imposing challenges for automatic analysis. Level-set segmentation methods are able to handle topology changes but require close initializations for accurate and efficient results. Global level-set methods using single image-wide optimization objective functions are difficult to cope with large illumination and shading changes. We propose to adopt Hough transform to initialize localized level-sets for cell segmentation. Experimental results on 480 images with 738 neurospheres show that our proposed method performs best over existing level-set methods without appropriate initial contours.

In this paper, we consider the problem of segmentation of large collections of images. We propose a semisupervised optimization model that determines an efficient segmentation of many input images. The advantages of the model are twofold. First, the segmentation is highly controllable by the user so that the user can easily specify what he/she wants. This is done by allowing the user to provide, either offline or interactively, some (fully or partially) labeled pixels in images as strong priors for the model. Second, the model requires only minimal tuning of model parameters during the initial stage. Once initial tuning is done, the setup can be used to automatically segment a large collection of images that are distinct but share similar features. We will show the mathematical properties of the model such as existence and uniqueness of solution and establish a maximum/minimum principle for the solution of the model. Extensive experiments on various collections of biological images suggest that the proposed model is effective for segmentation and is computationally efficient.

We propose a variant of the Mumford-Shah model for the segmentation of a pair of overlapping objects with additive intensity value. Unlike standard segmentation models, it does not only determine distinct objects in the image, but also recover the possibly multiple membership of the pixels. To accomplish this, some a priori knowledge about the smoothness of the object boundary is integrated into the model. Additivity is imposed through a soft constraint which allows the user to control the degree of additivity and is more robust than the hard constraint. We also show analytically that the additivity parameter can be chosen to achieve some stability conditions. To solve the optimization problem involving geometric quantities efficiently, we apply a multiphase level set method. Segmentation results on synthetic and real images validate the good performance of our model, and demonstrate the model's applicability to images with multiple channels and multiple objects.

BACKGROUND: Computational comparison of two protein structures is the starting point of many methods that build on existing knowledge, such as structure modeling (including modeling of protein complexes and conformational changes), molecular replacement, or annotation by structural similarity. In a commonly used strategy, significant effort is invested in matching two sets of atoms. In a complementary approach, a global descriptor is assigned to the overall structure, thus losing track of the substructures within. RESULTS: Using a small set of geometric features, we define a reduced representation of protein structure, together with an optimizing function for matching two representations, to provide a pre-filtering stage in a database search. We show that, in a straightforward implementation, the representation performs well in terms of resolution in the space of protein structures, and its ability to make new predictions. CONCLUSIONS: Perhaps unexpectedly, a substantial discriminating power already exists at the level of main features of protein structure, such as directions of secondary structural elements, possibly constrained by their sequential order. This can be used toward efficient comparison of protein (sub)structures, allowing for various degrees of conformational flexibility within the compared pair, which in turn can be used for modeling by homology of protein structure and dynamics.

The Mumford-Shah model is one of the most successful image segmentation models. However, existing algorithms for the model are often very sensitive to the choice of the initial guess. To make use of the model effectively, it is essential to develop an algorithm which can compute a global or near global optimal solution efficiently. While gradient descent based methods are well-known to find a local minimum only, even many stochastic methods do not provide a practical solution to this problem either. In this paper, we consider the computation of a global minimum of the multiphase piecewise constant Mumford-Shah model. We propose a hybrid approach which combines gradient based and stochastic optimization methods to resolve the problem of sensitivity to the initial guess. At the heart of our algorithm is a well-designed basin hopping scheme which uses global updates to escape from local traps in a way that is much more effective than standard stochastic methods. In our experiments, a very high-quality solution is obtained within a few stochastic hops whereas the solutions obtained with simulated annealing are incomparable even after thousands of steps. We also propose a multiresolution approach to reduce the computational cost and enhance the search for a global minimum. Furthermore, we derived a simple but useful theoretical result relating solutions at different spatial resolutions.