MOTIVATION: High-throughput sequencing (HTS) is a modern sequencing technology used to profile microbiomes by sequencing thousands of short genomic fragments from the microorganisms within a given sample. This technology presents a unique opportunity for artificial intelligence to comprehend the underlying functional relationships of microbial communities. However, due to the unstructured nature of HTS data, nearly all computational models are limited to processing DNA sequences individually. This limitation causes them to miss out on key interactions between microorganisms, significantly hindering our understanding of how these interactions influence the microbial communities as a whole. Furthermore, most computational methods rely on post-processing of samples which could inadvertently introduce unintentional protocol-specific bias. RESULTS: Addressing these concerns, we present SetBERT, a robust pre-training methodology for creating generalized deep learning models for processing HTS data to produce contextualized embeddings and be fine-tuned for downstream tasks with explainable predictions. By leveraging sequence interactions, we show that SetBERT significantly outperforms other models in taxonomic classification with genus-level classification accuracy of 95%. Furthermore, we demonstrate that SetBERT is able to accurately explain its predictions autonomously by confirming the biological-relevance of taxa identified by the model. AVAILABILITY AND IMPLEMENTATION: All source code is available at https://github.com/DLii-Research/setbert. SetBERT may be used through the q2-deepdna QIIME 2 plugin whose source code is available at https://github.com/DLii-Research/q2-deepdna.

Transformers, although first designed for sequence processing, can also handle unordered sets like point cloud data. Additionally, contrastive pretraining has emerged as a successful technique in image processing but remains unexplored for point cloud data. We develop and integrate a new point cloud pretraining technique inspired by the Simple Framework for Contrastive Learning (SimCLR) into the Set Transformer (ST) and Point Cloud Transformer (PCT) architectures and explore model performance using a novel 3D body scan dataset and the canonical datasets ShapeNet and ModelNet. For the 3D body scan dataset, this integration boosts initial training performance and maintains overall higher performance for classification tasks, and demonstrates better stability/convergence for regression tasks in comparison to non-pretrained (Naïve] counterparts. Furthermore, experiments examining strong generalization (relative performance on previously unseen classes) show improvement for pretrained models compared to Naïve models. Consistent benefits across tasks and data sets are observed based on additional experiments performed on the ShapeNet core dataset. Overall, we show how contrastive pretraining for point cloud data is a viable strategy for improving the performance of Transformers on downstream tasks and accelerating the training process.

SARS-CoV-2 is a novel virus that crossed over into humans in 2019 and declared a pandemic in early 2020. To understand how the virus infects a new host, we need to understand the mechanistic functions involved with the binding process. To address this need, we generate homology models of SARS-CoV-2 spikes as monomer and trimer to determine the feasibility of reduced computational requirements by using monomer structures. We further generate homology models of the conserved region of SARS-CoV-2 spike subunit s1 noted as the receptor binding domain (RBD) and the human angiotensin-converting enzyme 2 (ACE2). To determine functional breadth of spike monomer, trimer and RBD in relation with ACE2, we apply Coulombs Law to determine an electric force between combinations with ACE2 across the range of pH from 3.0 to 9.0 in 0.1 increments. The results indicate that spike trimer should be used to determine mechanistic binding function and these data indicate that variations of spike sequence influence breadth of function. Our results also indicate the RBD has a broader range of function across pH compared to spike trimer, but is influenced by the range of function presented by the spike trimer.

Understanding how pH modulates underlying protein-protein interaction requires in-depth analysis across combinations of HIV proteins and broadly neutralizing antibodies (bnAbs). Traditional labs are not practical to screen all sequence variations and titrate of pH, highlighting the need to analyze such interactions theoretically. To fill this need, we generate homology models of predetermined HIV-1 gp120 in complex with bnAbs, through computational simulations to observe the binding energies as environmental pH varies. We compare simulation data to experimental data of the same structures at pH 5.5 and pH 7.4 specifically to present an 83.3% agreement between the two approaches to support the hypothesis that binding is stronger at lower pH. We then make observations of binding energy predictions across broad spectrum pH to provide insight into factors limiting the effectiveness of bnAbs in vivo. We conclude that theoretical models, and simulations involving them, provide data that closely mimic those of laboratory results.

Neurobiologically-inspired working memory models demonstrate human/animal capabilities to rapidly adapt and alter responses to the environment via context-switching and error monitoring. However, the application of these models outside of reinforcement learning problems has been relatively unexplored. We present a new framework compatible with Tensorflow/Keras enabling the integration of working memory-inspired mechanisms into typical neural network architectures. These mechanisms allow models to autonomously learn multiple tasks, statically or dynamically allocated. We also examine the generalization of the framework across a variety of multi-context supervised learning and reinforcement learning tasks. The resulting experiments successfully integrate these mechanisms with multi-layer and convolutional neural network architectures and the diversity of problems solved demonstrates the framework's generalizability across a variety of architectures and tasks.

Transfer learning allows for knowledge to generalize across tasks, resulting in increased learning speed and/or performance. These tasks must have commonalities that allow for knowledge to be transferred. The main goal of transfer learning in the reinforcement learning domain is to train and learn on one or more source tasks in order to learn a target task that exhibits better performance than if transfer was not used (Taylor and Stone 2009). Furthermore, the use of output-gated neural network models of working memory has been shown to increase generalization for supervised learning tasks (Kriete and Noelle 2011; Kriete et al. 2013). We propose that working memory-based generalization plays a significant role in a model's ability to transfer knowledge successfully across tasks. Thus, we extended the Holographic Working Memory Toolkit (HWMtk) (Dubois and Phillips 2017; Phillips and Noelle 2005) to utilize the generalization benefits of output gating within a working memory system. Finally, the model's utility was tested on a temporally extended, partially observable 5x5 2D grid-world maze task that required the agent to learn 3 tasks over the duration of the training period. The results indicate that the addition of output gating increases the initial learning performance of an agent in target tasks and decreases the learning time required to reach a fixed performance threshold.

An integral function of fully autonomous robots and humans is the ability to focus attention on a few relevant percepts to reach a certain goal while disregarding irrelevant percepts. Humans and animals rely on the interactions between the Pre-Frontal Cortex (PFC) and the Basal Ganglia (BG) to achieve this focus called Working Memory (WM). The Working Memory Toolkit (WMtk) was developed based on a computational neuroscience model of this phenomenon with Temporal Difference (TD) Learning for autonomous systems. Recent adaptations of the toolkit either utilize Abstract Task Representations (ATRs) to solve Feedback-Based (FB) tasks or storage of past input features to solve Sensory-Based (SB) tasks, but not both. We propose a new model, SBFBWMtk, which combines both approaches, ATRs and input storage, with a static or dynamic number of ATRs. The results of our experiments show that SBFBWMtk performs effectively for tasks that exhibit SB, FB, or both properties.

After three decades of research on human immunodeficiency virus (HIV), the causative agent of acquired immunodeficiency syndrome (AIDS), a vaccine has yet to be discovered. Most theoretical and experimental work on HIV vaccines has focused on the relevant molecular interactions at systemic pH levels, but HIV is typically transmitted sexually at mucosal pH levels. We previously developed a computational approach for calculating pH-sensitivity which predicted optimal transmission at mucosal pH levels, and was validated by experimental electrophoretic measurements and envelope protein binding assays. We have recently augmented this approach using a unique combination of protein dynamical modeling, parallel computation, and data compression tools which enable high-throughput calculations. The resulting fully-automated pipeline was capable of predicting pH sensitivity for a recent study involving more than 250 unique HIV envelope proteins utilizing approximately 1 million individual electrostatic surface calculations. We provide strong evidence that supports the previous hypothesis of a computational approach to determining the pH sensitivity of HIV envelopes. Furthermore, a PCA-based indexing method is proposed that allows for a comparison of biomolecular structures in terms of electrostatic pH sensitivity. We utilize the results to predict highly transmissible HIV variants with implications for vaccine design and efficacy.

BACKGROUND: Molecular dynamics (MD) simulation is a powerful technique for sampling the meta-stable and transitional conformations of proteins and other biomolecules. Computational data clustering has emerged as a useful, automated technique for extracting conformational states from MD simulation data. Despite extensive application, relatively little work has been done to determine if the clustering algorithms are actually extracting useful information. A primary goal of this paper therefore is to provide such an understanding through a detailed analysis of data clustering applied to a series of increasingly complex biopolymer models. RESULTS: We develop a novel series of models using basic polymer theory that have intuitive, clearly-defined dynamics and exhibit the essential properties that we are seeking to identify in MD simulations of real biomolecules. We then apply spectral clustering, an algorithm particularly well-suited for clustering polymer structures, to our models and MD simulations of several intrinsically disordered proteins. Clustering results for the polymer models provide clear evidence that the meta-stable and transitional conformations are detected by the algorithm. The results for the polymer models also help guide the analysis of the disordered protein simulations by comparing and contrasting the statistical properties of the extracted clusters. CONCLUSIONS: We have developed a framework for validating the performance and utility of clustering algorithms for studying molecular biopolymer simulations that utilizes several analytic and dynamic polymer models which exhibit well-behaved dynamics including: meta-stable states, transition states, helical structures, and stochastic dynamics. We show that spectral clustering is robust to anomalies introduced by structural alignment and that different structural classes of intrinsically disordered proteins can be reliably discriminated from the clustering results. To our knowledge, our framework is the first to utilize model polymers to rigorously test the utility of clustering algorithms for studying biopolymers.