In the dynamic landscape of machine learning applications on streaming data, the constant evolution of models and input data complicates optimal model deployment. The static selection of a model risks suboptimal performance as data patterns evolve, while frequent redeployments increase operational costs. This paper proposes a self-adaptive system that autonomously selects interchangeable models for processing streaming data while balancing the tradeoff of performance and redeployment frequency. Inspired by the MAPE-K reference model, our approach utilizes an adaptive model selection control loop to continuously monitor model performance on production and experimental data. "what-if" environments are introduced to collect additional experimental data, simulating production-like scenarios. A selection algorithm that employs two distinct adaptation policies is introduced that strategically plans the selection of the most suitable module for upcoming data. Leveraging a learning-based method, we improve the efficiency of our system by recognizing the patterns of selection eliminating the need for further experimental data collection. Empirical evaluation on an energy forecasting use case spans over 16 years of data demonstrates a substantial reduction in errors up to 34% compared to the best static selection, affirming the proposed framework’s effectiveness. Our findings reveal the potential to discontinue experimental "what-if" analyses with just 12% of historical data, which underlines the practicality of our adaptive strategy on a long-lasting task.

Abstract Conventional digital twins (DT) for critical infrastructures are widely used to model and simulate the system’s state. But fundamental environment changes bring challenges for DT adaptation to new conditions, leading to a progressively decreasing correspondence of the DT to its physical counterpart. This paper introduces the DiTEC system, a Digital Twin for Evolutionary Changes in Water Distribution Networks (WDN). This framework combines novel techniques, including semantic rule learning, graph neural network-based state estimation, and adaptive model selection, to ensure that changes are adequately detected, processed and the DT is updated to the new state. The DiTEC system is tested on the Dutch Oosterbeek region WDN, with results showing the superiority of the approach compared to traditional methods.

BACKGROUND: Once aligned, long-reads can be a useful source of information to identify the type and position of structural variations. However, due to the high sequencing error of long reads, long-read structural variation detection methods are far from precise in low-coverage cases. To be accurate, they need to use high-coverage data, which in turn, results in an extremely time-consuming pipeline, especially in the alignment phase. Therefore, it is of utmost importance to have a structural variation calling pipeline which is both fast and precise for low-coverage data. RESULTS: In this paper, we present SVNN, a fast yet accurate, structural variation calling pipeline for PacBio long-reads that takes raw reads as the input and detects structural variants of size larger than 50 bp. Our pipeline utilizes state-of-the-art long-read aligners, namely NGMLR and Minimap2, and structural variation callers, videlicet Sniffle and SVIM. We found that by using a neural network, we can extract features from Minimap2 output to detect a subset of reads that provide useful information for structural variation detection. By only mapping this subset with NGMLR, which is far slower than Minimap2 but better serves downstream structural variation detection, we can increase the sensitivity in an efficient way. As a result of using multiple tools intelligently, SVNN achieves up to 20 percentage points of sensitivity improvement in comparison with state-of-the-art methods and is three times faster than a naive combination of state-of-the-art tools to achieve almost the same accuracy. CONCLUSION: Since prohibitive costs of using high-coverage data have impeded long-read applications, with SVNN, we provide the users with a much faster structural variation detection platform for PacBio reads with high precision and sensitivity in low-coverage scenarios.

BACKGROUND: Long reads provide valuable information regarding the sequence composition of genomes. Long reads are usually very noisy which renders their alignments on the reference genome a daunting task. It may take days to process datasets enough to sequence a human genome on a single node. Hence, it is of primary importance to have an aligner which can operate on distributed clusters of computers with high performance in accuracy and speed. RESULTS: In this paper, we presented IMOS, an aligner for mapping noisy long reads to the reference genome. It can be used on a single node as well as on distributed nodes. In its single-node mode, IMOS is an Improved version of Meta-aligner (IM) enhancing both its accuracy and speed. IM is up to 6x faster than the original Meta-aligner. It is also implemented to run IM and Minimap2 on Apache Spark for deploying on a cluster of nodes. Moreover, multi-node IMOS is faster than SparkBWA while executing both IM (1.5x) and Minimap2 (25x). CONCLUSION: In this paper, we purposed an architecture for mapping long reads to a reference. Due to its implementation, IMOS speed can increase almost linearly with respect to the number of nodes in a cluster. Also, it is a multi-platform application able to operate on Linux, Windows, and macOS.