Tsetlin Machines (TMs) have garnered increasing interest for their ability to learn concepts via propositional formulas and their proven efficiency across various application domains. Despite this, the convergence proof for the TMs, particularly for the AND operator (conjunction of literals), in the generalized case (inputs greater than two bits) remains an open problem. This paper aims to fill this gap by presenting a comprehensive convergence analysis of Tsetlin automaton-based Machine Learning algorithms. We introduce a novel framework, referred to as Probabilistic Concept Learning (PCL), which simplifies the TM structure while incorporating dedicated feedback mechanisms and dedicated inclusion/exclusion probabilities for literals. Given n features, PCL aims to learn a set of conjunction clauses Ci each associated with a distinct inclusion probability pi. Most importantly, we establish a theoretical proof confirming that, for any clause k, PCL converges to a conjunction of literals when pk is between 0.5 and 1. This result serves as a stepping stone for future research on the convergence properties of Tsetlin automaton-based learning algorithms. Our findings not only contribute to the theoretical understanding of Tsetlin automaton-based learning algorithms but also have implications for their practical application, potentially leading to more robust and interpretable machine learning models.

Running a numerical weather model such as WRF at kilometre or sub-kilometre grid spacing over a regional domain is computationally expensive. We present physics-informed deeplearning models that ingest a single 9km WRF wind field and simultaneously predict two finer-scale wind fields at 3 km and 1 km resolution via dual decoder heads. Four representative architectures are benchmarked-Deep Residual U-Net (DeepRU), DEVINE, a bespoke 3-D Transformer, and a Fourier Neural Operator (FNO)-each trained with divergence-free, vorticity, and Navier-Stokes residual constraints plus Charbonnier and gradient perceptual losses. We train and validate our models on the city of Oslo for the year 2018. DeepRU achieves$R^{2}=0.94$(RMSE$=0.050$) at$\mathbf{3 k m}$and$R^{2}=0.89(\mathbf{R M S E}=0.065)$at 1 km. DEVINE, Transformer 3-D, and FNO yield 3 km scores of$0.91-0.93$, with$\mathbf{1} \mathbf{ k m}$scores lower by$0.02-0.08$, illustrating the increased difficulty of finer-scale reconstruction. Physicsinformed losses improve all models compared to MSE-only baselines, and the residual architecture (DeepRU) remains most effective for this dual-scale task.

This study introduces deep learning (DL) methods for imputing missing allele-frequency information in large-scale genome-wide pooled re-sequencing (Pool-Seq) data, using the comprehensive DEST dataset based on over 270 global samples of the vinegar fly Drosophila melanogaster as a use case. The primary challenge addressed here is gap filling in DNA sequences, a critical issue in large-scale genomic studies. An empirical baseline for missing allele frequencies was established using an inverse-distance-weighting (IDW) method, leveraging geographical and temporal proximity among densely sampled populations. Additionally, a machine learning (ML) approach with k-means clustering grouped populations based on allele frequencies, independent of their spatiotemporal context. The core contribution of this research is the application of advanced DL models, specifically Masked Autoencoders (MAE), Variational Autoencoders (VAE) and Generative Adversarial Networks (GAN). These models excel in learning the data distribution and generating plausible imputations for missing sequences, outperforming the IDW and k-means based methods. Their effectiveness is due to their ability to handle high-dimensional genetic sequences and capture complex data correlations while maintaining sequential integrity. The study demonstrates the efficacy of DL in genomic data analysis, particularly for large-scale, complex datasets. VAE and GAN models offer a significant advancement over traditional ML methods, providing more accurate and efficient solutions for gap filling in genetic sequences. This research highlights the potential of DL in genomics, setting a precedent for future AI applications in biological data analysis and demonstrating a novel application area for deep learning techniques in handling complex long-sequence biological datasets.

This paper presents a novel deep learning-based methodology for estimating environmental noise across diverse European cities, significantly enhancing the coverage and precision of noise assessment and management. Utilizing an extensive range of environmental, traffic, and satellite-based land monitoring datasets, our models achieve a comprehensive and dynamic understanding of noise pollution patterns. The proposed method recontextualizes noise estimation as an image classification problem, employing advanced data preprocessing techniques and deep convolutional neural networks (CNNs) to predict noise levels with high accuracy. Our approach demonstrates scalability and cost-effectiveness in continuous noise monitoring, delivering critical insights into the health and policy implications of environmental noise. By filling data gaps in compliance with the European Environmental Noise Directive, our work facilitates the development of robust noise reduction strategies and informed urban planning. The results underscore the transformative potential of deep learning in environmental monitoring and decision-making, contributing significantly to the discourse on sustainable and resilient urban development.1

In response to the escalating ecological challenges that threaten global sustainability, there’s a need to investigate alternative methods of commerce, such as rental economies. Like most online commerce, rental or otherwise, a functioning recommender system is crucial for their success. Yet the domain has, until this point, been largely neglected by the recommender system research community.

Logic-based machine learning has the crucial advantage of transparency. However, despite significant recent progress, further research is needed to close the accuracy gap between logic-based architectures and deep neural network ones. This paper introduces a novel variant of the Tsetlin machine (TM) that randomly drops clauses, the logical learning element of TMs. In effect, TM with Drop Clause ignores a random selection of the clauses in each epoch, selected according to a predefined probability. In this way, the TM learning phase becomes more diverse. To explore the effects that Drop Clause has on accuracy, training time and robustness, we conduct extensive experiments on nine benchmark datasets in natural language processing (IMDb, R8, R52, MR, and TREC) and image classification (MNIST, Fashion MNIST, CIFAR-10, and CIFAR-100). Our proposed model outperforms baseline machine learning algorithms by a wide margin and achieves competitive performance compared with recent deep learning models, such as BERT-Large and AlexNet-DFA. In brief, we observe up to +10% increase in accuracy and 2x to 4x faster learning than for the standard TM. We visualize the patterns learnt by Drop Clause TM in the form of heatmaps and show evidence of the ability of drop clause to learn more unique and discriminative patterns. We finally evaluate how Drop Clause affects learning robustness by introducing corruptions and alterations in the image/language test data, which exposes increased learning robustness.

Tsetlin Machine (TM) is a logic-based machine learning approach with the crucial advantages of being transparent and hardware-friendly. While TMs match or surpass deep learning accuracy for an increasing number of applications, large clause pools tend to produce clauses with many literals (long clauses). As such, they become less interpretable. Further, longer clauses increase the switching activity of the clause logic in hardware, consuming more power. This paper introduces a novel variant of TM learning -- Clause Size Constrained TMs (CSC-TMs) -- where one can set a soft constraint on the clause size. As soon as a clause includes more literals than the constraint allows, it starts expelling literals. Accordingly, oversized clauses only appear transiently. To evaluate CSC-TM, we conduct classification, clustering, and regression experiments on tabular data, natural language text, images, and board games. Our results show that CSC-TM maintains accuracy with up to 80 times fewer literals. Indeed, the accuracy increases with shorter clauses for TREC and BBC Sports. After the accuracy peaks, it drops gracefully as the clause size approaches one literal. We finally analyze CSC-TM power consumption and derive new convergence properties.

This paper introduces an interpretable contextual bandit algorithm using Tsetlin Machines, which solves complex pattern recognition tasks using propositional (Boolean) logic. The proposed bandit learning algorithm relies on straightforward bit manipulation, thus simplifying computation and interpretation. We then present a mechanism for performing Thompson sampling with Tsetlin Machine, given its non-parametric nature. Our empirical analysis shows that Tsetlin Machine as a base contextual bandit learner outperforms other popular base learners on eight out of nine datasets. We further analyze the interpretability of our learner, investigating how arms are selected based on propositional expressions that model the context.

Deep reinforcement learning (RL) is achieving significant success in various applications like control, robotics, games, resource management, and scheduling. However, the important problem of emergency evacuation, which clearly could benefit from RL, has been largely unaddressed. Indeed, emergency evacuation is a complex task that is difficult to solve with RL. An emergency situation is highly dynamic, with a lot of changing variables and complex constraints that make it challenging to solve. Also, there is no standard benchmark environment available that can be used to train RL agents for evacuation. A realistic environment can be complex to design. In this article, we propose the first fire evacuation environment to train RL agents for evacuation planning. The environment is modeled as a graph capturing the building structure. It consists of realistic features like fire spread, uncertainty, and bottlenecks. The implementation of our environment is in the OpenAI gym format, to facilitate future research. We also propose a new RL approach that entails pretraining the network weights of a DQN-based agent [DQN/Double-DQN (DDQN)/Dueling-DQN] to incorporate information on the shortest path to the exit. We achieved this by using tabular$Q$-learning to learn the shortest path on the building model’s graph. This information is transferred to the network by deliberately overfitting it on the$Q$-matrix. Then, the pretrained DQN model is trained on the fire evacuation environment to generate the optimal evacuation path under time varying conditions due to fire spread, bottlenecks, and uncertainty. We perform comparisons of the proposed approach with state-of-the-art RL algorithms like DQN, DDQN, Dueling-DQN, PPO, VPG, state-action-reward-state-action (SARSA), actor–critic method, and ACKTR. The results show that our method is able to outperform state-of-the-art models by a huge margin including the original DQN-based models. Finally, our model is tested on a large and complex real building consisting of 91 rooms, with the possibility to move to any other room, hence giving 8281 actions. In order to reduce the action space, we propose a strategy that involves one step simulation. That is, an action importance vector is added to the final output of the pretrained DQN and acts like an attention mechanism. Using this strategy, the action space is reduced by 90.1%. In this manner, the model is able to deal with large action spaces. Hence, our model achieves near optimal performance on the real world emergency environment.

In this paper, we propose a model for the Environment Sound Classification Task (ESC) that consists of multiple feature channels given as input to a Deep Convolutional Neural Network (CNN) with Attention mechanism. The novelty of the paper lies in using multiple feature channels consisting of Mel-Frequency Cepstral Coefficients (MFCC), Gammatone Frequency Cepstral Coefficients (GFCC), the Constant Q-transform (CQT) and Chromagram. Such multiple features have never been used before for signal or audio processing. And, we employ a deeper CNN (DCNN) compared to previous models, consisting of spatially separable convolutions working on time and feature domain separately. Alongside, we use attention modules that perform channel and spatial attention together. We use some data augmentation techniques to further boost performance. Our model is able to achieve state-of-the-art performance on all three benchmark environment sound classification datasets, i.e. the UrbanSound8K (97.52%), ESC-10 (95.75%) and ESC-50 (88.50%). To the best of our knowledge, this is the first time that a single environment sound classification model is able to achieve state-of-the-art results on all three datasets. For ESC-10 and ESC-50 datasets, the accuracy achieved by the proposed model is beyond human accuracy of 95.7% and 81.3% respectively.

In this paper, we try to find the most efficient optimization algorithm that can be used to resolve the hydropower optimization problem. We propose a novel optimization technique is called the Split-window method. The method is relatively simple and reduces the complexity of the optimization problem by split-ting the planning horizon (and datasets) into equal windows and assigning the same values to policies(actions) within each part. After splitting, a meta-heuristic technique is used to optimize the actions, and the dataset is split again until a split contains only one instance (timestep). The unique values to be optimized during each iteration is equal to the number of splits which makes it very fast and requires fewer computations. We also propose a novel initialization method based on ranking of price and assigning a higher value of production and hatch release for higher prices. We apply this initialization technique to most of the algorithms used in this paper. We compare the split-window technique with meta-heuristic methods such as hill climbing, simulated annealing, line search, and genetic algorithms by running simulations on the data collected from a real-world hydropower river system in southern Norway. In total, we benchmark the performance of seven different optimization algorithms for a large number of hydrological and price scenarios. The results show that the Split-window method is able to beat other techniques in terms of performance score, speed of convergence and core algorithmic complexity by a considerable margin.

Abstract Recent advances in intrusion detection systems based on machine learning have indeed outperformed other techniques, but struggle with detecting multiple classes of attacks with high accuracy. We propose a method that works in three stages. First, the ExtraTrees classifier is used to select relevant features for each type of attack individually for each (ELM). Then, an ensemble of ELMs is used to detect each type of attack separately. Finally, the results of all ELMs are combined using a softmax layer to refine the results and increase the accuracy further. The intuition behind our system is that multi-class classification is quite difficult compared to binary classification. So, we divide the multi-class problem into multiple binary classifications. We test our method on the UNSW and KDDcup99 datasets. The results clearly show that our proposed method is able to outperform all the other methods, with a high margin. Our system is able to achieve 98.24% and 99.76% accuracy for multi-class classification on the UNSW and KDDcup99 datasets, respectively. Additionally, we use the weighted extreme learning machine to alleviate the problem of imbalance in classification of attacks, which further boosts performance. Lastly, we implement the ensemble of ELMs in parallel using GPUs to perform intrusion detection in real time.