Traditional neural networks (NNs) learn primarily from data, which limits their capacity to represent relational knowledge or handle symbolic relational data effectively. Although graph neural networks (GNNs) address this limitation at the level of relational data, they continue to struggle at learning relational knowledge. Neural-symbolic learning offers a solution by combining machine learning with knowledge representation, enabling the development of interpretable logic-based models learned from neural networks. Bottom clause propositionalization (BCP) is a prominent approach that transforms relational knowledge into attribute-value examples. A bottom clause is a logical representation created from each example as a starting point for the search process. BCP can be used with symbolic learners or neural networks to tackle relational domains. However, BCP often faces significant memory storage problems when handling larger datasets due to the volume of logical literals that it generates. Semi-propositionalization can alleviate these storage problems by grouping logical literals. However, it does not eliminate the substantial time requirements to create a bottom clause for each example. This paper investigates the application of sampling to the examples used for bottom clause generation. The hypothesis is that the number of examples needed to generate bottom clauses can be reduced significantly. Finding representative bottom clauses from data should enable relational learning to take place at an adequate level of abstract relational knowledge rather than simply at the level of the relations between any two data points. We evaluate this hypothesis by training a classifier with different sampling from five relational datasets. We experimentally validate the size of each sampling for each dataset. Experimental results show that training classifiers with fewer relational examples produces competitive results compared to using the entire dataset. The best results are obtained with up to 50% reduction in the set of examples.

The analysis of structure in complex datasets has become essential to solving difficult Machine Learning problems. Relational aspects of data, capturing relationships between objects, play a crucial role in understanding the underlying data structure. While traditional graph algorithms have been widely used for binary relations, recent evidence suggests that hypergraphs can provide a more effective approach for modeling complex, non-binary relations. Hypergraph Neural Networks (HGNN) have been shown to offer a small improvement in performance when compared to Graph Neural Networks (GNN). In this paper, a new approach is proposed for inserting relational domain knowledge into HGNNs using a logic clause expressing non-binary relations. We evaluate the performance of this new hypergraph model, called Bottom-clause HGNN (BHGNN), in comparison with well-known approaches. Results show that BHGNN can achieve statistically significant improvement of performance, based on the Wilcoxon signed-ranks test, in comparison with HGNN and GNNs.

Data on software vulnerabilities, products, and exploits are typically collected from multiple non-structured sources. Valuable information, e.g., on which products are affected by which exploits, is conveyed by matching data from those sources, i.e., through their relations. In this paper, we leverage this simple albeit unexplored observation to introduce a statistical relational learning (SRL) approach for the analysis of vulnerabilities, products, and exploits. In particular, we focus on the problem of determining the existence of an exploit for a given product, given information about the relations between products and vulnerabilities, and vulnerabilities and exploits, focusing on Industrial Control Systems (ICS), the National Vulnerability Database, and ExploitDB. Using RDN-Boost, we were able to reach an AUC ROC of 0.80 and an AUC PR of 0.65 for the problem at hand. To reach that performance, we indicate that it is instrumental to include textual features, e.g., extracted from the description of vulnerabilities, as well as structured information, e.g., about product categories. In addition, using interpretable relational regression trees, we report simple rules that shed insight on factors impacting the weaponization of ICS products.

In order to practice a legal profession in Brazil, law graduates must be approved in the OAB national unified bar exam.For their topic coverage and national reach, the OAB exams provide an excellent benchmark for the performance of legal information systems, as it provides objective metrics and are challenging even for humans, as only 20% of its candidates are approved.After constructing a new data set on the exams and doing shallow experiments on it, we now employ the OpenWordnet-PT to verify whether using word senses and relations we can improve previous results.We discuss the results, possible future ideas and the additions to the OpenWordnet-PT that we made.

FORTE_MBC is a First-Order Logic theory revision system, built upon the FORTE system but with an important supplement: it makes use of (i) a Bottom Clause to define the search space of literals and (ii) a set of mode declarations to validate the yielded clauses. Introducing the Bottom Clause was essential to reduce the runtime of the revision process (experimental results showed an average speed-up of 55|$\times$|⁠), since one of the key steps of the revision process is to create literals to add to clauses. However, the effectiveness and efficiency of learning and revising as a refinement process heavily rely on other factors, such as the generality relation that induces a generalization model for the search space, and, as a consequence, the refinement operators. These components have never been formally analysed in FORTE(_MBC). In this work, we contribute with (i) an adaptation of the existing theoretical frameworks that characterize refinement operators to define the search space and revision operators of systems like FORTE_MBC, which use a set of Bottom Clauses and mode declarations to constrain the search; (ii) an improvement of the theory refinement operators of FORTE_MBC to make them ideal for the defined space. We present the feasibility of these modified operators by implementing them in the FORTE_MBC system. Experimental results show that we are indeed able to obtain a more efficient revision process by using the proposed ideal operators.

MOTIVATION: Given a protein sequence and a number of potential domains matching it, what are the domain content and the most likely domain architecture for the sequence? This problem is of fundamental importance in protein annotation, constituting one of the main steps of all predictive annotation strategies. On the other hand, when potential domains are several and in conflict because of overlapping domain boundaries, finding a solution for the problem might become difficult. An accurate prediction of the domain architecture of a multi-domain protein provides important information for function prediction, comparative genomics and molecular evolution. RESULTS: We developed DAMA (Domain Annotation by a Multi-objective Approach), a novel approach that identifies architectures through a multi-objective optimization algorithm combining scores of domain matches, previously observed multi-domain co-occurrence and domain overlapping. DAMA has been validated on a known benchmark dataset based on CATH structural domain assignments and on the set of Plasmodium falciparum proteins. When compared with existing tools on both datasets, it outperforms all of them. AVAILABILITY AND IMPLEMENTATION: DAMA software is implemented in C++ and the source code can be found at http://www.lcqb.upmc.fr/DAMA. CONTACT: [email protected] or [email protected] SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Traditional protein annotation methods describe known domains with probabilistic models representing consensus among homologous domain sequences. However, when relevant signals become too weak to be identified by a global consensus, attempts for annotation fail. Here we address the fundamental question of domain identification for highly divergent proteins. By using high performance computing, we demonstrate that the limits of state-of-the-art annotation methods can be bypassed. We design a new strategy based on the observation that many structural and functional protein constraints are not globally conserved through all species but might be locally conserved in separate clades. We propose a novel exploitation of the large amount of data available: 1. for each known protein domain, several probabilistic clade-centered models are constructed from a large and differentiated panel of homologous sequences, 2. a decision-making protocol combines outcomes obtained from multiple models, 3. a multi-criteria optimization algorithm finds the most likely protein architecture. The method is evaluated for domain and architecture prediction over several datasets and statistical testing hypotheses. Its performance is compared against HMMScan and HHblits, two widely used search methods based on sequence-profile and profile-profile comparison. Due to their closeness to actual protein sequences, clade-centered models are shown to be more specific and functionally predictive than the broadly used consensus models. Based on them, we improved annotation of Plasmodium falciparum protein sequences on a scale not previously possible. We successfully predict at least one domain for 72% of P. falciparum proteins against 63% achieved previously, corresponding to 30% of improvement over the total number of Pfam domain predictions on the whole genome. The method is applicable to any genome and opens new avenues to tackle evolutionary questions such as the reconstruction of ancient domain duplications, the reconstruction of the history of protein architectures, and the estimation of protein domain age. Website and software: http://www.lcqb.upmc.fr/CLADE.

BACKGROUND: An important problem in computational biology is the automatic detection of protein families (groups of homologous sequences). Clustering sequences into families is at the heart of most comparative studies dealing with protein evolution, structure, and function. Many methods have been developed for this task, and they perform reasonably well (over 0.88 of F-measure) when grouping proteins with high sequence identity. However, for highly diverged proteins the performance of these methods can be much lower, mainly because a common evolutionary origin is not deduced directly from sequence similarity. To the best of our knowledge, a systematic evaluation of clustering methods over distant homologous proteins is still lacking. RESULTS: We performed a comparative assessment of four clustering algorithms: Markov Clustering (MCL), Transitive Clustering (TransClust), Spectral Clustering of Protein Sequences (SCPS), and High-Fidelity clustering of protein sequences (HiFix), considering several datasets with different levels of sequence similarity. Two types of similarity measures, required by the clustering sequence methods, were used to evaluate the performance of the algorithms: the standard measure obtained from sequence-sequence comparisons, and a novel measure based on profile-profile comparisons, used here for the first time. CONCLUSIONS: The results reveal low clustering performance for the highly divergent datasets when the standard measure was used. However, the novel measure based on profile-profile comparisons substantially improved the performance of the four methods, especially when very low sequence identity datasets were evaluated. We also performed a parameter optimization step to determine the best configuration for each clustering method. We found that TransClust clearly outperformed the other methods for most datasets. This work also provides guidelines for the practical application of clustering sequence methods aimed at detecting accurately groups of related protein sequences.

Multiple instance learning is an increasingly important area in machine learning. In multi-instance learning, the training set is structured into subsets (or bags) of instances. The bags are labelled, but the label of each instance is unknown or irrelevant. In this paper, we revisit the connectionist approach to multi-instance learning. We propose a recurrent neural network model for multi-instance learning. We have applied the new model to a benchmark multi-instance dataset. The results provide evidence that connectionist multi-instance learning is more promising than previously anticipated. We argue that a principled connectionist approach should provide robust and efficient multi-instance learning, yet comparative results should be taken with caution as a result of varying methodologies.

Genetic Algorithms (GAs) are known for their capacity to explore large search spaces and due to this ability, they were to some extent applied to Inductive Logic Programming (ILP) problem. Although Estimation of Distribution Algorithms (EDAs) perform better in most problems when compared to standard GAs, this kind of algorithm have not been applied to ILP. This work presents an ILP system based on EDA. Preliminary results show that the proposed system is superior when compared to a "standard" GA and it is very competitive when compared to the state of the art ILP system Aleph.

BACKGROUND: Remote homology detection is a hard computational problem. Most approaches have trained computational models by using either full protein sequences or multiple sequence alignments (MSA), including all positions. However, when we deal with proteins in the "twilight zone" we can observe that only some segments of sequences (motifs) are conserved. We introduce a novel logical representation that allows us to represent physico-chemical properties of sequences, conserved amino acid positions and conserved physico-chemical positions in the MSA. From this, Inductive Logic Programming (ILP) finds the most frequent patterns (motifs) and uses them to train propositional models, such as decision trees and support vector machines (SVM). RESULTS: We use the SCOP database to perform our experiments by evaluating protein recognition within the same superfamily. Our results show that our methodology when using SVM performs significantly better than some of the state of the art methods, and comparable to other. However, our method provides a comprehensible set of logical rules that can help to understand what determines a protein function. CONCLUSIONS: The strategy of selecting only the most frequent patterns is effective for the remote homology detection. This is possible through a suitable first-order logical representation of homologous properties, and through a set of frequent patterns, found by an ILP system, that summarizes essential features of protein functions.

The performance of Evolutionary Algorithms for combinatorial problems can be significantly improved by adding Local Search, thus obtaining a Genetic Local Search (GLS) also called Memetic Algorithm. In this work, we adapt a previous Stochastic Local Search (SLS) algorithm and embed it into a GBML system. The adapted SLS algorithm works as a module of the system that tries to improve a random individual in the population. We perform experiments to evaluate this adapted SLS procedure and results show that this new GLS system is very effective, not losing in any of the 10 UCI datasets tested when compared to the system without the SLS procedure. The system either obtained significantly more accurate concepts using lower number of rules and features or it achieved the same accuracy as the system without the SLS procedure, but reduced the number of rules and features, and also the time taken to develop the solution.

Naive Bayes for regression (NBR) uses the naive Bayes methodology to numeric prediction tasks. The main reason for its poor performance is the independence assumption. Although many recent researches try to improve the performance of naive Bayes by relaxing the independence assumption, none of them can be directly applied to the regression framework. The objective of this work is to present a new approach to improve the results of the NBR algorithm, by combining attributes by means of an auxiliary regression algorithm.

Aguilar-Ruiz et al proposed crossover operators, both discrete and continuous, for the natural representation (henceforth called NCO). NCO showed advantages in accuracy and in efficiency compared to the binary ones. However, they do not explore the search space like the two points crossover when the binary coding is used. In order to do so, in our previous work we proposed a new natural discrete crossover operator, which gave very good results compared to C4.5 in several UCI databases. Nonetheless, it was not experimentally compared to NCO. So, in this work, we perform this comparison in the same datasets and define a new natural continuous crossover operator, which is also compared to the continuous NCO operator. The experimental results showed the advantages of both new natural operators: our discrete operator achieves better accuracy and simpler concepts using less time, whereas our continuous operator is also able to explore the search space in a more efficient way, leading to better results in less time.

Particle filters (PF) are sequential Monte Carlo methods based in the representation of probability densities with mass points. They can be applied to any state-space model and generalize the traditional Kalman filter methods, providing better results. However, currently most researches involving time series forecasting use the traditional methods. The REC analysis is a powerful technique for visualization and comparison of regression models. The objective of this work is to advocate the use of REC curves in order to compare traditional Kalman filter methods with particle filters and analyze their use in ensembles, which can achieve a better performance.

BACKGROUND: Remote homology detection is a challenging problem in Bioinformatics. Arguably, profile Hidden Markov Models (pHMMs) are one of the most successful approaches in addressing this important problem. pHMM packages present a relatively small computational cost, and perform particularly well at recognizing remote homologies. This raises the question of whether structural alignments could impact the performance of pHMMs trained from proteins in the Twilight Zone, as structural alignments are often more accurate than sequence alignments at identifying motifs and functional residues. Next, we assess the impact of using structural alignments in pHMM performance. RESULTS: We used the SCOP database to perform our experiments. Structural alignments were obtained using the 3DCOFFEE and MAMMOTH-mult tools; sequence alignments were obtained using CLUSTALW, TCOFFEE, MAFFT and PROBCONS. We performed leave-one-family-out cross-validation over super-families. Performance was evaluated through ROC curves and paired two tailed t-test. CONCLUSION: We observed that pHMMs derived from structural alignments performed significantly better than pHMMs derived from sequence alignment in low-identity regions, mainly below 20%. We believe this is because structural alignment tools are better at focusing on the important patterns that are more often conserved through evolution, resulting in higher quality pHMMs. On the other hand, sensitivity of these tools is still quite low for these low-identity regions. Our results suggest a number of possible directions for improvements in this area.

We present two new systems that approximate probability density functions (pdFs) in order to predict continuous values of time series: the fuzzy multi-hidden Markov predictor (FMHMP) and the multi-hidden Markov model for regression (MHMMR). They use fuzzification or discretization of continuous data and dynamic Bayesian networks (DBN's) to estimate pdfs and then make continuous predictions. A DBN is a Bayesian network that represents a temporal probability model. The employed DBN is a generalization of the hidden Markov model that allows multiple hidden variables. The new systems are applied to the task of monthly electric load single-step forecasting and successfully compared with other fuzzy and discrete probabilistic predictors, two Kalman filter models, and two traditional forecasting methods, Box-Jenkins and Winters exponential smoothing. The employed time series present a sudden significant changing behavior at their last years, as it occurs in an energy rationing.

We present two new systems that approximate probability density functions (pdFs) in order to predict continuous values of time series: the fuzzy multi-hidden Markov predictor (FMHMP) and the multi-hidden Markov model for regression (MHMMR). They use fuzzification or discretization of continuous data and dynamic Bayesian networks (DBN's) to estimate pdfs and then make continuous predictions. A DBN is a Bayesian network that represents a temporal probability model. The employed DBN is a generalization of the hidden Markov model that allows multiple hidden variables. The new systems are applied to the task of monthly electric load single-step forecasting and successfully compared with other fuzzy and discrete probabilistic predictors, two Kalman filter models, and two traditional forecasting methods, Box-Jenkins and Winters exponential smoothing. The employed time series present a sudden significant changing behavior at their last years, as it occurs in an energy rationing.

We present three different approaches for multi-step prediction using the fuzzy Markov predictor (FMP). The FMP is a modification of the hidden Markov model in order to enable it to predict numerical values. In the first approach, the one normally used in neural networks, past predictions are used as input for the next predictions. The second and third approaches follow the standard way of making multi-step prediction in a dynamic Bayesian network. FMP using these three approaches is applied to the task of monthly electric load multi-step forecasting and successfully compared with two Kalman filter models, BATS and STAMP, and two traditional forecasting methods, Box-Jenkins and Winters exponential smoothing.

Pinkas defined (1991, 1992) a bi-directional mapping between propositional logic formulas and energy functions of symmetric neural networks. He showed that determining whether a propositional logic formula is satisfiable is equivalent to finding whether the global minimum of its associated energy function is equal to zero. He also defined how to transform a first-order resolution-based theorem proof of a formula (query Q) from a given set of formulas (knowledge base KB) into a set of constraints described by a set of propositional logic formulas C. Then he showed that the satisfaction of C is sound and complete with respect to a first-order resolution-based proof of Q from KB. Therefore finding that the global minimum of the energy function associated to C is equal to zero is sound and complete with respect to a first-order resolution-based proof of Q from KB. The proof itself could be extracted from the state of the neurons when the network stops in the global minimum. Pinkas did not implement his system. In this work we point out some adjustments to C and we present an implementation of the revised system. We also show some experimental results and point out some future works.

We have proposed for the task of hourly electric load forecasting a hybrid neural system combining unsupervised and supervised learning. The system consists of a recurrent neural gas (RNG) network and many Elman neural networks (ENs). RNG is a modification we introduced in the neural gas (NG) network in order to enable it to do clustering using a sequence of input data. For verifying the RNG's performance, many architectures are compared in the learning of global and local models. In a global model only one supervised network is trained and in a local model the training examples are grouped by a clustering algorithm and each one of these groups is sent to different supervised networks. These architectures use different clustering algorithms (NG and RNG) or different supervised networks for prediction (ENs that are trained by backpropagation or backpropagation through time, and feedforward networks).

Cooperative information systems (CIS) often consist of applications that access shared resources such as databases. Since centralized systems may have a great impact on the system performance, parallel and distribution techniques are needed for attaining scalability. Distributed databases are, then, crucial for the development of cooperative applications. However, in order to improve performance, it is very important to design information distribution properly, which is the goal of distribution design. Considering the various difficulties embedded in the design of distributed object oriented databases, this work presents an algorithm to assist distribution designers in their task. The analysis algorithm indicates the most adequate fragmentation technique (vertical, horizontal or mixed) for each class in the database schema, and we propose the use of a machine learning method-inductive logic programming-to uncover some implicit issues to be considered in the distribution design, thus revising the proposed analysis algorithm.