Automatic detection of Major Depression Disorder (MDD) from brain MRI images with machine learning has been an active area of study. In this paper several methods are explored for MDD detection by combining features from structural and functional brain MRI images, and combining Atlas-based and spatial cube-based features. Experiments demonstrate good classification performance on an imbalanced dataset. The paper also presents a visualization that captures the spatial overlapping between the top discriminating spatial cube pairs and the regions of interests in the Harvard Atlas.

Due to the heavy computational costs of pure simulation-based reservoir models when used for hydraulic fracturing design, machine learning (ML) approaches become promising in petroleum industry to provide effective and efficient well completion optimization solutions. In this work, we propose a framework of using ML approaches to optimize multi-stage hydraulic fracturing design for economical development of unconventional wells in Permian basin, West Texas. Since the goals of maximizing oil production and minimizing completion cost are generally conflicting, combining both as a bi-objective optimization for maximal profit is the main objective in this study. Two ML regression models are selected to predict the $1^{\mathrm{s}\mathrm{t}}$-year oil production and completion cost, respectively. The bi-objective optimization is shown to help gain more profit than the single-objective optimizations of oil production or completion cost only. From the case study of an exam well, we predict the profit as ${\$}$12.5 millions when optimizing oil production only, ${\$}$7.9 millions when optimizing completion cost only, and ${\$}$13.4 millions when optimizing both. This bi-objective optimization approach in this study can help to increase profit up to ${\$}$1 million. Moreover, the framework can help the decision makers to select solutions of optimal completion design parameters based on their preferences.

Major Depression Disorder (MDD) affects people's life and it is a common disorder worldwide. Finding useful diagnostic biomarkers would help clinicians to diagnosis MDD in its early stages. Having automated methods to find biomarkers for MDD is beneficial although it is challenging. In this paper, we investigate deep learning approaches versus functional connectivity-based approach for MDD classification using resting state fMRI data. To reduce data dimension, we have used Smith atlas. We compare several feature extraction methods. Experiments show promising results with connectivity-based approach. Moreover, applying T-test for finding discriminative features effectively improves the performance.

Machine learning models especially deep neural network models have shown great potential in making decisions when analyzing clustered or longitudinal data. However, lack of model transparency is a major concern in risk sensitive domains such as social science and medical diagnosis. Despite the early success of explaining machine learning models, there is a lack of explanation methods that can be applied to any predictors on clustered data since most of the existing models assume that all observations are independent of each other. In this paper, we address this deficiency and propose to use a linear mixed model to mimic the local behavior of any complex model on clustered data, which can also improve the fidelity of the explanation method to the complex models. We apply our method to explain several models including a deep neural network model on two tasks including movie recommendation and medical record diagnosis. Experiment results show that our model outperforms the baseline models on several metrics such as fidelity and exactness.

Major Depression Disorder (MDD) affects people's life and it is a common disorder worldwide. Finding useful diagnostic biomarkers would help clinicians to diagnosis MDD in its early stages. Machine learning algorithms for brain imaging classification of MDD is beneficial although it is challenging. In this paper, we investigate utilizing augmentation, feature extraction and classification for MDD detection from sMRI Images. Using images, we extract 10 slices with highest entropy values from each 3D sMRI image as raw features. The selected slices are the most representative 2D slices of each volume and the problem is reduced to classifying these slices. Next, augmentation is done by either duplication or rotation to balance the data. Then deep learning algorithms such as convolutional neural network and pre-trained networks with or without fine tuning are applied to extract features automatically. We compared several feature extraction methods in combination with SVM classifiers with linear or RBF kernels. Experiments show that raw features work pretty well for MDD classification. Also, pre-trained VGG16 with fine-tuning produces good results. However, pre-trained network without fine tuning returns acceptable result with short training time.

This study is motivated by the magnitude of the problem of Louisiana high school dropout and its negative impacts on individual and public well-being. Our goal is to predict students who are at risk of high school dropout, by examining Louisiana administrative dataset. Due to the imbalanced nature of the dataset, imbalanced learning techniques including resampling, case weighting, and cost-sensitive learning have been applied to enhance the prediction performance on the rare class. Performance metrics used in this study are F-measure, recall and precision of the rare class. We compare the performance of several machine learning algorithms such as neural networks, decision trees and bagging trees in combination with the imbalanced learning approaches using an administrative dataset of size of 366k+ from Louisiana Department of Education. Experiments show that application of imbalanced learning methods produces good results on recall but decreases precision, whereas base classifiers without regard of imbalanced data handling gives better precision but poor recall. Overall application of imbalanced learning techniques is beneficial, yet more studies are desired to improve precision.

Railroad traffic safety is a major worldwide concern. Since 2000, there have been 48,083 crashes at highway-rail grade crossings in the US resulting in 6,103 fatalities, 18,851 injuries, and $302,065,336 cost of vehicle damages. Federal Railroad Administrator (FRA) seeks to improve safety and consolidate the grade crossing in risky areas. Towards the goal of better highway-rail cross safety, this paper aims to explore the reasons behind highway-rail crossing crashes by text mining within the narrative information in the crash data. The similarity of the extracted crash reasons between the 50 states and Washington DC in the USA is also calculated to create a comprehensive document for Departments of Transportation (DOTs) and railroad agencies. Important features are extracted from the FRA crash data along with two new features (geographical region and weight of narrative crash data) to classify the type of train-vehicle crashes into "train struck car" and "car struck train". The weight of narrative field was calculated using text mining techniques. Random Forest and Logistic Regression machine learning algorithms have been applied to build prediction models for the classification task. Experimental results indicate an overall accuracy of 0.86 and 0.74 for the proposed Random Forest and Logistic Regression models respectively.

This paper explores the application of machine learning techniques in finance to predict if a publicly-traded firm will issue a Seasoned Equity Offering (SEO) by analyzing the firm's 10-Q filing documents with Security and Exchange Commissions (SEC). Specifically, using the information content in the Management Discussion and Analysis section (MD&A) of 10-Q filings, we train five different algorithms, including Logistic Regression (LR), Support Vector Classification (SVC), Multinomial Naïve Bayes (NB), Artificial Neural Network (ANN) and Random Forest (RF). Two types of features, unigrams and phrases are considered. Term frequency-inverse document (TF-IDF) scores are used as independent variables in these models. Experimental results show that the accuracy of phrases-only models has a range of 0-2% improvement for LR, NB, and RF compared with unigrams-only models. The accuracy of phrase-only model for SVC is close to that of unigrams-only model. The 74.53% accuracy of unigrams-only model for SVC classifier performs the best among all tested classifiers. The precision of all models varies between 60% and 75%, while the recall varies between 55% and 85%. Further, we tune model parameters of one linear model (LR) and one non-linear model (RF) to see how these parameters will impact the models' performance. Finally, we apply RF to find the most important features on prediction and find that "merger" is the most important feature in both unigrams-only model and phrases-only model. We conclude that text mining with SEC financial document filings could be an effective tool to predict important corporate events such as SEO.

Achieving scalability of learning algorithms has become an increasingly critical issue in knowledge discovery from "big data". Sampling techniques can be exploited as one of the approaches to address the issue of scalability. We present in this paper a method to employ a newly developed sampling-based ensemble learning method by boosting for multi-class (non-binary) classification. This current research extends our previous work on multi-class classification with sampling-based ensemble learning method, in which the base classifiers are the most simplistic, such as decision stumps. Here we generalize the sampling-based method to handle more complex base classifiers such as decision trees in building an ensemble, which require sampling a set of instances before building a base classifier. We present experimental results using bench-mark data sets from the UC-Irvine ML data repository that confirm the efficiency and competitive prediction accuracy of the proposed adaptive boosting method for the multi-class classification task.

Scalability has become an increasingly important issue for data mining and knowledge discovery in this “big data” era. Random sampling can be used to tackle the problem of scalable learning. Adaptive sampling is superior to traditional batch sampling

With the advent of the "big data" era, the data mining community is facing an increasingly critical problem of developing scalable algorithms capable of mining knowledge from massive amount of data. This paper develops a sampling-based method to address the issue of scalability. We show how to utilize the new, adaptive sampling method in [4] to develop a scalable learning algorithm by boosting, an ensemble learning method. We present experimental results using bench-mark data sets from the UC-Irvine ML data repository that confirm the much improved efficiency and thus scalability, and competitive prediction accuracy of the new adaptive boosting method, in comparison with existing approaches.

Sampling is an important technique for parameter estimation and hypothesis testing widely used in statistical analysis, machine learning and knowledge discovery. Adaptive sampling offers advantages over traditional batch sampling methods in that adaptive sampling often uses much lower number of samples and thus better efficiency while assuring guaranteed level of estimation accuracy and confidence. In our previous works, a new adaptive sampling method was developed, and applied to build an efficient, scalable boosting learning algorithm. In this paper, we present a preliminary theoretical analysis of the proposed sampling method. A new variant of the sampling method is also presented. Empirical simulation results indicate that our methods, both the new variant and the original algorithm, often use significantly lower sample size (i.e., the number of sampled instances) while maintaining competitive accuracy and confidence when compared with batch sampling method.

Scalability has become an increasingly critical problem for successful data mining and knowledge discovery applications in real world where we often encounter extremely huge data sets that will render the traditional learning algorithms infeasible. Among various approaches to scalable learning, sampling techniques can be exploited to address the issue of scalability. This paper presents a brief outline on how to utilize the new sampling method in [3] to develop a scalable ensemble learning method with Boosting. Preliminary experimental results using benchmark data sets from the UC-Irvine ML data repository are also presented confirming the efficiency and competitive prediction accuracy of the proposed adaptive boosting method.

We recently proposed the Edgewise Greedy Algorithm (EGA) for learning a decomposable Markov network of treewidth k approximating a given joint probability distribution of n discrete random variables. The main ingredient of our algorithm is the stepwise forward selection algorithm (FSA) due to Deshpande, Garofalakis, and Jordan. EGA is an efficient alternative to the algorithm (HGA) by Malvestuto, which constructs a model of treewidth k by selecting hyperedges of order k+1. In this paper, we present results of empirical studies that compare HGA, EGA and FSA-K which is a straightforward application of FSA, in terms of approximation accuracy (measured by KL-divergence) and computational time. Our experiments show that (1) on the average, all three algorithms produce similar approximation accuracy; (2) EGA produces comparable or better approximation accuracy and is the most efficient among the three. (3) Malvestuto's algorithm is the least efficient one, although it tends to produce better accuracy when the treewidth is bigger than half of the number of random variabls; (4) EGA coupled with local search has the best approximation accuracy overall, at a cost of increased computation time by 50 percent.

Ontology of a domain mainly consists of concepts, hierarchical relations, and non-hierarchical relations. Even though there exists a variety of methods for extracting concepts and hierarchical relations, very little concentration is on identification and labeling of non-hierarchical relations. In this paper, we present an unsupervised technique for the identification of non-hierarchical relations between the concepts using VF*ICF metric and log-likelihood ratios. The proposed approach is experimented with the electronic voting domain texts and is also compared with one of the existing approaches

A Hidden Markov Model (HMM) is a probabilistic model that has been widely applied to a number of fields since its inception over 30 years ago. Computational Biology, Speech Recognition, and Image Processing are but a few of the application areas of HMMs. We propose an efficient algorithm for learning the parameters of a first order HMM from a state distribution (SD) oracle. The SD oracle provides the learner with the state distribution vector corresponding to a query string in the model. The SD oracle is shown to be necessary for polynomial-time learning in the sense that the consistency problem involving learning HMM parameters from a training set of state distribution vectors without the ability to query the SD oracle, is NP-complete. The learning algorithm proposed is based on an algorithm described by Tzeng for learning Probabilistic Automata.

In this paper, a hybrid decision tree learning approach is presented that combines fuzzy C-means method and the ID3 algorithm in decision tree construction from continuous-valued features. The fuzzy C-means method is applied to find a number of central means for each continuous-valued feature and thus discretize such features. The ID3 algorithm is subsequently used to build a decision tree from the discretized data. Preliminary experiments using a real-world time-series data set from the Louisiana coast are reported that compare our method with the OC1 system for oblique decision tree learning. The experiment results seem to suggest that the proposed hybrid method achieves better or comparable classification accuracy.

We present an integrated approach to information retrieval, which combines fuzzy clustering and fuzzy inference in order to improve textual retrieval performance. We capture the relationships among index terms by using fuzzy logic rules (with truth value assignment in [0,1]). We adapt fuzzy clustering methods (e.g., fuzzy c-means and fuzzy hierarchical clustering) in order to cluster documents with respect to the terms. The clusters generated provide a basis for building fuzzy logic rules concerning the terms, and the clusters can also be used to form hyperlinks between documents. The fuzzy logic rules are applied via fuzzy inference in order to derive query modification. In addition, relevance feedback is discussed as an alternative way to employ the fuzzy clusters. We explore retrieving an entire fuzzy cluster in response to a query. Finally, we note the need to test this approach more thoroughly on a larger standard test bed. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

We present an integrated approach to information retrieval which combines fuzzy clustering and fuzzy inference techniques in order to achieve optimal retrieval performance. To capture the relationships among index terms, fuzzy logic rules are used. We adapt several fuzzy clustering methods to the task of clustering documents with respect to the index terms. The clusters generated provide a basis for building the fuzzy logic rules. The clusters can also be used to form hyperlinks between documents. The fuzzy logic rules are applied with fuzzy inference to derive useful modifications of the initial query, which will guide the search for relevant documents. Alternative ways to use the fuzzy clusters are explored in this work as well. Our method combines fuzzy clustering and fuzzy inference with traditional relevance feedback approach for retrieval. The advantage of this approach is the emphasis on semantic information which relates the terms through the fuzzy clusters and fuzzy rules. A series of experiments have been conducted in order to validate this approach; a description of those experiments along with the results are presented.

In this paper, we present a method for embedding prioritized circumscription of a clausal theory into general disjunctive logic programs (GDP) with negation as failure in the head. In recent works, Sakama and Inoue show that parallel circumscription can be embedded in GDP. They also show that prioritized circumscription of a clausal theory can be represented in the framework of GDP extended with priorities. Wang et al. recently have also shown that prioritized circumscription can be expressed in their priority logic by applying monotonic inference rules constrained by a priority relation among the rules. In our method, the priorities of minimization in the circumscription policy are translated into the syntactical form of logic program rules, and the models of the circumscription are precisely captured by the stable models of the program. Thus we show that prioritized circumscription can be directly embedded in GDP without extending the GDP logic programming framework. This result further asserts the expressive power of the class of general disjunctive programs and supports its use for knowledge representation in Artificial Intelligence. The results of this paper also lead to embeddings of prioritized circumscription into the MBNF logic of Lifschitz and AE logic of Moore.

We present a sound and complete proof theory for the generalized logic of only knowing (GOL). The GOL logic is a modal logic proposed recently (Chen 1997) as a unified logical framework for non-monotonic reasoning, which extends the logic of only knowing (OL) of Levesque (1990), and covers the important notion of epistemic specification (Gelfond 1991, Gelfond and Przymusinska 1993) which is very useful for knowledge representation. The OL logic has been shown to have close connections with Lifschitz's minimal belief and negation as failure (Lifschitz 1994) and various other major non-monotonic formalisms. The GOL logic is more general and powerful than OL. The completeness result for the GOL proof theory establishes the theoretical foundation for answering queries in epistemic theories. This work is restricted to the propositional case only.

We propose here a new notion of a minimal model for forming circumscription in belief-logic by incorporating the notion of minimal models in the prepositional logic. This results in a new circumscription operation for belief-formulas than the one considered in [4-6]. An important feature of the new circumscription operation is its close relationship with the circumscription of prepositional formulas. For instance, we have CIRC[Bφ] = B(CIRC[φ]), where φ is a prepositional formula. We give several examples to show that the new notion of circumscription fits quite well with intuition.

GOL, the generalized logic of only knowing, is defined and its use proposed as a unified framework for non-monotonic reasoning. The GOL logic is a generalization of logic of Levesque's only knowing (OL) and it covers Gelfond's important notion of epistemic specification (ES), which is very useful for knowledge representation. By giving a model-theoretic account of epistemic specifications, the GOL helps clarify the conceptual understanding of epistemic theories. The GOL logic contains the OL logic as a subset and thus it retains the important features of OL The OL logic is a modal logic which can be used to formalize an agent's introspective reasoning and to answer epistemic queries to databases. The relations between OL logic and MBNF (the logic of minimal belief and negation as failure by Lifschitz), and between OL and extended logic programs have recently been established and it has been shown that OL is a fairly general logical framework for non-monotonic reasoning. In this paper, the OL logic is further generalized to enhance the expressive power to include epistemic specifications. A sound proof theory for GOL is also presented. This work is restricted to the prepositional case only.

Computing circumscription of a first-order theory is difficult because it involves, in general, a second-order quantifier. It is therefore important to study the cases where the circumscription can be expressed as a first-order theory. Lifschitz and Rabinov have previously shown that the class of separable theories and the class of collapsible theories have this property. Here, we introduce a new class of theories δ, called finitary theories, for which the circumscription CIRC(δ,P,Q) is given by a first order theory δ ∪ δpc, where δpc is the set of predicate completion formulas for predicates in P defined in a suitable way. There are many finitary theories which are interesting and which do not belong to the class of separable or collapsible theories.

We propose to use the logic of only knowing (OL) by Levesque [10] as a unified framework that encompasses various non-monotonic formalisms and logic programming. OL is a modal logic which can be used to formalize an agent's introspective reasoning an

In this paper, we relate Lifschitz’s logic of minimal belief and negation as failure (MBNF) to Levesque’s logic of only knowing (OL). MBNF can be viewed as an integration of Lin and Shoham’s logic of grounded knowledge and the theory of epistemic queries by Levesque and Reiter. Lifschitz showed that MBNF can be used as a general framework to compare different non-monotonic formalisms such as default logic and circumscription, as well as several forms of logic programs, including disjunctive logic programs with classical negation. Levesque’s logic of only knowing is an epistemic formalism which can be used to model agent’s knowledge and belief and to answer epistemic queries. The OL logic allows one to express the statement is all that is known’. Levesque showed that autoepistemic logic of Moore can be embedded in OL and he also gave a proof theory for OL. We show that a substantial subset of propositional MBNF can be embedded in OL (and hence in autoepistemic logic). In particular, the class of theories in MBNF which corresponds to the translation of various logic programs, a large subclass of default theories, as well as circumscription, can be embedded in OL. This result has two implications: (1) it shows that the logic of only-knowing (OL) can also be used as a general framework that encompasses various kinds of non-monotonic reasoning formalisms and several forms of logic programs. In particular, it shows that extended logic programs with classical negation can be embeded in autoepistemic logic, due to the close connection between autoepistemic logic and OL; (2) for a large class of theories in MBNF, the query-answering task (and thus the inferencing task in the related formalisms) can be reduced to inferencing in OL. Since the proof theory for the propositional OL is sound and complete, this gives a sound and complete proof theory for a large sub-class of propositional MBNF. We also expect that the proof theory for the general OL logic (with predicates and equality) will give a sound inference procedure for answering queries in first order MBNF theories. © 1994 Taylor & Francis.

An approach is presented to the optimization of decision trees. A decision tree is considered optimal if it correctly classifies the known data set and has the minimal number of nodes. It is shown that it is important to decide the right order of attributes to test, for this can reduce the number of checking nodes in a decision tree.>

The concept of extension plays an important role in default logic. The notion of an ordered seminormal default theory has been introduced (Etherington 1987) to characterize a class of seminormal default theories which have extensions. However, the original definition has a drawback because of its dependence on specific representations of the default theory. We introduce the ‘canonical representation’ of a default theory and redefine the orderedness of a default theory based on its canonical representation. We show that under the new definition, the orderedness of a default theory Δ = (W,D) is intrinsic to the theory itself, independent of the specific representations of W and D. We present a modification of the algorithm in Etherington (1987) for computing extensions of a default theory. More importantly, we prove the conjecture (Etherington 1987) that a modified version of the algorithm in Etherington (1987) converges for general ordered, finite seminormal default theories, while the original algorithm was proven (Etherington 1987) to converge for ordered, finite network default theories which form a proper subset of the theories considered in this paper.

The authors present a system, called SAFE (strategy acquisition from experience), which incorporates novel methods for discovering domain-dependent problem-solving heuristics. SAFE is implemented as a sorting system whose sorting strategies are represented as production rules. SAFE initially uses the insertion sort strategy to solve problems. After solving each given problem. SAFE learns symbolic rules from the solution path which is obtained by applying the existing heuristic information. By one or several processes of learning. SAFE is able to obtain the heuristics to sort new problems with minimum exchanges of elements. The notion of shortcut, an effective inductive learning bias for reducing the hypothesis space to be searched during learning, is introduced.>