The US Centers for Disease Control and Prevention (CDC), in 2019, designated Methicillin-resistant Staphylococcus aureus (MRSA) as a serious antimicrobial resistance threat. The risk of acquiring MRSA and suffering life-threatening consequences due to it remains especially high for hospitalized patients due to a unique combination of factors, including: co-morbid conditions, immuno suppression, antibiotic use, and risk of contact with contaminated hospital workers and equipment. In this paper, we present a novel generative probabilistic model, GenHAI, for modeling sequences of MRSA test results outcomes for patients during a single hospitalization. This model can be used to answer many important questions from the perspectives of hospital administrators for mitigating the risk of MRSA infections. Our model is based on the probabilistic programming paradigm, and can be used to approximately answer a variety of predictive, causal, and counterfactual questions. We demonstrate the efficacy of our model by comparing it against discriminative and generative machine learning models using two real-world datasets.

As the Age of Information has evolved over the last several decades, the demand for technology which stores, analyzes, and utilizes data has increased substantially. Countless industries such as the medical, the retail, and the aircraft rely on this technology to guide their decision making. In the present paper, we propose a hybrid machine learning algorithm consisting of Decision Trees and Neural Networks which can effectively and efficiently classify data of varying volume and variety. The structure of the hybrid algorithm consists of a decision tree where each node of the tree is a neural network trained to classify a specific category of the output using binary classification. The data with which we used to train and test the classification ability of our algorithm is the Federal Aviation Administration's (FAA's) Boeing 737 maintenance dataset which consists of 137,236 unique records each composed of 72 variables. We perform this by classifying the discrepancy, or cause, of the incident into whether or not the incident occurred during scheduled maintenance operations and then further classifying specific details relating to the incident. Our results indicate that our hybrid algorithm is able to effectively classify incidents with high accuracy and precision. Additionally the algorithm is able to identify the most significant inputs regarding a classification allowing for higher performance and greater optimization. This demonstrates the algorithm's applicability in real-world scenarios while also showcasing the benefits of combining decision trees and neural networks as opposed to using them individually.

Researchers often rely on the byline order in a publication to estimate relative contributions made by its authors, an assumption on which existing author contribution measures are based. This byline-based approach is, however, incompatible with the alphabetical author ordering, a practice still employed by many research fields. Manually requesting authors to state their contributions can overcome the limitation of the existing methods. Such approaches, however, require resource-intensive data acquisition and preprocessing, rendering them ungeneralizable to existing bodies of bibliographic records. The present paper proposed a possibility of order-independent automatic author contribution measure by focusing on distinguishing the main contributors from the rest of the authors using machine learning algorithms, bypassing the limitation of both the byline-based numerical author contribution methods and ungeneralizable manual approaches. The experiment validated the proposed approach by successfully classifying both the primary and the corresponding authors shown as the first and the last author without utilizing byline orders. The Random Forest classifier showed the best performances, successfully classifying the first author, the last author, and both with the accuracy of 0.90, 0.89, and 0.76 respectively.

Understanding the current research topics and their histories allow researchers to focus their capabilities on the current research trends. The field of topic evolution helps the understanding by automatically model and detect the set of shared research fields in the academic papers as topics. The authors propose a novel topic evolution method for identifying and predicting the emergence of new topics under the assumption that neighborhoods of new topics in the future have distinguishable structural features. Eight journals were selected from the Microsoft Academic Graph dataset, each representing topics networks with varying size, history, and research domains. Both retrospective classification and prospective prediction showed promising performance with classifications above 0.89 for six journals and coefficients of determination exceeding 0.95 for five journals. The result showed both the retrospective identification and the prospective prediction can be done, validating the assumption that topic evolution events can be predicted with a network-based approach.

Polysomnography (PSG) screening for obstructive sleep apnea (OSA) is time consuming. The OSA classification is very important for medical scientists and machine learning researchers. In the current work, we developed a classification method for electrocardiogram (ECG) data. The data set has two labels: sleep disorders or not. As a result, Active Learning is used as a classification technique. Data stream classification in a non-stationary environment is attaining more attention recently. It is a highly challenging task, since the concept drift and limited labeled data. Therefore, a classification model is needed to be struggling with concept drift detection and the need of labeled data. To solve these issues, we propose an efficient semi-supervised method in this paper which uses Active Learning to detect concept drift in an unsupervised way and Classifiers Ensemble to keep higher predictions combined with weighted majority voting. Experiments results on real-world and synthetic data-sets show the effectiveness of the proposed approach. For the initial experiment, we use an existing data set. The data set includes data for every 10 seconds, up to 6000 seconds, and 35 patients. We have used 80% of the data for training purposes and 20% of the data for testing purposes. Active Learning results show that our method can effectively detect OSA. The accuracy of the predicted result is 71%. Future research in this area will be to obtain data from hospitals and use our developed algorithms for OSA classification and prediction.

Journey by aircraft is the only option for long distance transportation and also one of the frequently used modes of transportation of passengers. As a result, safety of passengers and efficiency of the aircraft depend on maintaining efficient running conditions. Although many safety standards are followed in the design of the aircraft and thus there are fewer accidents, it is necessary to perform a thorough analysis to avoid risks that may occur during flight time. In the present work, we propose a maintenance strategy, Failing And Not Falling (F&!F), based on the Federal Aviation Administration (FAA) data in the United States. We work with the dataset of Boeing 737. The data consists of 72 features with 137,236 records which describe an aircraft accident or incident. These features are used to predict whether an incident will be identified during aircraft maintenance or during aircraft operation and what specific type of incident will occur. The prediction method is based on the integration of a decision tree and a unique neural network at each node of the decision tree. The results obtained using different architectures show how deep the neural networks should be, how to identify the relevant features, and the success of combining decision trees and neural networks. Moreover the neural networks and the decision tree approach also successfully identified the important features of maintenance. This method can be used for the maintenance of any data in multiple domains.

Recently, research in machine learning has become more reliant on data-driven approaches. However, understanding the general theory behind optimal neural network architecture is, arguably, just as important. With the proliferation of deep learning and neural networks, finding optimal neural network architecture is vital for both accuracy and performance. Recently, extensive research on neural network architecture has been performed [3],[4]. Additionally, while there has been plenty of research on hidden layer neural network architecture [2], activation functions are often not considered. In a network, an activation function defines the output of a neuron and introduces non-linearities into the neural network, enabling it to be a universal function approximator [12]. In terms of activation functions, one significant paper is Krizhevsky's seminole work on ImageNet classification and the creation of the ReLU activation function [1]. In the paper, Krizhevsky outlines the construction of an image recognition model using the Rectified Linear Unit activation function (ReLU) for the ImageNET LSVRC-2010 competition which outperformed the state-of-the-art image recognition systems at the time [1]. Since then, ReLU has increased in popularity. In their 2018 conference paper, Bircanoğlu and Arica, with the assistance of 231 distinct training procedures, found ReLU to be the best general activation function [12]. In addition to comparisons of activation functions, Nwankpa, Ijomah, Gachagan, and Marshall conducted a meta analysis of the field of research centered around activation functions and found ReLU to be the most popular activation function choice [5]. In terms of optimizers, gradient descent has historically been the most popular loss optimization algorithm, but with Kingma and Ba's 2014 paper [8], Adam: A Method for Stochastic Optimization, Adam optimizer is slowly becoming the industry standard [11]. In their paper, Kingma and Ba cleverly combine momentum descent, RMSprop, and Adagrad optimization into one algorithm, Adam (or adaptive moment estimation) [8]. In addition to Adam, there are plenty of other optimizers to choose from, including gradient descent, RMSprop [9], Adagrad [10], and Adadelta [7]. Recently, many breakthroughs have been made in terms of neural network performance, improved GPU performance and adaptation to deep learning tasks has created massive efficiency increases for the whole field of machine learning. Furthermore, as machine learning becomes increasingly optimized, the importance of efficiency improvements will continue to rise. Thus, understanding the optimal activation function and optimizer choice for a neural network is relevant. The goal of this paper is to make comparisons between activation functions, optimizers, and, more generally, entire neural network architectures, through measured error in a training environment. In this paper, we examine the performance of a wide variety of neural network configurations on randomly generated polynomial data sets of fixed degree. To do this, we compare various neural network activation functions and optimizers while controlling for hidden layer configurations and degree of the underlying polynomial dataset. Curiously, we find that the Sigmoid activation function is more accurate than ReLU and Tanh for regression tasks on low-featured polynomial data. We also reach the same conclusion regarding Stochastic Gradient Descent (SGD) in comparison to the Adam optimization function and Root Mean Square Propagation (RMSprop). Additionally, we observe that SGD is more efficient in the short term for finding local minimums than Adam or RMSprop; however, after sufficiently many epochs, performance differences between the optimizers vanished.

Single objective evolutionary constrained optimization has been widely researched by plethora of researchers in the last two decades whereas multi-objective constraint handling using evolutionary algorithms has not been actively proposed. However, real-world multi-objective optimization problems consist of one or many non-linear and non-convex constraints. In the present work, we develop an evolutionary algorithm based on hybrid constraint handling methodology (HyCon) to deal with constraints in bi-objective optimization problems. HyCon is a combination of an Evolutionary Multi-objective Optimization (EMO) coupled with classical weighted sum approach and is an extended version of our previously developed constraint handling method for single objective optimization. A constrained bi-objective problem is converted into a tri-objective problem where the additional objective is formed using summation of constrained violation. The performance of HyCon is tested on four constrained bi-objective problems. The non-dominated solutions are compared with a standard evolutionary multi-objective optimization algorithm (NSGA-II) with respect to hypervolume and attainment surface. The simulation results illustrates the effectiveness of the HyCon method. The HyCon either outperformed or produced similar performance as compared to NSGA-II.

A robust probabilistic constraint handling approach in the framework of joint evolutionary-classical optimization has been presented earlier. In this work, the theoretical foundations of the method are presented in detail. The method is known as bi-objective method, where the conventional penalty function approach is implemented. The present work highlights the dynamic variation of the commensurate penalty parameter for each objective treated as constraint. It is shown that the constraint parameters collectively define the right slope of the tangent as to the optimal front during the search. The robust and sustained convergence throughout the search up to micro level in the range of 10-10or beyond is explained. The work here is presented as a further note in connection with the previous publication, where the subtle theoretical considerations and their details had been omitted for the sake of detailed results of the experiments demonstrating the effective working of the approach. In contrast to the implementation-centered reporting of the previous work, this work can be considered as a description of the detailed probabilistic basis underlying the previous work. Therefore, this study is of great importance to let the researchers conveniently gain the insight into the work and its implications reported earlier.

Robot gripper design is an active research area due to its wide spread applicability in automation, especially for high-precision micro-machining. This paper deals with a multiobjective optimization problem which is nonlinear, multimodal, and originally formulated. The previous work, however, had treated the actuator as a blackbox. The system model has been modified by integrating an actuator model into the robotic gripper problem. A generic actuation system (for example, a voice coil actuator) which generates force proportional to the applied voltage is considered. The actuating system is modeled as a stack consisting of the individual actuator elements arranged in series and parallel arrays in four different combinations. With the incorporation of voltage into the problem, which is related to both actuator force and manipulator displacement, the problem becomes more realistic and can be integrated with many real-life gripper simulations. Multiobjective evolutionary algorithm is used to solve the modified biobjective problem and to optimally find the dimensions of links and the joint angle of a robot gripper. A force voltage relationship can be obtained from each of the nondominated solutions which helps the user to determine the voltage to be applied depending on the application. An innovization study is further carried out to find suitable relationships between the decision variables and the objective functions.

Robot gripper design is an active research area due to its wide spread applicability in automation. The present work deals with the actuator analysis of a non-linear, multi-modal and multi-objective optimization problem which is originally formulated by Osyczka [1]. The previous work [1] had treated the actuator as a blackbox. In the present work, the formulation has been changed by incorporating actuator analysis into the robotic gripper problem. The piezoelectric actuating system is modeled as a stack consisting of the individual actuator elements arranged in series and parallel, leading to four different cases. With the incorporation of voltage as an input (it is related to both the actuator force and the manipulator displacement), the problem becomes more realistic and can be integrated with any real life situation. The relationships connecting the force developed and the voltage applied are arrived at by using the basic constitutive equations and the geometry of the problem. The future work aims to take up an existing optimization problem with actuator analysis and make it more realistic and wider in scope, by incorporating actuator analysis into it.

Constrained optimization is one of the popular research areas since constraints are usually present in most real world optimization problems. The purpose of this work is to develop a gradient free constrained global optimization methodology to solve this type of problems. In the methodology proposed, the single objective constrained optimization problem is solved using a Multi-Objective Evolutionary Algorithm (MOEA) by considering two objectives simultaneously, the original objective function and a measure of constraint violation. The MOEA incorporates a penalty function where the penalty parameter is estimated adaptively. The use of penalty function method will enable to further improve the current best solution by decreasing the level of constraint violation, which is made using a gradient free local search method. The performance of the proposed methodology was assessed on a set of benchmark test problems. The results obtained allowed to conclude that the present approach is competitive when compared with other methods available.

The holy grail of constrained optimization is the development of an efficient, scale invariant and generic constraint handling procedure in single and multi-objective constrained optimization problems. In this paper, an individual penalty parameter based methodology is proposed to solve constrained optimization problems. The individual penalty parameter approach is a hybridization between an evolutionary method, which is responsible for estimation of penalty parameters for each constraint and the initial solution for local search. However the classical penalty function approach is used for its convergence property. The aforesaid method adaptively estimates penalty parameters linked with each constraint and it can handle any number of constraints. The method is tested over multiple runs on six mathematical test problems and a engineering design problem to verify its efficacy. The function evaluations and obtained solutions of the proposed approach is compared with three of our previous results. In addition to that, the results are also verified with some standard methods taken from literature. The results show that our method is very efficient compared to some recently developed methods.

Optimization for single main objective with multi constraints is considered using a probabilistic approach coupled to evolutionary search. In this approach the problem is converted into a bi-objective problem, treating the constraint ensemble as a second objective subjected to multi-objective optimization for the formation of a Pareto front, and this is followed by a local search for the optimization of the main objective function. In this process a novel probabilistic modeling is applied to the constraint ensemble, so that the stiff constraints are effectively taken care of, while the model parameter is adaptively determined during the evolutionary search. In this way the convergence to the solution is significantly accelerated and an accurate solution is established. The improvements are demonstrated by means of example problems including comparisons with the standard benchmark problems, the solutions of which are reported in the literature.

A hybrid adaptive normalization based constraint handling approach is proposed in the present study. In most constrained optimization problems, constraints may be of different scale. Normalization of constraints is crucial for the efficient performance of a constraint handling algorithm. A growing number of researchers have proposed different strategies using bi-objective methodologies. Classical penalty function approach is another common method among both evolutionary and classical optimization research communities due to its simplicity and ease of implementation. In the present study, we propose a hybrid approach of both bi-objective method and the penalty function approach where constraints are normalized adaptively during the optimization process. The proposed bi-objective evolutionary method estimates the penalty parameter and the starting solution needed for the penalty function approach. We test and compare our algorithm on seven mathematical test problems and two engineering design problems taken from the literature. We compare our obtained results with our previous studies in terms of function evaluations and solution accuracy. The obtained optima are also compared with those of other standard algorithms. In many cases, our proposed methodology perform better than all algorithms considered in this study. Results are promising and motivate further application of the proposed adaptive normalization strategy.

This paper is concerned with the determination of optimum forces extracted by robot grippers on the surface of a grasped rigid object -- a matter which is crucial to guarantee the stability of the grip without causing defect or damage to the grasped object. A multi-criteria optimization of robot gripper design problem is solved with two different configurations involving two conflicting objectives and a number of constraints. The objectives involve minimization of the difference between maximum and minimum gripping forces and simultaneous minimization of the transmission ratio between the applied gripper actuator force and the force experienced at the gripping ends. Two different configurations of the robot gripper are designed by a state-of-the-art algorithm (NSGA-II) and the obtained results are compared with a previous study. Due to presence of geometric constraints, the resulting optimization problem is highly non-linear and multi-modal. For both gripper configurations, the proposed methodology outperforms the results of the previous study. The Pareto-optimal solutions are thoroughly investigated to establish some meaningful relationships between the objective functions and variable values. In addition, it is observed that one of the gripper configurations completely outperforms the other one from the point of view of both objectives, thereby establishing a complete bias towards the use of one of the configurations in practice.

Machining parameters optimization is very crucial in any machining process. This research focuses on Multi-objective Evolutionary Algorithm based optimization technique, to determine optimal cutting parameters (cutting speed, feed, and depth of cut) in turning operation. Two conflicting objectives (operation time and tool life) with three constraints, which depends on the turning parameters, are optimized using Genetic algorithm (GAs). The Pareto-optimal front of the bi-objective problem is obtained using Non-dominated Sorting Genetic Algorithm (NSGA-II). The extreme and intermediate points of Pareto optimal front is verified using Real coded Genetic Algorithm (RGA) as well as ε-constraint method. The performance of NSGA-II is found to be more effective and efficient as compared to micro-GA. Innovization study carried out to correlate cutting parameters with the aforementioned objective functions. The effect of cutting speed is found more as compared to feed rate and depth of cut.

Evolutionary algorithms are modified in various ways to solve constrained optimization problems. Of them, the use of a bi-objective evolutionary algorithm in which the minimization of the constraint violation is included as an additional objective, has received a significant attention. Classical penalty function approach is another common methodology which requires an appropriate knowledge of the associated penalty parameter. In this paper, we combine a bi-objective evolutionary approach with the penalty function methodology in a manner complementary to each other. The bi-objective optimization approach provides a good estimate of the penalty parameter, while the unconstrained penalty function approach using classical means provides the overall hybrid algorithm its convergence property. We demonstrate the working of the procedure on a two-variable problem and then solve a number of standard numerical test problems from the EA literature. In all cases, our proposed hybrid methodology is observed to take one or more orders of magnitude smaller number of function evaluations to find the constrained minimum solution accurately. To the best of our knowledge, no previous evolutionary constrained optimization algorithm has reported such a fast and accurate performance on the chosen problems.

Genetic Algorithms (GAs) are a highly successful population based approach to solve global optimization problems. They have carved out a niche for themselves in solving optimization problems of varying difficulty levels involving single and multiple objectives. Most real-world optimization problems involve equality and / or inequality constraints and hence posed as constrained optimization problems. The most common approach to solve such problems using GAs is the method of penalty functions, which however suffers from the drawback of appropriate selection of penalty parameters for their optimal functioning. Given the nature of the problems at hand, we have used an adaptive mutation based Real-Coded GA (RGA), which uses a popular penalty parameter-less approach to handle constraints and search the feasible region effectively for the global best solution, and at the same time use an adaptive mutation strategy to maintain diversity in the population to enable creation of new solutions. We have coupled our RGA with ideas from the gradient projection method to specifically handle equality constraints. We have found our simple procedure working quite well in most of the test problems provided as part of the competition on Single-objective Constrained Real Parameter Optimization in CEC 2010 and hence simplicity remains the hallmark of our study here.