Automated sortation has emerged as a major trend in logistics, enabling scalable and time-efficient sorting of items through the deployment of robotic agents. This study examines the use of capacity-enhanced agents in automated sorting domain, by evaluating task assignment strategies that improve the Token Passing with Multiple Capacity (TPMC) algorithm for the Multi-Agent Pickup and Delivery with Capacities (MAPDC) problem. In a simulated sorting domain, eight methods leveraging agent waypoint information are evaluated against the commonly used nearest-pickup task selection method. The results show that task assignment heuristics incorporating Closeness Centrality, Hausdorff Distance, and cost-based measures significantly improve solution quality of path planning in sortation systems. Moreover, the gains are greater in the sortation domain with disjoint pickup and delivery areas than in automated warehouse settings without spatial constraints, highlighting the importance of environment structure in MAPDC.

Reinforcement Learning (RL) agents in partially observable environments need some form of memory to make effective decisions. Current methods either store the entire history, which greatly increases the learning complexity, or use blackbox approaches like LSTM layers, which lack interpretability. In this paper, we introduce a simple and general external memory framework for RL agents in visual tasks, called Visual Self-Memory Management (VSMM). VSMM enables agents to manage their external memory while encouraging effective usage through intrinsic motivation. The intrinsic reward is based on the reconstruction error of an autoencoder, which highlights novel observations and diminishes over time, maintaining a balance between exploration and exploitation. Our experiments demonstrate that VSMM is good at sample efficiency, adapts better to new tasks, and offers interpretable and reliable memory management.

Various methods have been proposed in the literature for identifying subgoals in discrete reinforcement learning (RL) tasks. Once subgoals are discovered, task decomposition methods can be employed to improve the learning performance of agents. In this study, we classify prominent subgoal identification methods for discrete RL tasks in the literature into the following three categories: graph-based, statistics-based, and multi-instance learning (MIL)-based. As contributions, first, we introduce a new MIL-based subgoal identification algorithm called EMDD-RL and experimentally compare it with a previous MIL-based method. The previous approach adapts MIL’s Diverse Density (DD) algorithm, whereas our method considers Expected-Maximization Diverse Density (EMDD). The advantage of EMDD over DD is that it can yield more accurate results with less computation demand thanks to the expectation-maximization algorithm. EMDD-RL modifies some of the algorithmic steps of EMDD to identify subgoals in discrete RL problems. Second, we evaluate the methods in several RL tasks for the hyperparameter tuning overhead they incur. Third, we propose a new RL problem called key-room and compare the methods for their subgoal identification performances in this new task. Experiment results show that MIL-based subgoal identification methods could be preferred to the algorithms of the other two categories in practice.

The multi-agent pathfinding (MAPF) problem is defined as finding conflict-free paths for more than one agent. There exist optimal and suboptimal solvers for MAPF, and most of the solvers focus on the MAPF problem in static environments, but the real world is far away from being static. Motivated by this requirement, in this paper, we introduce an incremental algorithm to solve MAPF. We focused on discrete-time and discrete space environments with the unit cost for all edges. We proposed an algorithm called incremental multi-agent path planning with LPA* (LIMP) and discrete lifelong planning A* (DLPA*) for solving I-MAPF (Incremental MAPF). LIMP is the combination of two algorithms which are the Conflict Based Search D*-lite (CBS-D*lite) (Semiz and Polat, 2021) and DLPA*. DLPA* is just a tailored version of the lifelong planning A* (Koenig et al., 2004) which is an incremental search algorithm for one agent. We have shown that LIMP outperforms Conflict Based Search replanner (CBS-replanner) and CBS-D*-lite (Semiz and Polat, 2021) in terms of speed. Moreover, in terms of cost, LIMP and CBS-D*-lite perform similarly, and they are close to CBS-replanner.

In most cases, feature sets available for machine learning algorithms require a feature engineering approach to pick the subset for optimal performance. During our link prediction research, we had observed the same challenge for features of Location Based Social Networks (LBSNs). We applied multiple reduction approaches to avoid performance issues caused by redundancy and relevance interactions between features. One of the approaches was the custom two-step method; starts with clustering features based on the proposed interaction related similarity measurement and ends with non-monotonically selecting optimal feature subset from those clusters. In this study, we applied well-known generic feature reduction algorithms together with our custom method for LBSNs to evaluate novelty and verify the contributions. Results from multiple data groups depict that our custom feature reduction approach makes higher and more stable effectivity optimizations for link prediction when compared with others.

One of the most attractive problems of social network analysis is the link prediction. Social networks' user growth is mostly supported with data driven friend recommendations which are provided by link predictors. Previously, we had studied new features to improve prediction accuracy in Location Based Social Networks (LBSNs) where users share temporal location information with check-in interactions. In this paper, we focused on the efficiency of link predictors as the speed of prediction is as critical as its accuracy in LBSNs. Extraction time costs and prediction accuracy of individual LBSN features are mined to pick a feature subset that is achieving faster link prediction while not losing from accuracy.

BACKGROUND: Drug repositioning is the process of identifying new targets for known drugs. It can be used to overcome problems associated with traditional drug discovery by adapting existing drugs to treat new discovered diseases. Thus, it may reduce associated risk, cost and time required to identify and verify new drugs. Nowadays, drug repositioning has received more attention from industry and academia. To tackle this problem, researchers have applied many different computational methods and have used various features of drugs and diseases. RESULTS: In this study, we contribute to the ongoing research efforts by combining multiple features, namely chemical structures, protein interactions and side-effects to predict new indications of target drugs. To achieve our target, we realize drug repositioning as a recommendation process and this leads to a new perspective in tackling the problem. The utilized recommendation method is based on Pareto dominance and collaborative filtering. It can also integrate multiple data-sources and multiple features. For the computation part, we applied several settings and we compared their performance. Evaluation results show that the proposed method can achieve more concentrated predictions with high precision, where nearly half of the predictions are true. CONCLUSIONS: Compared to other state of the art methods described in the literature, the proposed method is better at making right predictions by having higher precision. The reported results demonstrate the applicability and effectiveness of recommendation methods for drug repositioning.

Following publication of the original article [1], the authors reported that there was an error in the spelling of the name of one of the authors.

Rapid development and wide usage of social networking applications have enabled large amounts of valuable data which can be analyzed for various reasons by companies, governments, non-profit organizations such as UN. This paper presents an evolutionary approach for detecting communities in social networks. We formulated a genetic algorithm that does not require the number of communities as input and is able to detect communities effectively in a very fast way. The performance of the proposed method is compared to its counterparts in order to show that good results can be generated. Additionally, we have done experiments using Newman's Spectral Clustering Method as a pre-processing step and it gave much better results.

Instance-based methods are proven tools to solve reinforcement learning problems with hidden states. Nearest Sequence Memory (NSM) is a widely known instance-based approach mainly based on k-Nearest Neighbor algorithm. It keeps the history of the agent in terms of action-observation-reward tuples and uses it to vote for the best upcoming action. In this work, an improving heuristic is proposed for the NSM algorithm which provides the agent an additional prior information, namely transitional bottlenecks, on the way to goal. Additionally, a tuple extension pattern is shown to further improve the heuristic by means of ambiguity reduction due to the nature of transitional bottlenecks, thus increase the learning speed. Empirical results indicate a significant improvement in learning performance, in terms of number of steps to goal.

In the reinforcement learning context, subgoal discovery methods aim to find bottlenecks in problem state space so that the problem can naturally be decomposed into smaller sub-problems. In this paper, we propose a concept filtering method that extends an existing subgoal discovery method, namely diverse density, to be used for both fully and partially observable RL problems. The proposed method is successful in discovering useful subgoals with the help of multiple instance learning. Compared to the original algorithm, the resulting approach runs significantly faster without sacrificing the solution quality. Moreover, it can effectively be employed to find observational bottlenecks of problems with perceptually aliased states.

External control of gene regulatory networks (GRNs) has received much attention in recent years. The aim is to find a series of actions to apply to a gene regulation system making it avoid its diseased states. In this work, we propose a novel method for controlling partially observable GRNs combining batch mode reinforcement learning (Batch RL) and TD() algorithms. Unlike the existing studies inferring a computational model from gene expression data, and obtaining a control policy over the constructed model, our idea is to interpret the time series gene expression data as a sequence of observations that the system produced, and obtain an approximate stochastic policy directly from the gene expression data without estimation of the internal states of the partially observable environment. Thereby, we get rid of the most time consuming phases of the existing studies, inferring a model and running the model for the control. Results show that our method is able to provide control solutions for regulation systems of several thousands of genes only in seconds, whereas existing studies cannot solve control problems of even a few dozens of genes. Results also show that our approximate stochastic policies are almost as good as the policies generated by the existing studies.

The day mankind met with smartphones, a new era started. Since then, daily mobile internet usage rates are increasing everyday and people have developed new habits like frequently sharing information (photo, video, location, etc.) on online social networks. Location Based Social Networks (LBSNs) are the platforms that empowers users to share place/location information with friends. As all other social networks, LBSNs aim to acquire more users with a smart friend recommendation. Solution for smart friend recommendation problem is studied under link prediction field by researchers. Check-in information is the main data for link prediction in LBSNs. Data extracted from check-in information plays vital role for predictor performance. In this study, we attempt to make use of detailed analysis of place category in order to exploit possible information gain enhancements through such semantic information. We proposed two new feature groups; Common Place Check-in Count Product Sum and Common Category Check-in Count Sum Product. For any link candidate pair; those features are calculated for each category. Use of new features improved the link prediction performance for multiple data subsets.

Social networks and location based social networks have many active users who provide various kind of data, such as where they have been, who their friends are, which items they like more, when they go to a venue. Location, social network and temporal information provided by them can be used by recommendation systems to give more accurate suggestions. Also, recommendation systems can provide dynamic recommendations based on the users' preferences, such that they can give different recommendations for different hours of the day or different days of the week. In this paper, we propose a recommendation system which considers the users' temporal preference to give dynamic recommendation. The recommendation method uses multi-objective optimization approach and gives point of interest (POI) recommendation using several different criteria, namely past check-in locations, hometown of users, time of check-ins, friendship and influence among users.

Options framework is a prominent way to improve learning speed by means of temporally extended actions, called options. Although various attempts focusing on how to derive high quality termination conditions for options exist, the impact of initiation set generation of an option is relatively unexplored. In this work, we propose an effective heuristic method to derive useful initiation set elements via an analysis of the recent history of events.

The need for determining a path from an initial location to a target one is a crucial task in many applications, such as virtual simulations, robotics, and computer games. Almost all of the existing algorithms are designed to find optimal or suboptimal solutions considering only a single objective, namely path length. However, in many real life application path length is not the sole criteria for optimization, there are more than one criteria to be optimized that cannot be transformed to each other. In this paper, we introduce a novel multiobjective incremental algorithm, multiobjective D* lite (MOD* lite) built upon a well-known path planning algorithm, D* lite. A number of experiments are designed to compare the solution quality and execution time requirements of MOD* lite with the multiobjective A* algorithm, an alternative genetic algorithm we developed multiobjective genetic path planning and the strength Pareto evolutionary algorithm.

Web-based platforms, such as social networks, review web-sites, and e-commerce web-sites, commonly use recommendation systems to serve their users. The common practice is to have each platform captures and maintains data related to its own users. Later the data is analyzed to produce user specific recommendations. We argue that recommendations could be enriched by considering data consolidated from multiple sources instead of limiting the analysis to data captured from a single source. Integrating data from multiple sources is analogous to watching the behavior and preferences of each user on multiple platforms instead of a limited one platform based vision. Motivated by this, we developed a recommendation framework which utilizes user specific data collected from multiple platforms. To the best of our knowledge, this is the first work aiming to make recommendations by consulting multiple social networks to produce a rich modeling of user behavior. For this purpose, we collected and anonymized a specific dataset that contains information from BlogCatalog, Twitter and Flickr web-sites. We implemented several different types of recommendation methodologies to observe their performances while using single versus multiple features from a single source versus multiple sources. The conducted experiments showed that using multiple features from multiple social networks produces a wider perspective of user behavior and preferences leading to improved recommendation outcome.

Gene regulatory networks (GRNs) are composed of biological components, including genes, proteins and metabolites, and their interactions. In general, computational methods are used to infer the connections among these components. However, computational methods should take into account the general features of the GRNs, which are sparseness, scale-free topology, modularity and structure of the inferred networks. In this work, observing the common aspects between recommendation systems and GRNs, we decided to map the GRNs inspiring problem into a recommendation problem and then used a known recommendation method to predict gene relationships based on multiple data sources, e.g., which molecules regulate others. The method we used is based on Pareto dominance and collaborative filtering. For the experiments, we used a combination of two datasets, namely microarray data and transcription factor (TF) binding data. The reported results show that using information from multiple sources improves the performance. Also, we observed that employing an approach from the recommendation systems domain revealed interesting results and good performance.

Temporal abstraction for reinforcement learning (RL) aims to decrease learning time by making use of repeated sub-policy patterns in the learning task. Automatic extraction of abstractions during RL process is difficult but has many challenges such as dealing with the curse of dimensionality. Various studies have explored the subject under the assumption that the problem domain is fully observable by the learning agent. Learning abstractions for partially observable RL is a relatively less explored area. In this paper, we adapt an existing automatic abstraction method, namely extended sequence tree, originally designed for fully observable problems. The modified method covers a certain family of model-based partially observable RL settings. We also introduce belief state discretization methods that can be used with this new abstraction mechanism. The effectiveness of the proposed abstraction method is shown empirically by experimenting on well-known benchmark problems.

Social networks, personal blog pages, on-line transaction web-sites, expertise web pages and location based social networks provide an attractive platform for millions of users to share opinions, comments, ratings, etc. Having this kind of diverse and comprehensive information leads to difficulti

It is difficult for the users to reach the most appropriate and reliable item for them among vast number of items and comments on these items. Recommendation systems and trust/reputation systems are one of the solutions to deal with this problem with the help of personalized services. These systems suggest items to the user by estimating the ratings that user would give to them. Use of trust data for giving recommendation has emerged as a new way for giving better recommendations. In the literature, it is shown that trust based recommendation approaches perform better than the ones that are only based on user similarity, or item similarity. In this paper, a comparative review of recommendation systems, trust/reputation systems, and their combined usage is presented. Then, a sample trust based agent oriented recommendation system is proposed and its effectiveness is justified with the help of some experiments.

Reinforcement learning with eligibility traces has been an effective way to solve problems with hidden state. Under certain conditions, it succeeds to build up a memoryless optimal policy over observations. Automatic generation of temporal abstractions, on the other hand, provides ways to extract and make use of useful sub-policies during reinforcement learning for a fully observable problem setting, so that the agent shall not need to repeatedly learn the same skill. One of the recent automatic abstraction techniques is the extended sequence tree method. We propose a novel way to bring together the extended sequence tree method and reinforcement learning for problems with hidden state. We expand the extended sequence tree method with a mechanism that helps the abstraction procedure to get rid of adverse effects of perceptual aliasing, letting the agent to make use of the remaining useful abstractions. Effectiveness of the method is shown empirically via experimentation on some benchmark problems.

The ever-growing need for gene-expression data analysis motivates studies in sample generation due to the lack of enough gene-expression data. It is common that there are thousands of genes but only tens or rarely hundreds of samples available. In this paper, we attempt to formulate the sample generation task as follows: first, building alternative Gene Regulatory Network (GRN) models, second, sampling data from each of them, and then filtering the generated samples using metrics that measure compatibility, diversity and coverage with respect to the original dataset. We constructed two alternative GRN models using Probabilistic Boolean Networks and Ordinary Differential Equations. We developed a multi-objective filtering mechanism based on the three metrics to assess the quality of the newly generated data. We presented a number of experiments to show effectiveness and applicability of the proposed multi-model framework.

Gene regulatory networks (GRNs) govern the protein transcription process in the cell and interactions among genes play a vital role in determining the biosynthesis rate of proteins. By using intervention techniques discovered by biological research it is possible to control a GRN, thus promoting or demoting the expression rate of a certain gene. In this work, this control task is studied in a partially observable setting where interventions lack perfect knowledge of the expression level of all genes. Moreover, we formulated the task as a lifelong control problem and developed a more flexible and scalable method than the alternatives described in the literature.

The past decades have witnessed advances in genomic technology; and this has allowed laboratories to generate vast amount of biological data, including microarray gene expression data. Effective analysis of the data helps in better understanding the mechanisms behind the complex behavior of the cell. Actually, a huge body of research focuses on the role of gene regulatory networks (GRNs) in controlling the cell. However, studying the heterogeneous interactions between mRNA and miRNA has received less attention. Fortunately, revealing the targets of miRNAs started to gain some consideration from the research community. Further, integrating mRNA gene expression and miRNA expression data is receiving more attention; the target is to understand the role of miRNA in regulating mRNA in different cell contexts; this could lead to predicting miRNA targets and constructing miRNA-mRNA interaction networks. On the other hand, we have already demonstrated the power of constraint-based learning as a promising technique to learn the structure of GRN , which are homogeneous in the sense that they contain one type of nodes, namely, genes. In this study, we extend our previous work to show how constraint-based learning can be effectively applied to tackle a more challenging problem, namely, to learn the structure of heterogeneous networks, like mRNA-miRNA network. In other words, to build the whole picture of the heterogeneous interactions, we used constraint-based learning algorithms which usually perform well on sparse graphs to predict the interactions within heterogeneous networks, namely, miRNA-mRNA interactions. We are able to achieve this by extending our PCPDPr algorithm, which works on homogeneous networks. The extended version named htrPCPDPr is capable of handling networks connecting two heterogeneous sets of nodes into a bipartite graph. This way, we propose a new learning mechanism to predict miRNA targets from expression profiles of both mRNA and miRNA, in addition to sequence-based prior knowledge about the interactions. The method has been applied to different set of genes related to the Alzheimer disease; the results reported in this paper demonstrate the novelty, applicability, and effectiveness of the proposed approach.

For certain domains, e.g. bioinformatics, producing more real samples is costly, error prone and time consuming. Therefore, there is a need for an intelligent automated process capable of substituting the real samples by artificial samples that carry the same characteristics as the real samples and hence could be used for running comprehensive testing of new methodologies. Motivated by this need, we describe a novel approach that integrates Probabilistic Boolean Network and genetic algorithm based techniques into a framework that uses some existing real samples as input and successfully produces new samples as output. The new samples will inspire the characteristics of the existing samples without duplicating them. This leads to diversity in the samples and hence a more rich set of samples to be used in testing. The developed framework incorporates two models (perspectives) for sample generation. We illustrate its applicability for producing new gene expression data samples, a high demanding area that has not received attention. The two perspectives employed in the process are based on models that are not closely related, the independence eliminates the bias of having the produced approach covering only certain characteristics of the domain and leading to samples skewed towards one direction. The produced results are very promising in showing the effectiveness, usefulness and applicability of the proposed multi-model framework.

Constraint-based structure learning algorithms generally perform well on sparse graphs. Although sparsity is not uncommon, there are some domains where the underlying graph can have some dense regions; one of these domains is gene regulatory networks, which is the main motivation to undertake the study described in this paper. We propose a new constraint-based algorithm that can both increase the quality of output and decrease the computational requirements for learning the structure of gene regulatory networks. The algorithm is based on and extends the PC algorithm. Two different types of information are derived from the prior knowledge; one is the probability of existence of edges, and the other is the nodes that seem to be dependent on a large number of nodes compared to other nodes in the graph. Also a new method based on Gene Ontology for gene regulatory network validation is proposed. We demonstrate the applicability and effectiveness of the proposed algorithms on both synthetic and real data sets.

Graph classification is important for different scientific applications; it can be exploited in various problems related to bioinformatics and cheminformatics. Given their graphs, there is increasing need for classifying small molecules to predict their properties such as activity, toxicity or mutagenicity. Using subtrees as feature set for graph classification in kernel methods has been shown to perform well in classifying small molecules. It is also well-known that feature selection can improve the performance of classifiers. However, most of the graph kernels are not selective in choosing which subtrees to include in the set of features. Instead, they use all subtrees of a certain property as their feature set. We argue that not all the latter features are needed for effective classification. In this paper, we investigate the effect of selecting subset of the subtrees as features for graph kernels, i.e., we try to identify and keep useful features; all the remaining subtrees are eliminated. A masking procedure, which boils down to feature selection, is proposed for classifying graphs. We conducted experiments on several molecule classification datasets; the results demonstrate the applicability and effectiveness of the proposed feature selection process.

Controlling gene regulatory networks (GRNs) is an important and hard problem. As it is the case in all control problems, the curse of dimensionality is the main issue in real applications. It is possible that hundreds of genes may regulate one biological activity in an organism; this implies a huge state space, even in the case of Boolean models. This is also evident in the literature that shows that only models of small portions of the genome could be used in control applications. In this paper, we empower our framework for controlling GRNs by eliminating the need for expert knowledge to specify some crucial threshold that is necessary for producing effective results. Our framework is characterized by applying the factored Markov decision problem (FMDP) method to the control problem of GRNs. The FMDP is a suitable framework for large state spaces as it represents the probability distribution of state transitions using compact models so that more space and time efficient algorithms could be devised for solving control problems. We successfully mapped the GRN control problem to an FMDP and propose a model reduction algorithm that helps find approximate solutions for large networks by using existing FMDP solvers. The test results reported in this paper demonstrate the efficiency and effectiveness of the proposed approach.

As the number of genes in a transcriptional regulatory network is large and the number of samples in biological data types is usually small, there is a need for integrating multiple data types for reverse engineering these networks. In this paper, we propose a method to integrate microarray gene expression, ChIP-chip and transcription factor binding motif data sets in a partial least squares regression model to derive transcription factors (TFs) -gene interactions. Both single and synergistic effects of TFs on the promoters are considered in the model. A method that dynamically updates the significance level based on ChIP-chip and binding motif data is proposed. The results evaluated by methods based on gene ontology demonstrate the effectiveness of the proposed approach.

In this correspondence, we address the problem of real-time moving target search in dynamic and partially observable environments, and propose an algorithm called real-time moving target evaluation search (MTES). MTES is able to detect the closed directions around the agent and determines the estimated best direction to capture a moving target avoiding the obstacles nearby. We have also developed a new prey algorithm (Prey-A*) to test the existing and our predator algorithms in our experiments. We have obtained an impressive improvement over moving target search, real-time target evaluation search, and real-time edge follow with respect to path length. Furthermore, we have also tested our algorithm against A*.

Control of Probabilistic Boolean Networks as models of gene regulation is an important problem; the solution may help researchers in various different areas. But as generally applies to control problems, the size of the state space in gene regulatory networks is too large to be considered for comprehensive solution to the problem; this is evident from the work done in the field, where only very small portions of the whole genome of an organism could be used in control applications. The Factored Markov Decision Problem (FMDP) framework avoids enumerating the whole state space by representing the probability distribution of state transitions using compact models like dynamic bayesian networks. In this paper, we successfully applied FMDP to gene regulatory network control, and proposed a model minimization method that helps finding better approximate policies by using existing FMDP solvers. The results reported on gene expression data demonstrate the applicability and effectiveness of the proposed approach.

Due to the complex structure and scale of gene regulatory networks, we support the argument that combination of multiple types of biological data to derive satisfactory network structures is necessary to understand the regulatory mechanisms of cellular systems. In this paper, we propose a simple but effective method of combining two types of biological data, namely microarray and transcription factor (TF) binding data, to construct gene regulatory networks. The proposed algorithm is based on and extends the well-known PC algorithm. Further, we developed a method for measuring the significance of the interactions between the genes and the TFs. The reported test results on both synthetic and real data sets demonstrate the applicability and effectiveness of the proposed approach; we also report the results of some comparative analysis that highlights the power of the proposed approach.

Scalability is one of the most important issues in control problems, including the control of gene regulatory networks. In this paper, we argue that it is possible to improve scalability of gene regulatory networks control by reducing the number of genes to be considered by the control policy; and consequently propose a novel method to estimate genes that are less important for control. The reported test results on real and synthetic data demonstrate the applicability and effectiveness of the proposed approach.

In this paper, we propose a novel approach to identify states with similar subpolicies and show how they can be integrated into the reinforcement learning framework to improve learning performance. The method utilizes a specialized tree structure to identify common action sequences of states, which are derived from possible optimal policies, and defines a similarity function between two states based on the number of such sequences. Using this similarity function, updates on the action-value function of a state are reflected onto all similar states. This allows experience that is acquired during learning to be applied to a broader context. The effectiveness of the method is demonstrated empirically.

Real-time path search is the problem of searching a path from a starting point to a goal point in real-time. In dynamic and partially observable environments, agents need to observe the environment to track changes, explore to learn unknowns, and search suitable routes to reach the goal rapidly. These tasks frequently require real-time search. In this paper, we address the problem of real-time path search for grid-type environments; we propose an effective heuristic method, namely a real-time edge follow alternative reduction method (RTEF-ARM), which makes use of perceptual information in a real-time search. We developed several heuristics powered by the proposed method. Finally, we generated various grids (random-, maze-, and U-type), and compared our proposal with real-time A*, and its extended version real-time A* with n-look-ahead depth; we obtained very significant improvements in the solution quality.

In this paper, we study the control problem and note that it is multi-objective by nature, and thus we develop an optimal multi-objective approach. Our approach includes formalizing components and identifying dimensions, resulting in few cases for concrete problem formulation. For a selected case, namely the finite control case, a single-objective from the literature and our multi-objective solutions are presented. It is demonstrated that the multi-objective solution avoids drawbacks of the single-objective solution, particularly the need for defining single objective out of many

The problem of identifying significantly differentially expressed genes for replicated microarray experiments is accepted as significant and has been tackled by several researchers. Patterns from Gene Expression (PaGE) and q-values are two of the well-known approaches developed to handle this problem. This paper proposes a powerful approach to handle this problem. We first propose a method for estimating the prior probabilities used in the first version of the PaGE algorithm. This way, the problem definition of PaGE stays intact and we just estimate the needed prior probabilities. Our estimation method is similar to Storey's estimator without being its direct extension. Then, we modify the problem formulation to find significantly differentially expressed genes and present an efficient method for finding them. This formulation increases the power by directly incorporating Storey's estimator. We report the preliminary results on the BRCA data set to demonstrate the applicability and effectiveness of our approach.

MOTIVATION: It is important to consider finding differentially expressed genes in a dataset of microarray experiments for pattern generation. RESULTS: We developed two methods which are mainly based on the q-values approach; the first is a direct extension of the q-values approach, while the second uses two approaches: q-values and maximum-likelihood. We present two algorithms for the second method, one for error minimization and the other for confidence bounding. Also, we show how the method called Patterns from Gene Expression (PaGE) (Grant et al., 2000) can benefit from q-values. Finally, we conducted some experiments to demonstrate the effectiveness of the proposed methods; experimental results on a selected dataset (BRCA1 vs BRCA2 tumor types) are provided. CONTACT: [email protected].

This paper proposes a co-operation framework for multiple role-based case-based reasoning (CBR) agents to handle the product recommendation problem for e-commerce applications. Each agent has different case structure with intersecting features and agents exploit all information related to the problem by co-operation, which is accomplished through the merge of distributed cases in order to form cases having better representation of the problem. The presented merge algorithm handles noisy distributed cases by negotiation on the difference values of the intersecting features. The role-based CBR agents merge the distributed cases by introducing a global heuristic function, which is used to evaluate the relevance of merged cases. The heuristic function exploits the relevancy of each merged case within the viewpoint of each agent and the satisfied/unsatisfied problem constraints. The viewpoint of an agent is represented by the value of consistency of distributed components of merged cases and agent s individual relevance values of the merged cases. Finally, the proposed framework has been tested for elective course recommendation.

This paper addresses the control formulation process for probabilistic boolean genetic networks. It is a major problem that has not been investigated enough yet. We argue that a monitoring stage is necessary after the control stage for providing guidance about the evolution of the investigated state. For this purpose, we developed methods for generating optimal control policies for each of the following five cases: finite control, infinite control, finite control-infinite monitoring, finite control-finite monitoring, and repeated finite control-finite monitoring. Our initial proposal was based on using action cost functions in the process. In this study, we propose Markov decision processes as an alternative to the action cost functions approach. We conducted experiments on two simple illustrative examples to demonstrate that the considered five cases are necessary, effective and really matter while developing optimal control policies; the obtained results are promising.

Cluster validity investigates whether generated clusters are true clusters or due to chance. This is usually done based on subsampling stability analysis. Related to this problem is estimating true number of clusters in a given dataset. There are a number of methods described in the literature to handle both purposes. In this paper, we propose three methods for estimating confidence in the validity of clustering result. The first method validates clustering result by employing supervised classifiers. The dataset is divided into training and test sets and the accuracy of the classifier is evaluated on the test set. This method computes confidence in the generalization capability of clustering. The second method is based on the fact that if a clustering is valid then each of its subsets should be valid as well. The third method is similar to second method; it takes the dual approach, i.e., each cluster is expected to be stable and compact. Confidence is estimated by repeating the process a number of times on subsamples. Experimental results illustrate effectiveness of the proposed methods.

This paper presents a multiagent-based approach to the identification and recognition of touching in free handwritten Hindi numerals. We do not restrict our domain to touching pairs; rather, we consider numerals with arbitrary number of digits. Two agents are presented in this paper. The first agent works directly on the scanned image of the original handwritten number. It locates possible touching based on the thickness of handwriting. The other agent works on the thinned image. It segments the image into four categories of segments and tries to locate possible touching based on the rules that govern the connection of segments to form digits. After each of the two agents applies its own rules and investigates candidate possible touching cases, and to increase touching recognition rate, the two agents negotiate and try to agree on the actual touching cases. The experiments conducted so far are promising and successful. The obtained results are very encouraging with a success factor of 92.7%.

Search is a fundamental problem‐solving method in artificial intelligence. Traditional off‐line search algorithms attempt to find an optimal solution whereas real‐time search algorithms try to find a suboptimal solution more quickly than traditional algorithms to meet real‐time constraints. In this work, a new multi‐agent real‐time search algorithm is developed and its effectiveness is illustrated on a sample domain, namely maze problems. Searching agents can see their environment with a specified visual depth and hence can partially observe their environment. An agent makes use of its partial observation to select a next move, instead of using only one‐move‐ahead information. Furthermore agents cooperate through a marking mechanism to be able to search different parts of the search space. When an agent selects its next move, it marks its direction of move before executing the move. When another agent comes to this position, it sees this mark and, if possible, moves in a different direction than the previously selected direction. In this way, marking helps agents coordinate their moves with other agents. Although coordination brings an overhead, from experiments we observe that this mechanism is effective in both search time and solution length in maze problems.

The paper presents two new multi-agent based domain independent coordination mechanisms for reinforcement learning. The first mechanism allows agents to learn coordination information from state transitions and the second one from the observed reward distribution. In this way, the latter mechanism tends to increase region-wide joint rewards. The selected experimented domain is Adversarial Food-Collecting World (AFCW), which can be configured both as single and multi-agent environments. Experimental results show the effectiveness of these mechanisms.

The paper addresses an important and vital problem within the general area of character recognition, namely the identification and recognition of touching in handwritten Hindi numerals. The basic idea is that while writing down numbers, it is possible to have adjacent digits touching each other. To handle this, we are developing a multi-agent system. So far, we have two agents, which are presented. The first agent works directly on the scanned image of the original handwritten number. It locates possible touching based on the thickness of handwriting. The other works on the thinned image. It segments the image into four categories of segments and tries to locate possible touching based on the rules that govern the connection of segments to form digits. After each of the two agents applies its own rules and investigates possible touching, and to increase touching recognition rate, the two agents negotiate and try to agree on the actual touching. The experiments carried out so far are promising and successful. The obtained results are very encouraging with a success factor of 92.70%.

Learning in a partially observable and nonstationary environment is still one of the challenging problems in the area of multiagent (MA) learning. Reinforcement learning is a generic method that suits the needs of MA learning in many aspects. This paper presents two new multiagent based domain independent coordination mechanisms for reinforcement learning; multiple agents do not require explicit communication among themselves to learn coordinated behavior. The first coordination mechanism is the perceptual coordination mechanism, where other agents are included in state descriptions and coordination information is learned from state transitions. The second is the observing coordination mechanism, which also includes other agents in state descriptions and additionally the rewards of nearby agents are observed from the environment. The observed rewards and agent's own reward are used to construct an optimal policy. This way, the latter mechanism tends to increase region-wide joint rewards. The selected experimented domain is adversarial food-collecting world (AFCW), which can be configured both as single and multiagent environments. Function approximation and generalization techniques are used because of the huge state space. Experimental results show the effectiveness of these mechanisms.

An object-algebra is presented as a formal query model for object-oriented data models. The algebra serves not only to access and manipulate the structure and behavior of objects, but it also supports the creation of new objects and the introduction of new relationships into the schema. It provides a more powerful and flexible tool than messages for effectively dealing with complex situations and meeting associative access requirements. Operands as well as the results of operations in the proposed algebra are formally characterized as a pair of sets—a set of objects capturing the states and a set of message expressions comprised of sequences of messages modeling the object behavior. The closure property is achieved in a natural way by letting the results of operations possess the same characteristics as the operands in an algebra expression. Some operators of the algebra resemble those of the relational algebra but with different syntax and semantics. Additional operators are introduced to complement them. A class is shown to posses the properties of an operand by defining a set of objects and deriving a set of message expressions for it. Furthermore, the result of an object algebra expression is shown to have the characteristics of a class whose superclass/subclass relationships with its operand class(es) can be established providing a mechanism to properly and persistently place it in the class lattice (schema).