In this study, we propose two methods to estimate static graphs from a single dynamic graph and integrate them into hybrid Graph Neural Networks (GNNs), which combine long-term static structure with transient dynamic interactions. Since static graphs are often unavailable and attributes may be difficult to use at scale or under privacy constraints, we introduce: (i) a “behavioral similarity” estimator based on normalized co-occurrence, which requires no attributes, and (ii) an attribute-aware K-means + k-NN estimator that is more efficient than cosine similarity. Experiments on multiple real-world datasets show that both methods consistently improve predictive accuracy and training efficiency, underscoring the importance of static graph choice in hybrid GNNs.

Counterfactual explanations in Explainable AI (XAI) identify which features to change to alter an outcome, but existing methods adjust only the features of a single agent. We present a new approach to re-evaluating rankings that is based on predictions of future features of the other agents in a ranking system. It uses an algorithm that provides a more realistic counterfactual explanation of changing the ranking of a particular agent. Computer experiments demonstrated that the proposed algorithm can capture the time variation of the entire ranking system in the inference results.

A knowledge graph (KG) contains rich information about users and items. The relationship among users and items can help to generate intuitive explanations for recommended items. Many variations of KG-based recommendation algorithms use the shortest path from the user to the item in order to generate an explanation of the recommendation. However, the simple shortest path may not be useful in the case when the path is long, because the interpretation of the long path is difficult. Also, there may be no path between the user and the recommended item. In order to overcome these difficulties, we proposed an extension of the existing framework based on random walk with KG embedding. In the proposed framework, we use the most probable path in a random walk as an explanation. Thereby, our framework can even explain items that have no connection in the KG due to the latent connection resulting from random walk teleportation. Comparison experiment demonstrated that the framework can provide more suitable recommendations than the existing method. In addition, the experiment show the ability of the proposed method to generate explanation for all recommendations that have no path in the graph.

We have developed an ensemble-based approach for online machine learning: adaptive rotation forest and AD-WIN adaptive rotation forest. We focused on rotation forest, an offline supervised ensemble algorithm with a particularly high prediction accuracy while all the features are continuous. Our objective was to develop a high-performance online ensemble method that uses a process similar to that of rotation forest in an online environment. Our experiments demonstrated that the proposed approach simplifies the tree structure used for the base learners, reduces memory consumption, and improves prediction accuracy for some data streams.

Factorization machines (FMs) are machine learning predictive models based on second-order feature interactions and FMs with sparse regularization are called sparse FMs. Such regularizations enable feature selection, which selects the most relevant features for accurate prediction, and therefore they can contribute to the improvement of the model accuracy and interpretability. However, because FMs use second-order feature interactions, the selection of features often causes the loss of many relevant feature interactions in the resultant models. In such cases, FMs with regularization specially designed for feature interaction selection trying to achieve interaction-level sparsity may be preferred instead of those just for feature selection trying to achieve feature-level sparsity. In this paper, we present a new regularization scheme for feature interaction selection in FMs. For feature interaction selection, our proposed regularizer makes the feature interaction matrix sparse without a restriction on sparsity patterns imposed by the existing methods. We also describe efficient proximal algorithms for the proposed FMs and how our ideas can be applied or extended to feature selection and other related models such as higher-order FMs and the all-subsets model. The analysis and experimental results on synthetic and real-world datasets show the effectiveness of the proposed methods.

A number of knowledge graph (KG)-based recommendation algorithms have been introduced; KGs enable users and items and their attributes to be treated in an integrated way and structural information to be captured through graphs. There are many variations of the KG based recommendation algorithms. Among them, KG embedding is often used, but doing this does not take advantage of the meta-path-level proximity between users and items. This paper presents a flexible framework combining random walk and KG embedding methods. The random walk model is formulated on the basis of the similarity between nodes revealed by the KG embedding. This enables the metapath level proximity of users and items to be efficiently utilized. Comparison testing demonstrated that the proposed framework performs better than random- walk-only methods and KG-embedding-only methods, and slightly better than the existing method we have extended.

Counterfactual explanations help users to understand the behaviors of machine learning models by changing the inputs for the existing outputs. For an image classification task, an example counterfactual visual explanation explains: "for an example that belongs to class A, what changes do we need to make to the input so that the output is more inclined to class B." Our research considers changing the attribute description text of class A on the basis of the attributes of class B and generating counterfactual images on the basis of the modified text. We can use the prediction results of the model on counterfactual images to find the attributes that have the greatest effect when the model is predicting classes A and B. We applied our method to a fine-grained image classification dataset and used the generative adversarial network to generate natural counterfactual visual explanations. To evaluate these explanations, we used them to assist crowdsourcing workers in an image classification task. We found that, within a specific range, they improved classification accuracy.

Although kernel methods efficiently use feature combinations without computing them directly, they do not scale well with the size of the training dataset. Factorization machines (FMs) and related models, on the other hand, enable feature combinations efficiently, but their optimization generally requires solving a non-convex problem. We present random feature maps for the itemset kernel, which uses feature combinations, and includes the ANOVA kernel, the all-subsets kernel, and the standard dot product. Linear models using one of our proposed maps can be used as an alternative to kernel methods and FMs, resulting in better scalability during both training and evaluation. We also present theoretical results for a proposed map, discuss the relationship between factorization machines and linear models using a proposed map for the ANOVA kernel, and relate the proposed feature maps to prior work. Furthermore, we show that the maps can be calculated more efficiently by using a signed circulant matrix projection technique. Finally, we demonstrate the effectiveness of using the proposed maps for real-world datasets..

Molecular graphs are one of the established representations for small molecules, and even steric or electronic information can be encoded as node and edge features. Naturally, graph neural networks have been intensively investigated to solve various chemical problems at molecular levels. However, it remains unclear how to encode relevant chemical information into graphs. We investigate this problem by proposing three models of graph neural networks with self-attention mechanisms at different levels to adaptively select relevant chemical information for each input. Using neural graph fingerprint (NFP) as a baseline, we introduce three types of attention mechanisms on the top of NFPs. Our experimental evaluations suggest that introducing these self-attention mechanisms contributes to not only improving the prediction accuracy but also providing quantitative interpretation using obtained attention coefficients.

Recommender systems using a knowledge graph can comprehensively organize users and items and their attributes and thereby improve recommendation performance. In addition, the relationship between users and items can be easily interpreted on the basis of entities and relations, thus giving explanations to recommendations. The algorithms and knowledge graphs used for generating explanations have not utilized review text. We have developed a recommendation method for predicting interactions between users and items using a knowledge graph and review text. The underlying user-item relationships are reflected and explanations are generated by predicting user-item interactions from the paths between a user and an item. The modeling is done using a recurrent neural network or a factorization machine. Items' aspects that interest users are extracted from review text and leveraged using an attention-like mechanism. Since the path between a user and an item can be easily interpreted, and the important aspects between a user and an item can be interpreted by observing the attention weight, the proposed model can generate a reasonable recommendation explanation. Testing using a real-world dataset demonstrated that the proposed model can explain the recommendations.

Although supervised learning requires a labeled dataset, obtaining labels from experts is generally expensive. For this reason, crowdsourcing services are attracting attention in the field of machine learning as a way to collect labels at relatively low cost. However, the labels obtained by crowdsourcing, i.e., from non-expert workers, are often noisy. A number of methods have thus been devised for inferring true labels, and several methods have been proposed for learning classifiers directly from crowdsourced labels, referred to as "learning from crowds." A more practical problem is learning from crowdsourced labeled data and unlabeled data, i.e., "semi-supervised learning from crowds." This paper presents a novel generative model of the labeling process in crowdsourcing. It leverages unlabeled data effectively by introducing latent features and a data distribution. Because the data distribution can be complicated, we use a deep neural network for the data distribution. Therefore, our model can be regarded as a kind of deep generative model. The problems caused by the intractability of latent variable posteriors is solved by introducing an inference model. The experiments show that it outperforms four existing models, including a baseline model, on the MNIST dataset with simulated workers and the Rotten Tomatoes movie review dataset with Amazon Mechanical Turk workers.

Data analysis education plays an important role in accelerating the efficient use of data analysis technologies in various domains. Not only the knowledge of statistics and machine learning, but also practical skills of deploying machine learning and data analysis techniques, are required for conducting data analysis projects in the real world. Data analysis competitions, such as Kaggle, have been considered as an efficient system for learning such skills by addressing real data analysis problems. However, current data analysis competitions are not designed for educational purposes and it is not well studied how data analysis competition platforms should be designed for enhancing educational effectiveness. To answer this research question, we built, and subsequently operated an educational data analysis competition platform called University of Big Data for several years. In this paper, we present our approaches for supporting and motivating learners and the results of our case studies. We found that providing a tutorial article is beneficial for encouraging active participation of learners, and a leaderboard system allowing an unlimited number of submissions can motivate the efforts of learners. We further discuss future directions of educational data analysis competitions.

Explaining recommendations helps users to make more accurate and effective decisions and improves system credibility and transparency. Current explainable recommender systems tend to provide fixed statements such as "customers who purchased this item also purchased....". This explanation is generated only on the basis of the purchase history of similar customers, so it does not include the preferences of customers who have purchased the item or a description of the item. Since user-generated reviews generally contain information about the reviewer's preferences and a description of the item, such reviews typically have more effect on purchase decisions. Therefore, using reviews to explain recommendations should be more useful than providing only a fixed statement explanation. Aiming to create a system that provides personalized explanations for recommendations, we have developed a recurrent neural network model that uses multicriteria evaluation data to generate reviews.

Artificial intelligence has shown remarkable performance in perfect information games. However, it is still no match for human players when it comes to most imperfect information games. Information set Monte Carlo tree search (ISMCTS) has been developed to reduce the effects of strategy fusion caused by determinization of the imperfect information and demonstrated advantages over the conventional Monte Carlo tree search (MCTS) that uses determinization. Because ISMCTS has only been used for games with relatively simple structure, it is still unknown whether it works effectively for more complex games. In this study, we take Pokemon as an example of a complex imperfect information game and implement a simulator to evaluate the effectiveness of ISMCTS. Experimental results show that ISMCTS outperforms the conventional MCTS that uses determinization.

A method is presented for predicting the weights used in neural networks that shortens the training time by omitting some weight updates. In general neural network training, convergence slows as the dimensions of the input data, the amount of data, and/or the complexity of the model increases. The proposed method advances convergence and facilitates the implementations of neural networks on low-performance hardware such as mobile devices. Experiments using the MNIST, CIFAR-10, and SVHN datasets demonstrated that the proposed weight prediction method shortens the training time.

Many organizations provide open data, and important insights can be gained by analyzing such data. Analysis of the potential causal relationships is a complex task. We have developed a framework for analyzing causal relationships that combines the intelligence of the crowd with state-of-the-art machine learning methods. The proposed framework takes into account the effect of possible confounding in causal analysis by collecting explanations of the correlation between variables. The validity of the collected explanations is tested using a causal discovery workflow including a conditional independence test step and a causal direction inference step. Application of this framework to data obtained from the World Bank Data website and open government data sources revealed several interesting causal relationships. The results demonstrate that the proposed framework can efficiently perform causal analysis of open data.

We present a comprehensive framework of search methods, such as simulated annealing and batch training, for solving nonconvex optimization problems. These methods search a wider range by gradually decreasing the randomness added to the standard gradient descent method. The formulation that we define on the basis of this framework can be directly applied to neural network training. This produces an effective approach that gradually increases batch size during training. We also explain why large batch training degrades generalization performance, which previous studies have not clarified.

Ratings by users on various items such as hotels and movies have become easily available on the Web. In many cases, other than overall rating for each item by each user, more detailed information such as ratings from different viewpoints and free text comments, as well as aggregated information such as the average of ratings by different users, are also available. We investigated the effectiveness of six existing collaborative filtering methods for large-scale sparse multicriteria rating data. We formulated rating aggregation as a collaborative filtering problem and applied six collaborative filtering methods to it. Furthermore, we extended three of the methods to calculate user similarity using indirect users and review comments and applied them to collaborative filtering and rating aggregation. The results show that multicriteria rating approaches perform better than single criterion rating approaches. The extended methods had better performance both in collaborative filtering and in rating aggregation.

Despite recent open data initiatives in many countries, a significant percentage of the data provided is in non-machine-readable formats like image format rather than in a machine-readable electronic format, thereby restricting their usability. This paper describes the first unified framework for converting legacy open data in image format into a machine-readable and reusable format by using crowdsourcing. Crowd workers are asked not only to extract data from an image of a chart but also to reproduce the chart objects in spreadsheets. The properties of the reconstructed chart objects give their data structures including series names and values, which are useful for automatic processing of data by computer. Since results produced by crowdsourcing inherently contain errors, a quality control mechanism was developed that improves the accuracy of extracted tables by aggregating tables created by different workers for the same chart image and by utilizing the data structures obtained from the reproduced chart objects. Experimental results demonstrated that the proposed framework and mechanism are effective.

We develop a flexible reward plan to elicit truthful predictive probability distribution over a set of uncertain events from workers. In our reward plan, the principal can assign rewards for incorrect predictions according to her similarity between events. In the spherical proper scoring rule, a worker's expected utility is represented as the inner product of her truthful predictive probability and her declared probability. We generalize the inner product by introducing a reward matrix that defines a reward for each prediction-outcome pair. We show that if the reward matrix is symmetric and positive definite, the spherical proper scoring rule guarantees the maximization of a worker's expected utility when she truthfully declares her prediction.

Machine learning is the basis of important advances in artificial intelligence. Unlike the general methods of machine learning, which use the same tasks for training and testing, the method of transfer learning uses different tasks to learn a new task. Among the various transfer learning algorithms in the literature, we focus on the attribute-based transfer learning. This algorithm realizes transfer learning by introducing attributes and transferring the results of training to another task with the common attributes. However, the existing method does not consider the frequency in which each attribute appears in feature vectors (called the observation probability). In this paper, we present a generative model with the observation probability. By the experiments, we show that the proposed method has achieved a higher accuracy rate than the existing method. Moreover, we see that it makes possible the incremental learning that was impossible in the existing method.

Software testing in object-oriented programming requires not only test input data, but also method sequences. Method sequences create new instances and change object states as desired. Method sequence generation to get desired object states is challenging. Seeker is one of the state-of-the-art implementation to generate test cases with method sequences. However, the technique generates method sequences to change a value of only a single variable. Therefore, it cannot cover the branches that need to modify values of multiple variables. In this paper, we extend the technique in order to cover branches which require multiple desired object states. The proposed approach detects all related variables in uncovered branches and uses a fitness function to give method sequences an evaluation value to reduce candidate methods. We applied Seeker and the proposed approach to four open source projects. The result shows a 1-5% improvement of branch coverage over Seeker and also indicates that the effectiveness of the proposed approach varies depending on some specific features of projects.

Monophonic sound source separation is an essential subject on the fields where sound, such as voice, music and noise, is dealt with. In particular, unsupervised approaches to this problem have high versatility in comparison with supervised approaches. Non-negative matrix factorization is the most frequently used algorithm for the monophonic sound source separation without prior knowledge. This is also applied to various applications, including data clustering, face recognition, gene expression classification. However, non-negative matrix factorization cannot be efficiently used in online learning. In order to solve this difficulty, the non-negative sparse autoencoder was proposed in the literature. Although several successful applications have been reported, this is not yet applied to the monophonic sound source separation. This paper shows that the non-negative sparse autoencoder can perform the monophonic sound source separation without prior knowledge in online learning.

Self-organizing neural networks are important tools for realizing unsupervised learning. Recently, a difficult task has involved the incremental, efficient and robust learning in noisy environments. Most of the existing techniques are poor in this regard. In this paper, we first propose a new topology generating method called enhanced competitive Hebbian learning (enhanced CHL), and then propose a novel incremental self-organizing neural network based on the enhanced CHL method, called enhanced incremental growing neural gas (Hi-GNG). The experiments presented in this paper show that the Hi-GNG algorithm can automatically and efficiently generate a topological structure with a suitable number of neurons and that the proposed algorithm is robust to noisy data.

Self-organizing neural network which is an unsupervised learning algorithm is to discover the inherent relationships of data. Such technique has become an important tool for data mining, machine learning and pattern recognition. Most self-organizing neural networks have a difficulty in reflecting data distributions precisely if data distributions are very complex. And meanwhile, it is also hard for these algorithms to learn new data incrementally without destroying the previous learnt data. In this paper, we propose a robust energy artificial neuron based incremental self-organizing neural network with a dynamic structure (REISOD). It can adjust the scale of network automatically to adapt the scale of the data set and learn new data incrementally with preserving the former learnt results. Moreover, several experiments show that our algorithm can reflect data distributions precisely.

The increasing interest in dynamically changing networks has led to growing interest in a more general link prediction problem called temporal link prediction in the data mining and machine learning communities. However, only links in identical time frames are considered in temporal link prediction. We propose a new link prediction problem called cross-temporal link prediction in which the links among nodes in different time frames are inferred. A typical example of cross-temporal link prediction is cross-temporal entity resolution to determine the identity of real entities represented by data objects observed in different time periods. In dynamic environments, the features of data change over time, making it difficult to identify cross-temporal links by directly comparing observed data. Other examples of cross-temporal links are asynchronous communications in social networks such as Face book and Twitter, where a message is posted in reply to a previous message. We adopt a dimension reduction approach to cross-temporal link prediction, that is, data objects in different time frames are mapped into a common low-dimensional latent feature space, and the links are identified on the basis of the distance between the data objects. The proposed method uses different low-dimensional feature projections in different time frames, enabling it to adapt to changes in the latent features over time. Using multi-task learning, it jointly learns a set of feature projection matrices from the training data, given the assumption of temporal smoothness of the projections. The optimal solutions are obtained by solving a single generalized eigenvalue problem. Experiments using a real-world set of bibliographic data for cross-temporal entity resolution showed that introducing time-dependent feature projections improves the accuracy of link prediction.

We propose a query-by-example geographic object search method for users that do not know well about the place they are in. Geographic objects, such as restaurants, are often retrieved using an attribute-based or keyword query. These queries, however, are difficult to use for users that have little knowledge on the place where they want to search. The proposed query-by-example method allows users to query by selecting examples in familiar places for retrieving objects in unfamiliar places. One of the challenges is to predict an effective distance metric, which varies for individuals. Another challenge is to calculate the distance between objects in heterogeneous domains considering the feature gap between them, for example, restaurants in Japan and China. Our proposed method is used to robustly estimate the distance metric by amplifying the difference between selected and non-selected examples. By using the distance metric, each object in a familiar domain is evenly assigned to one in an unfamiliar domain to eliminate the difference between those domains. We developed a restaurant search using data obtained from a Japanese restaurant Web guide to evaluate our method.

We describe methods to search with a query by example in a known domain for information in an unknown domain by exploiting Web search engines. Relational search is an effective way to obtain information in an unknown field for users. For example, if an Apple user searches for Microsoft products, similar Apple products are important clues for the search. Even if the user does not know keywords to search for specific Microsoft products, the relational search returns a product name by querying simply an example of Apple products. More specifically, given a tuple containing three terms, such as (Apple, iPod, Microsoft), the term Zune can be extracted from the Web search results, where Apple is to iPod what Microsoft is to Zune. As a previously proposed relational search requires a huge text corpus to be downloaded from the Web, the results are not up-to-date and the corpus has a high construction cost. We introduce methods for relational search by using Web search indices. We consider methods based on term co-occurrence, on lexico-syntactic patterns, and on combinations of the two approaches. Our experimental results showed that the combination methods got the highest precision, and clarified the characteristics of the methods.

We propose a supervised approach to enable accurate determination of the number of clusters in object identification. We use the aggregated attribute values of the data set to be clustered as explanatory variables in the prediction model. Attribute aggregation can be done in linear time with respect to the number of data items, so our method can be used to predict the number of clusters with a low computational burden. To deal with skewed target values, we introduce a two-stage method as well as a method using a higher-order combination of explanatory variables. Experiments demonstrate our methods enable more accurate prediction than existing methods.

This keynote presents an overview of the current status of the standardization process for ITS and illustrates its perspectives in Japan. It firstly covers current dedicated short range communications (DSRC) standards development activities including vehicle to vehicle (V2V) and vehicle to roadside (V2R) communications for vehicle safety applications in Japan, US and Europe. It secondly describes Japanese projects for vehicle safety such as ITS-Safety 2010 or Ubiquitous ITS R&D and also looks at the potential usage of the newly reallocated 700 MHz band.

A method is described for discovering coordinate terms, such as "Honda'' and "Nissan,'' for a given term, such as "Toyota,'' as well as their common topic terms, from the query logs of a Web search engine. Coordinate terms are good candidates for use in making comparisons. A HITS-based algorithm is applied to a bipartite graph between coordinate term candidates and co-occurrence patterns to identify coordinate and topic terms. Spectral analysis is used to distinguish coordinate terms corresponding to different aspects of the search term. As a result, we can discover terms related to the terms in a search engine query that reflect the needs and interests of the user.

Domain-specific Web search engines are effective tools for reducing the difficulty experienced when acquiring information from the Web. Existing methods for building domain-specific Web search engines require human expertise or specific facilities. However, we can build a domain-specific search engine simply by adding domain-specific keywords, called "keyword spices," to the user's input query and forwarding it to a general-purpose Web search engine. Keyword spices can be effectively discovered from Web documents using machine learning technologies. The paper describes domain-specific Web search engines that use keyword spices for locating recipes, restaurants, and used cars.

Conventional Web bookmarks only contain URLs and titles of Web pages that users are interested in. This makes the process of remembering, sharing or ranking such pages difficult. The "context" of users' navigation can be described as collections of browsed pages. Conventional bookmarks do not contain such information. We believe that such context information conveys the users' intention and the importance of bookmarks. We introduce a notion of context-dependent Web bookmarks that reflects users' browsing histories. A context-dependent Web bookmark consists of (1) representative keywords of bookmarked pages and browsed pages, (2) the ranking value of bookmarked pages calculated by its context, as well as the URL and title of the page that the user bookmarked. Context-dependent bookmarks will make it possible for users to remember the situation of the bookmarking process, grasp the degree of significance of the bookmark, and share the bookmark among multiple users. Furthermore, it becomes possible to re-use context-dependent bookmarks as queries, which could be executed for unvisited Web pages. We also describe our Web browser prototype system based on the context-dependent bookmark function, and our experimental results.

A digital city is a social information infrastructure for urban life (including shopping, business, transportation, education, welfare and so on). We started a project to develop a digital city for Kyoto based on the newest technologies including cooperative information agents. This paper presents an architecture for digital cities and shows the roles of agent interfaces in it. We propose two types of cooperative information agents as follows: (a) the front-end agents determine and refine users' uncertain goals, (b) the back-end agents extract and organize relevant information from the Internet, (c) Both types of agents opportunistically cooperate through a blackboard. We also show the research guidelines towards social agents in digital cities; the agent will foster social interaction among people who are living in/visiting the city.