Metric learning projects samples into an embedded space, where similarities and dissimilarities are quantified based on their learned representations. However, existing methods often rely on label-guided representation learning, where representations of different modalities, such as audio and visual data, are aligned based on annotated labels. This approach tends to underutilize latent complex features and potential relationships inherent in the distributions of audio and visual data that are not directly tied to the labels, resulting in suboptimal performance in audio-visual embedding learning. To address this issue, we propose a novel architecture that integrates cross-modal triplet loss with progressive self-distillation. Our method enhances representation learning by leveraging inherent distributions and dynamically refining soft audio-visual alignments—probabilistic aligns between audio and visual data that capture the inherent relationships beyond explicit labels. Specifically, the model distills audio-visual distribution-based knowledge from annotated labels in a subset of each batch. This self-distilled knowledge is used to automatically generate soft-alignment labels for the remaining audio-visual samples. These soft-alignment labels are used to construct soft cross-modal triplets, which in turn are employed to fine-tune the model’s parameters. Experimental results on two audio-visual benchmark datasets demonstrate the effectiveness of our proposed method in the cross-modal retrieval task, achieving state-of-the-art performance with improvements of 2.13% and 1.82% on the AVE and VEGAS datasets, respectively, in terms of average MAP metrics.

Existing works on stock price prediction have largely treated stocks in a market independently of one another. Nevertheless, recent advances in graph neural networks (GNNs) have enabled the efficient processing of diverse stock relations. This paper introduces the Financial SIR-GCN (FinSIR) for market-aware stock price prediction and recommendation. By modeling stock markets as spatio-temporal graphs, FinSIR addresses the key architectural limitation of existing graph-based models. Notably, the proposed model integrates the soft-isomorphic relational graph convolution network (SIR-GCN) with the "sandwich" structure employed in GNN for time series analysis (GNN4TS) to jointly process the two key dimensions of stock market graphs and to contextualize hidden states with both spatial and temporal stock relations. Backtesting results on the New York Stock Exchange (NYSE) and the National Association of Securities Dealers Automatic Quotation System (NASDAQ) reveal FinSIR consistently achieving up to 65% and 36% larger cumulative investment returns, respectively, compared to baseline models. Additionally, an ablation study further highlights the contribution of each FinSIR module in providing better investment recommendations. Overall, the paper incorporates recent advances in GNN and GNN4TS to provide a new perspective on graph-based solutions for improved stock price prediction and recommendation.

In-Context Learning (ICL) approaches, such as Zero-Shot and Few-Shot prompting, allow Large Language Models (LLMs) to tackle reasoning tasks without additional fine-tuning. However, Zero-Shot prompting often struggles with more complex tasks, whereas Few-Shot prompting demands considerable manual effort and domain expertise to design effective prompts. Although existing work has attempted to alleviate these issues by extracting reasoning rules from carefully crafted, task-specific representative examples, creating or obtaining such examples can be impractical in real-world scenarios. In this paper, we propose a novel approach that enhances the inference accuracy by injecting reasoning principles extracted from QA data, without relying on representative Few-Shot exemplars. This offers a lightweight yet adaptive way to boost accuracy on complex reasoning tasks, while avoiding manual effort and the high exploration costs typical of prior methods. Experiments on benchmarks show that, using GPT-4o, our method outperforms similarity-based Few-Shot and Zero-Shot prompting methods on challenging benchmarks such as GPQA-diamond, achieving an absolute accuracy improvement of up to 2% in scenarios where carefully crafted Few-Shot examples are unavailable.

We hypothesize that optimal system responses emerge from adaptive strategies grounded in causal and counterfactual knowledge.Counterfactual inference allows us to create hypothetical scenarios to examine the effects of alternative system responses.We enhance this process through causal discovery, which identifies the strategies informed by the underlying causal structure that govern system behaviors.Moreover, we consider the psychological constructs and unobservable noises that might be influencing user-system interactions as latent factors.We show that these factors can be effectively estimated.We employ causal discovery to identify strategy-level causal relationships among user and system utterances, guiding the generation of personalized counterfactual dialogues.We model the user utterance strategies as causal factors, enabling system strategies to be treated as counterfactual actions.Furthermore, we optimize policies for selecting system responses based on counterfactual data.Our results using a real-world dataset on social good demonstrate significant improvements in persuasive system outcomes, with increased cumulative rewards validating the efficacy of causal discovery in guiding personalized counterfactual inference and optimizing dialogue policies for a persuasive dialogue system.

Metric learning minimizes the gap between similar (positive) pairs of data points and increases the separation of dissimilar (negative) pairs, aiming at capturing the underlying data structure and enhancing the performance of tasks like audio-visual cross-modal retrieval (AV-CMR). Recent works employ sampling methods to select impactful data points from the embedding space during training. However, the model training fails to fully explore the space due to the scarcity of training data points, resulting in an incomplete representation of the overall positive and negative distributions. In this paper, we propose an innovative Anchor-aware Deep Metric Learning (AADML) method to address this challenge by uncovering the underlying correlations among existing data points, which enhances the quality of the shared embedding space. Specifically, our method establishes a correlation graph-based manifold structure by considering the dependencies between each sample as the anchor and its semantically similar samples. Through dynamic weighting of the correlations within this underlying manifold structure using an attention-driven mechanism, Anchor Awareness (AA) scores are obtained for each anchor. These AA scores serve as data proxies to compute relative distances in metric learning approaches. Extensive experiments conducted on two audio-visual benchmark datasets demonstrate the effectiveness of our proposed AADML method, significantly surpassing state-of-the-art models. Furthermore, we investigate the integration of AA proxies with various metric learning methods, further highlighting the efficacy of our approach.

The application of reinforcement learning (RL) involving interactions with individuals has grown significantly in recent years. These interactions, influenced by factors such as personal preferences and physiological differences, causally influence state transitions, ranging from health conditions in healthcare to learning progress in education. As a result, different individuals may exhibit different state-transition processes. Understanding individualized state-transition processes is essential for optimizing individualized policies. In practice, however, identifying these state-transition processes is challenging, as individual-specific factors often remain latent. In this paper, we establish the identifiability of these latent factors and introduce a practical method that effectively learns these processes from observed state-action trajectories. Experiments on various datasets show that the proposed method can effectively identify latent state-transition processes and facilitate the learning of individualized RL policies.

Even though large language models (LLMs) accumulate tremendous knowledge, dialogue systems built with LLMs induce hallucinations, leading to the generation of non-factual responses. How to provide proper references to achieve interpretable hallucination detection is a key issue that needs to be addressed. In this paper, we propose a graph neural network (GNN)-based method to achieve high-performance and interpretable hallucination detection for domain-specific dialogue systems. The method involves performing graph matching between a reference knowledge graph obtained from a knowledge database and a response knowledge graph extracted from the response to detect non-factual responses. By comparing with strong baselines, our method achieves a recall improvement of up to 11% and infers the cause of hallucinations with a probability of over 79%.

We introduce a novel feature-based active learning approach that leverages leading feature selection methods from explainable AI (XAI) to elicit user corrections on both the classification label and the leading features contributing to the classification decision. Uniquely, our approach can handle such user corrections, which identify missing and superfluous features in a model-agnostic manner that can be applied to any supervised classification model. Our approach generates augmented samples in a fine-grained manner according to user corrections, and then adds them to the training data to teach relevant and irrelevant features. Our results show that our approach outperforms traditional active learning and a leading baseline in two domains, which includes improvement in classification performance from 0.572 to 0.637 given the same number of queries.

We posit that predicting sensor event sequence (SES) in a smart home can proactively support resident activities or recognize activities that have not been completed as intended and alert the resident. To realize this application, we propose a framework to support accurate SES prediction by leveraging online activity recognition. Our framework includes a novel method of applying a GPT2-based model, which is a sentence generation model, for SES prediction by taking advantage of the property that the relationship between ongoing activity and SES patterns is similar to the relationship between topic and word sequence patterns in NLP. We evaluated our method empirically using two real-world datasets where residents perform their usual daily activities. Our experimental results show the use of the GPT2-based model significantly improves the F1 value of SES prediction from 0.461 to 0.708 compared to the state-of-the-art method, and that using ongoing activity can further improve performance to 0.837. We found that the performance of the online activity recognition model required to achieve these SES predictions was about 80%, which could be achieved using simple feature engineering and modeling.

The cross-modal retrieval models leverage the potential of triple loss optimization to learn robust embedding spaces. However, existing methods often train these models in a singular pass, overlooking the distinction between semi-hard and hard triples in the optimization process, which will lead to suboptimal model performance. In this paper, we introduce a novel approach rooted in curriculum learning to address this problem. We propose a two-stage training paradigm that guides the model’s learning process from semi-hard to hard triplets. In the first stage, the model is trained with a set of semi-hard triplets, starting from a low-loss base. Subsequently, in the second stage, the model mines the hardest triplet with the primary aim of mitigating the risk of overfitting by addressing the highest loss. Extensive experimental results conducted on the audio-visual dataset show a significant improvement of approximately 9.8% in terms of average Mean Average Precision (MAP) over the current state-of-the-art method, MSNSCA, for the Audio-Visual Cross-Modal Retrieval (AV-CMR) task on the AVE dataset, indicating the effectiveness of our proposed method.

Previous studies have shown that the personality composition of a group significantly affects learners’ satisfaction during collaborative learning. However, while these studies investigated a group as a whole by focusing on group statistics, such as the mean and standard deviation of the members’ personalities, they paid little attention to the personality differences of individual pairs within the group, albeit the group contains many pairwise interactions. In this paper, we studied whether and how pairwise personality differences between a learner and groupmates affect the learner’s satisfaction. Examining data collected from an employee training program during which learners had reflective group discussions, we confirmed that pairwise personality differences significantly affect a learner’s level of satisfaction in the program. Specifically, satisfaction is affected by (1) the average of the personality differences between the learner and each individual groupmate, which reflects the degree to which the learner is different from the groupmates on average, and (2) the personality difference from the groupmate who has the most different/similar personality from/to the learner.

The Japanese language is an essential skill for many foreigners who plan to study or work in Japan, but it is very hard and time-consuming to learn. Given the current COVID19 pandemic, the use of online and computer-assisted tools for Japanese language learning is indispensable. However, many of the currently available tools do not offer personalized content based on the user’s performance and lack social interaction, which can lower the engagement level of the users. In this paper, we propose JaSenpai, a Japanese language E-learning platform that features an automatic generation of vocabulary exercises, a recommendation system based on previous answers, and a multiplayer game for social interaction. We believe these elements can provide a more engaging and effective learning experience.

The heterogeneity gap problem is the main challenge in cross-modal retrieval. Because cross-modal data (e.g. audiovisual) have different distributions and representations that cannot be directly compared. To bridge the gap between audiovisual modalities, we learn a common subspace for them by utilizing the intrinsic correlation in the natural synchronization of audio-visual data with the aid of annotated labels. TNN-C-CCA is the best audio-visual cross-modal retrieval (AV-CMR) model so far, but the model training is sensitive to hard negative samples when learning common subspace by applying triplet loss to predict the relative distance between inputs. In this paper, to reduce the interference of hard negative samples in representation learning, we propose a new AV-CMR model to optimize semantic features by directly predicting labels and then measuring the intrinsic correlation between audio-visual data using complete cross-triple loss. In particular, our model projects audio-visual features into label space by minimizing the distance between predicted label features after feature projection and ground label representations. Moreover, we adopt complete cross-triplet loss to optimize the predicted label features by leveraging the relationship between all possible similarity and dissimilarity semantic information across modalities. The extensive experimental results on two audio-visual double-checked datasets have shown an improvement of approximately 2.1% in terms of average MAP over the current state-of-the-art method TNN-C-CCA for the AV-CMR task, which indicates the effectiveness of our proposed model.

Graph neural networks (GNNs) have been widely used to learn vector representation of graph-structured data and achieved better task performance than conventional methods. The foundation of GNNs is the message passing procedure, which propagates the information in a node to its neighbors. Since this procedure proceeds one step per layer, the range of the information propagation among nodes is small in the lower layers, and it expands toward the higher layers. Therefore, a GNN model has to be deep enough to capture global structural information in a graph. On the other hand, it is known that deep GNN models suffer from performance degradation because they lose nodes' local information, which would be essential for good model performance, through many message passing steps. In this study, we propose multi-level attention pooling (MLAP) for graph-level classification tasks, which can adapt to both local and global structural information in a graph. It has an attention pooling layer for each message passing step and computes the final graph representation by unifying the layer-wise graph representations. The MLAP architecture allows models to utilize the structural information of graphs with multiple levels of localities because it preserves layer-wise information before losing them due to oversmoothing. Results of our experiments show that the MLAP architecture improves the graph classification performance compared to the baseline architectures. In addition, analyses on the layer-wise graph representations suggest that aggregating information from multiple levels of localities indeed has the potential to improve the discriminability of learned graph representations.

A single shepherd dog can herd a flock of sheep to a gate. Despite a heuristic algorithm of a dog based on adaptive switching between collecting the sheep when they are too dispersed and driving them once they are aggregated, it remains unknown how the dog learns the algorithm of switching. In fact, reinforcement learning models have not succeeded so far in reproducing the switching algorithm without explicitly making two strategies. Here, we show that an imitation learning model can reproduce the switching algorithm, that is, the dog learns the algorithm from demonstrations by an expert. We also confirmed that the dog does not simply copy the demonstrations but learns the required task by showing that it can herd more sheep than those in the given demonstrations.

As Intelligent Transport Systems (ITS) advances, more and more people will have the opportunities to drive vehicles with autonomous capabilities. This rise in number of semi-autonomous vehicles also gives rise to several challenges with regards on how human factors come into play in interacting with the vehicle's Automated Driving System (ADS). One important interaction of an ADS with Level 3 Conditional Automated Driving capabilities is Request-to-Intervene (RTI), which alerts drivers to takeover the vehicle during an automated driving session, however, the driver is not necessarily ready to receive the authority. To see whether an ADS can detect the readiness of the user for RTI, in this preliminary study we evaluated the mental states of ADS users in naturalistic driving conditions by comparing them with those of drivers and passengers. The mental states were evaluated by measuring their heart rate and by calculating specific features of Heart Rate Variability (HRV), specifically NN50 and pNN50 indices, during driving events (turning, lane changing, and stopping) and no-events. The results showed the NN50 and pNN50 values of manual driving were significantly different from those of ADS driving and passenger, suggesting that ADS driving has a higher level of relaxed state. In addition, events such as lane-changing in the ADS driving did not induce significantly different NN50 and pNN50 from non-event situation, which may imply the participants did not pay attention to such events.

The skip-connection and the batch-normalization (BN) in ResNet enable an extreme deep neural network to be trained with high performance. However, the reasons for its high performance are still unclear. To clear that, we study the effects of the skip-connection and the BN on the class-related signal propagation through hidden layers because a large ratio of the between-class distance to the within-class distance of feature vectors at the last hidden layer induces high performance. Our result shows that the between-class distance and the within-class distance change differently through layers: the deep multilayer perceptron with randomly initialized weights degrades the ratio of the between-class distance to the within-class distance and the skip-connection and the BN relax this degradation. Moreover, our analysis implies that the skip-connection and the BN encourage training to improve this distance ratio. These results imply that the skip-connection and the BN induce high performance.

For learning tasks where the data (or losses) may be heavy-tailed, algorithms based on empirical risk minimization may require a substantial number of observations in order to perform well off-sample. In pursuit of stronger performance under weaker assumptions, we propose a technique which uses a cheap and robust iterative estimate of the risk gradient, which can be easily fed into any steepest descent procedure. Finite-sample risk bounds are provided under weak moment assumptions on the loss gradient. The algorithm is simple to implement, and empirical tests using simulations and real-world data illustrate that more efficient and reliable learning is possible without prior knowledge of the loss tails.

Empathic conversations have increasingly been important for dialogue systems to improve the users' experience, and increase their engagement with the system, which is difficult for many existing monotonous systems. Existing empathic dialogue systems are designed for limited domain dialogues. They respond fixed phrases toward observed user emotions. In open domain conversations, however, generating empathic responses for a wide variety of topics is required. In this paper, we draw on psychological studies about empathy, and propose an empathic dialogue system in open domain conversations. The proposed system generates empathic utterances based on observed emotions in user utterances, thus is able to build empathy with users. Our experiments have proven that users were able to feel more empathy from the proposed system, especially when their emotions were explicitly expressed in their utterances.

As a disaster is an unpredictable event, it may be tough to escape damage entirely even if evacuation drills are carried out periodically. In such a case, persons requiring evacuation assistance like aged people and disabled people are prone to be victims. For minimizing the damage, a fire department and a local government need to swiftly grasp the situation of the afflicted area in order to execute rescue operation. In this paper, to support the rescue operation, we propose a design and implementation of an Internet of Things (IoT) device to grasp the presence of persons requiring evacuation assistance. The IoT device captures Wi-Fi signal and packet from smartphones that victims have, and it then provides an estimated area for a presence of them. In the estimation of the area, we introduce three ways to select center of an area, i.e., random, average, and median selections. Through the experimental results, we showed that the average and the median selections make more accurate presence area than the random selection. Also, the system presented the visualization of the area to support the rescue operation.

In this paper, we design a crowd-powered system to efficiently collect data for training dialogue systems. Conventional systems assign dialogue roles to a pair of crowd workers, and record their interaction on an online chat. In this framework, the pair is required to work simultaneously, and one worker must wait for the other when he/she is writing a message, which decreases work efficiency. Our proposed system allows multiple workers to create dialogues in an asynchronous manner, which relieves workers from time restrictions. We have conducted an experiment using our system on a crowdsourcing platform to evaluate the efficiency and the quality of dialogue collection. Results show that our system can reduce the necessary time to input a message by 68% while maintaining quality.

This paper presents an augmenting path based online max-flow algorithm. The proposed algorithm handles graph changes in chunk manner, updating residual graph in response to edge capacity increase, decrease, edge/node adding and removal. All possible graph changes are abstracted into two key graph changes, which are capacity decrease and in- crease. For capacity decrease, we release the occupied capacity by cycle cancellation and path de-augmentation to enable the capacity decrease. For capacity increase, we augment all s-t paths newly formed to update the current max-flow model. The theoretical guarantee of our algorithm is that online max- flow is always equal to batch retraining. Experiments show the deterministic computational cost save (i.e., gain) of our algorithm w.r.t batch retraining in handling graph edge adding.

The increasing popularity of mobile crowdsourcing platforms has enabled crowd workers to accept jobs wherever/whenever they are, and also provides opportunity for task requesters to order time/location specific tasks to workers. Since workers on mobile platforms are working on the go, the situation of the workers is expected to influence their performance. However, the effects of mobile worker situations to task performance is an uninvestigated area. In this paper, our research question is, "do worker situations affect task completion, price and quality on mobile crowdsourcing platforms?" We draw on economics and psychology research to examine whether worker situations such as busyness, fatigue and presence of companions affect their performance. Our three-week between-subjects field experiment revealed that worker busyness caused 30.1% relative decrease of task completion rate. Mean accepted task price increased by 7.6% when workers are with companions. Worker fatigue caused 37.4% relative decrease of task quality.

Experiments with drug-induced epilepsy in rat brains and epileptic human brain region reveal that focal cooling can suppress epileptic discharges without affecting the brain's normal neurological function. Findings suggest a viable treatment for intractable epilepsy cases via an implantable cooling device. However, precise mechanisms by which cooling suppresses epileptic discharges are still not clearly understood. Cooling experiments in vitro presented evidence of reduction in neurotransmitter release from presynaptic terminals and loss of dendritic spines at post-synaptic terminals offering a possible synaptic mechanism. We show that termination of epileptic discharges is possible by introducing a homogeneous temperature factor in a neural mass model which attenuates the post-synaptic impulse responses of the neuronal populations. This result however may be expected since such attenuation leads to reduced post-synaptic potential and when the effect on inhibitory interneurons is less than on excitatory interneurons, frequency of firing of pyramidal cells is consequently reduced. While this is observed in cooling experiments in vitro, experiments in vivo exhibit persistent discharges during cooling but suppressed in magnitude. This leads us to conjecture that reduction in the frequency of discharges may be compensated through intrinsic excitability mechanisms. Such compensatory mechanism is modelled using a reciprocal temperature factor in the firing response function in the neural mass model. We demonstrate that the complete model can reproduce attenuation of both magnitude and frequency of epileptic discharges during cooling. The compensatory mechanism suggests that cooling lowers the average and the variance of the distribution of threshold potential of firing across the population. Bifurcation study with respect to the temperature parameters of the model reveals how heterogeneous response of epileptic discharges to cooling (termination or suppression only) is exhibited. Possibility of differential temperature effects on post-synaptic potential generation of different populations is also explored.

Robotic solutions to clothing assistance can significantly improve quality of life for the elderly and disabled. Real-time estimation of the human-cloth relationship is crucial for efficient learning of motor skills for robotic clothing assistance. The major challenge involved is cloth-state estimation due to inherent nonrigidity and occlusion. In this study, we present a novel framework for real-time estimation of the cloth state using a low-cost depth sensor, making it suitable for a feasible social implementation. The framework relies on the hypothesis that clothing articles are constrained to a low-dimensional latent manifold during clothing tasks. We propose the use of manifold relevance determination (MRD) to learn an offline cloth model that can be used to perform informed cloth-state estimation in real time. The cloth model is trained using observations from a motion capture system and depth sensor. MRD provides a principled probabilistic framework for inferring the accurate motion-capture state when only the noisy depth sensor feature state is available in real time. The experimental results demonstrate that our framework is capable of learning consistent task-specific latent features using few data samples and has the ability to generalize to unseen environmental settings. We further present several factors that affect the predictive performance of the learned cloth-state model.

Paid crowdsourcing marketplaces have gained popularity by using piecework, or payment for each microtask, to incentivize workers. This norm has remained relatively unchallenged. In this paper, we ask: is the pay-per-task method the right one? We draw on behavioral economic research to examine whether payment in bulk after every ten tasks, saving money via coupons instead of earning money, or material goods rather than money will increase the number of completed tasks. We perform a twenty-day, between-subjects field experiment (N=300) on a mobile crowdsourcing application and measure how often workers responded to a task notification to fill out a short survey under each incentive condition. Task completion rates increased when paying in bulk after ten tasks: doing so increased the odds of a response by 1.4x, translating into 8% more tasks through that single intervention. Payment with coupons instead of money produced a small negative effect on task completion rates. Material goods were the most robust to decreasing participation over time.

In this paper, we present a reinforcement learning model of the shepherding of a flock of sheep by a dog. The shepherding task, a heuristic model originally proposed by Strombom, et al., describes the dynamics of the sheep while being herded by a dog to a predefined target. This study recreates the proposed model using SARSA, an algorithm for learning the optimal policy in reinforcement learning. Results show that with a discretized state and action space, the dog is able to successfully herd a flock of a sheep to the target position by first learning to reach a subgoal. A reward is awarded when the dog reaches the neighbourhood of a subgoal, while a penalty is incurred for each time the shepherding task is not completed. The stochasticity of the interaction among sheep and dog, including the existence of multiple subgoals affect the learning time of the agent. Finally, we present an example of the learned shepherding task which shows the agent's continuous success after the 350th episode.

A fundamental yet new challenge in information retrieval is the identification of patterns behind search queries. For example, the query "NY restaurant" and "boston hotel" shares the common pattern "LOCATION SERVICE". However, because of the diversity of real queries, existing approaches require data preprocessing by humans or specifying the target query domains, which hinders their applicability.

From turbine control systems at wind farms to extreme weather early-warning systems, short-term probabilistic wind speed forecasts are seeing widespread use in industrial applications. Successful modern forecast methods, often Weibull-based, have been shown to be extremely sensitive to even minor changes in location. We contend that this lack of robustness stems not from model selection, but rather the parameter estimation methods used, and propose a new proper scoring rule to be dynamically minimized. Tested on a weather array spanning the islands of Japan, we verify both superior short-term forecasting performance and model fit of the proposed method over all standard references, and empirically confirm the desired location robustness.

Real-time estimation of human-cloth relationship is crucial for efficient learning of motor skills in robotic clothing assistance. However, cloth state estimation using a depth sensor is a challenging problem with inherent ambiguity. To address this problem, we propose the offline learning of a cloth dynamics model by incorporating reliable motion capture data and applying this model for the online tracking of human-cloth relationship using a depth sensor. In this study, we evaluate the performance of using a shared Gaussian Process Latent Variable Model in learning the dynamics of clothing articles. The experimental results demonstrate the effectiveness of shared GP-LVM in capturing cloth dynamics using few data samples and the ability to generalize to unseen settings. We further demonstrate three key factors that affect the predictive performance of the trained dynamics model.

Network data show the relationship among one kind of objects, such as social networks and hyperlinks on the Web. Many statistical models have been proposed for analyzing these data. For modeling cluster structures of networks, the infinite relational model (IRM) was proposed as a Bayesian nonparametric extension of the stochastic block model. In this brief, we derive the inference algorithms for the IRM of network data based on the variational Bayesian (VB) inference methods. After showing the standard VB inference, we derive the collapsed VB (CVB) inference and its variant called the zeroth-order CVB inference. We compared the performances of the inference algorithms using six real network datasets. The CVB inference outperformed the VB inference in most of the datasets, and the differences were especially larger in dense networks.

Diffuse optical tomography (DOT) reconstructs 3-D tomographic images of brain activities from observations by near-infrared spectroscopy (NIRS) that is formulated as an ill-posed inverse problem. This brief presents a method for NIRS DOT based on a hierarchical Bayesian approach introducing the automatic relevance determination prior and the variational Bayes technique. Although the sparseness of the estimation strongly depends on the hyperparameters, in general, our method has less dependency on the hyperparameters. We confirm through numerical experiments that a schematic phase diagram of sparseness with respect to the hyperparameters has two regions: in one region hyperparameters give sparse solutions and in the other they give dense ones. The experimental results are supported by our theoretical analyses in simple cases.

From optimal supply decisions to anticipatory control systems, wind-based energy applications rely heavily upon accurate, local, short-term forecasts of future wind speed. Recent studies have shown continuous ranked probability score (CRPS) minimizing models with Gaussian assumptions to be effective for well-researched sites where those assumptions are appropriate. We consider the more general case where Gaussianity is not assumed and access to historical data may be constrained. Deriving a CRPS expression for a minimum Extreme Value distribution, we use it to propose a site-adaptive Weibull-based CRPS-minimizing model, which is tested and shown to perform better than both deterministic and probabilistic reference models on a ground-based array of weather observation sites in northern Japan.

Prediction of driving behaviors is important problem in developing the next-generation driving support system. In order to take account of diverse driving situations, it is necessary to deal with multiple time series data considering commonalities and differences among them. In this paper we utilize the beta process autoregressive hidden Markov model (BP-AR-HMM) that can model multiple time series considering common and different features among them using the beta process as a prior distribution. We apply the BP-AR-HMM to actual driving behavior data to estimate VAR process parameters that represent the driving behaviors, and with the estimated parameters we predict the driving behaviors of unknown test data. The results suggest that it is possible to identify the dynamical behaviors of driving operations using BP-AR-HMM, and to predict driving behaviors in actual environment.

The statistical properties of support vector machines (SVMs) for non-separable problems are studied. SVMs with hard margins are not always solvable for non-separable problems. Introducing soft margin alleviates this difficulty, but SVMs still fail to successfully solve these problems for heavily overlapped data. From the practical viewpoint, increasing the velocity of a soft margin depending of the number of examples is a way to adapt to increasing data generated from an identity distribution. However, systematic control of soft margin from the theoretical viewpoint is in development. A concept called “lifting up” for overlapped distributions gives a slightly different geometrical structure from the one for linearly separable distributions. In this study, the probability that an SVM can not solve a problem properly is mathematically derived in the one-dimensional case for both a hard-margin and a soft-margin. Some computer simulations confirm the theoretical validity.

Diffuse optical tomography is a method used to reconstruct tomographic images from brain activities observed by near-infrared spectroscopy. This is useful for brain-machine interface and is formulated as an ill-posed inverse problem. We apply a hierarchical Bayesian approach, automatic relevance determination (ARD) prior and the variational Bayes method, that can introduce localization into the estimation of the problem. Although ARD enables sparse estimation, it is still open how hyperparameters affect the sparseness and accuracy of the estimation. Through numerical experiments, we present a schematic phase diagram of sparseness with respect to the hyperparameters in the method, which indicates the region of the hyperparameters where sparse estimation is achievable.

IAM ABOUT to start my second year of service as the Editor-in-Chief (EiC) of the IEEE TRANSACTIONS ON NEURAL NETWORKS (TNN). Needless to say, my first year as the EiC has been full of excitement and challenges. Transitioning this position from my predecessor to me went very smoothly during the months of September 2009 to January 2010. During the past year, we have accumulated 50+ Associate Editors (AEs) handling roughly 600 new submissions (not counting resubmissions and revised submissions). With the help of these AEs and my predecessor, I was quickly able to learn to do my job, and as such, the transition had very few glitches. The easy part of my job is checking whether a submission is in compliance with our guidelines and where it is within the scope of the TRANSACTIONS, before it is assigned to an AE for handling. The difficult part of my job has been dealing with some papers with three or more reviewers, all of whom agreed to review them but for some reason failed to respond to repeated automatic-review reminders. AEs handling these papers have to take several extra steps to remind reviewers through phone calls or e-mails, look for replacement reviewers, or review the papers themselves. Most authors have been appreciative of the work of the AEs and reviewers, and they accept our decisions without a problem. The backlog of papers has been kept short over the last year. We have maintained an organized printing and paperacceptance schedule, with papers typically printed in the journal within 2‐3 months of acceptance. Our page budget has been kept constant in the past few years (roughly 2060 pages per year), and we expect to hold the same page count for next year.

In this paper, we study probabilistic modeling of heterogeneously attributed multi-dimensional arrays. The model can manage the heterogeneity by employing an individual exponential-family distribution for each attribute of the tensor array. These entries are connected by latent variables and are shared information across the different attributes. Because a Bayesian inference for our model is intractable, we cast the EM algorithm approximated by using the Lap lace method and Gaussian process. This approximation enables us to derive a predictive distribution for missing values in a consistent manner. Simulation experiments show that our method outperforms other methods such as PARAFAC and Tucker decomposition in missing-values prediction for cross-national statistics and is also applicable to discover anomalies in heterogeneous office-logging data.

In this paper, we propose a new overlay multicast protocol designed for inter-active multimedia streaming applications. The protocol considers the heterogeneity of end-hosts and tries to minimize the negative impact (data outage) of end-hosts' unannounced departures. For this purpose, it concentrates on end- hosts' reliability (lifetime for instance) and constructs a shared tree called ms-DDBMSST (multiple-source Degree and Delay Bounded Maximum Stability Spanning Tree) as an overlay network that involves all the participants of the application, in a distributed manner. For a given set of nodes where some of them are senders, ms-DDBMSST is a spanning tree where the receive path stability of the entire tree is maximized while satisfying the delay-from-source constraint and degree constraint for each node. We believe that this is the first approach that defines ms-DDBMSST construction problem and presents a distributed protocol for the purpose. Our performance evaluation is based on experiments in both simulated networks and PlanetLab that strongly shows the efficiency and usefulness of the proposed protocol.

We propose generic relay and subcarrier allocation schemes for multicarrier (MC) system with amplify-and-forward (AF) relays. The outage probability bounds are derived analytically for each scheme. Simulation results show that these bounds are very tight and better than the bounds obtained straightforwardly from the analysis in the Single-Carrier (SC) case. This is because in our analysis we reckon with the increased degree of freedom brought by the parallel channels. One of the proposed schemes, the Average Best Relay Selection scheme, is best suited for practical implementation since it approaches the best performance while minimizing the required amount of signaling.

Quasi-additive (QA) algorithms are a kind of online learning algorithms having two parameter vectors: one is an accumulation of input vectors and the other is a weight vector for prediction associated with the former by a non-linear function. We show that the vectors have a dually-flat structure from the information-geometric point of view, which makes it easier to discuss the convergence properties of the algorithms, as presented here.

How we should choose a kernel function in support vector machines (SVMs), is an important but difficult problem. In this paper, we discuss the properties of the solution of the v-SVM's, a variation of SVM's, for normalized feature vectors in two extreme cases: All feature vectors are almost orthogonal and all feature vectors are almost the same. In the former case, the solution of the v-SVM is nearly the center of gravity of the examples given while the solution is approximated to that of the v-SVM with the linear kernel in the latter case. Although extreme kernels are not employed in practice, analyzes are helpful to understand the effects of a kernel function on the generalization performance.

We exhibit an important property called the asymptotic equipartition property (AEP) on empirical sequences in an ergodic multiagent Markov decision process (MDP). Using the AEP which facilitates the analysis of multiagent learning, we give a statistical property of multiagent learning, such as reinforcement learning (RL), near the end of the learning process. We examine the effect of the conditions among the agents on the achievement of a cooperative policy in three different cases: blind, visible, and communicable. Also, we derive a bound on the speed with which the empirical sequence converges to the best sequence in probability, so that the multiagent learning yields the best cooperative result.

Support vector machines (SVMs) are known to have a high generalization ability, yet a heavy computational load since margin maximization results in a quadratic programming problem. It is known that this maximization task results in a pth-order programming problem if we employ the L/sub P/ norm instead of the L/sub 2/ norm. In this paper, we theoretically show the effects of p on the learning properties of SVMs by clarifying its geometrical meaning.

An information geometrical method is developed for characterizing or classifying neurons in cortical areas, whose spike rates fluctuate in time. Under the assumption that the interspike intervals of a spike sequence of a neuron obey a gamma process with a time-variant spike rate and a fixed shape parameter, we formulate the problem of characterization as a semiparametric statistical estimation, where the spike rate is a nuisance parameter. We derive optimal criteria from the information geometrical viewpoint when certain assumptions are added to the formulation, and we show that some existing measures, such as the coefficient of variation and the local variation, are expressed as estimators of certain functions under the same assumptions.

By employing the L1 or Linfinity norms in maximizing margins, support vector machines (SVMs) result in a linear programming problem that requires a lower computational load compared to SVMs with the L2 norm. However, how the change of norm affects the generalization ability of SVMs has not been clarified so far except for numerical experiments. In this letter, the geometrical meaning of SVMs with the Lp norm is investigated, and the SVM solutions are shown to have rather little dependency on p.

The generalization properties of learning classifiers with a polynomial kernel function are examined. In kernel methods, input vectors are mapped into a high-dimensional feature space where the mapped vectors are linearly separated. It is well-known that a linear dichotomy has an average generalization error or a learning curve proportional to the dimension of the input space and inversely proportional to the number of given examples in the asymptotic limit. However, it does not hold in the case of kernel methods since the feature vectors lie on a submanifold in the feature space, called the input surface. In this letter, we discuss how the asymptotic average generalization error depends on the relationship between the input surface and the true separating hyperplane in the feature space where the essential dimension of the true separating polynomial, named the class, is important. We show its upper bounds in several cases and confirm these using computer simulations.

In this paper, we regard the sequence of returns as outputs from a parametric compound source. Utilizing the fact that the coding rate of the source shows the amount of information about the return, we describe l-learning algorithms based on the predictive coding idea for estimating an expected information gain concerning future information and give a convergence proof of the information gain. Using the information gain, we propose the ratio w of return loss to information gain as a new criterion to be used in probabilistic action-selection strategies. In experimental results, we found that our w-based strategy performs well compared with the conventional Q-based strategy.

The normalized LMS (N-LMS) algorithm has a disadvantage that the convergence rate is much worse when the input signal is colored. To overcome this, the affine projection algorithm and the block orthogonal projection (BOP) algorithm which applied the block signal processing technique to the N-LMS algorithm are proposed although the reason why they are tough against the colored signal is not given yet. This paper gives the convergence rate of the BOP algorithm for colored input signals, which shows the superiority of the BOP algorithm. To put it concretely, we derive the expression of the convergence rate, propose an approximation method to calculate it, and confirm the result by computer simulations. We also consider the relation between the block size and the convergence rate formally and geometrically.

The numerical property of an adaptive filter algorithm is the most important problem in practical applications. Most fast adaptive filter algorithms have the numerical instability problem and the fast Newton transversal filter (FNTF) algorithms are no exception. In this paper, we propose a numerically stable fast Newton type adaptive filter algorithm. Two problems are dealt with in the paper. First, we derive the proposed algorithm from the order-update fast least squares (FLS) algorithm. This derivation is direct and simple to understand. Second, we give a stability analysis using a linear time-variant state-space method. The transition matrix of the proposed algorithm is given. The eigenvalues of the ensemble average of the transition matrix are shown to be asymptotically all less than unity. This results in a much improved numerical performance compared with the FNTF algorithms. The computer simulations implemented by using a finite-precision arithmetic have confirmed the validity of our analysis.

A three-dimensional topographical simulator PEACE (photo and electron beam lithography analyzing computer engineering system) is discussed. One of the difficulties in resist topographical simulation exists due to the three-dimensional resist development algorithm. An algorithm based on the cell removal model provides accurate and stable results for the three-dimensional resist development process. The program has been adapted to a supercomputer for quick computation. The simulator can successfully perform the three-dimensional development in an absolutely stable manner, and good agreement can be obtained with experiments for both photo and electron beam lithography.>