Reinforcement learning (RL) is the training of smart agents to take action in an interactive environment in order to maximize the cumulative reward. It has achieved great success in many applications. However, in some domains, e.g., in financial markets, two issues hampered the wide usage of RL. First, it is expensive to interact with the environment as global computation is incurred when receiving new signals. Second, an agent itself may find it difficult to sense changes in the environment. In this paper, we introduce a general framework named Dirichlet Policy Reuse (DPR) to tackle these issues in a financial investment setting. On one hand, DPR will reuse the historical experience as much as possible to control trial and error within a lower level. On the other hand, DPR is ought to be aggressive enough to make exploration when the agent encounters disagreement from the model consensus in our policy library. We validated our framework on several tasks in the financial investment context, including single asset trading and portfolio investment. We show in experiments that DPR can reuse historical knowledge, detect the potential changes in the environment and make appropriate adaptation timely. This paper is a preliminary study on the dynamic policy self-adaptation strategy between knowledge reuse and environment exploration. It lays a good foundation for future research on RL problems under non-stationary environments and especially for investment scenarios.

Film release time play an important part in box office revenues due to obvious seasonality demand in the film industry and severe competition among films shown at the same time. In this paper, we study how film studios choose release time for movies they produce to maximize their box offices. We first formalize this problem as an attraction competition game where players (film studios) consider both potential profits and competitors' choices when deciding the release time. Then we prove that there always exists a pure Nash equilibrium and give the sufficient condition of the uniqueness of the Nash equilibrium. Our model can be generalized to an extensive game and we compute the subgame-perfect equilibrium for homogeneous players. For the case that one film studio could have multiple movies to release, we prove that finding a player's best response is NP-hard and it does not guarantee the existence of a pure Nash equilibrium. Experiments are provided to support the soundness of our model. In the final state, most of the film studios, accounting for 84 percent of the market, would not change their release time. The behaviors of film studios imply they are following some strategies to reach a Nash equilibrium.

In classification problem, Bayesian networks play an important role because of its efficiency and interpretability. Bayesian networks learning methods require enough data to produce reliable results. Unfortunately, in practice, the training data are often either too few, expensive to label, or easy to be outdated. However, there may be sufficient labeled data that are available in a different but related domain. Learning reliable Bayesian networks from limited data is difficult; and transfer learning might be used to improve the robustness of learned networks by combining data from auxiliary and related labeled dataset. In this paper, we propose a novel transfer learning method for Bayesian networks for classification that considers both structure and parameter learning. Our solution is to first construct the initial Bayesian networks model for auxiliary labeled data, and then revise the model according to an Expectation-Maximization (EM) algorithm, structure and parameters are revised by turns, in order to make it applicable to the target unlabeled dataset. We mainly apply our method on a special type of Bayesian networks, namely tree-based Bayesian network. To validate our approach, we evaluated the method on a real and typical classification scenario - text classification problem. We compared our method with other transfer learning method as well as the traditional supervised and semi-supervised learning algorithms. The experimental results show that our algorithm is very effective and obtains a significant improvement when we transfer knowledge from related dataset.

This paper is concerned with model-based clustering of discrete data. Latent class models (LCMs) are usually used for the task. An LCM consists of a latent variable and a number of attributes. It makes the overly restrictive assumption that the attributes are mutually independent given the latent variable. We propose a novel method to relax the assumption. The key idea is to partition the attributes into groups such that correlations among the attributes in each group can be properly modeled by using one single latent variable. The latent variables for the attribute groups are then used to build a number of models and one of them is chosen to produce the clustering results. Extensive empirical studies have been conducted to compare the new method with LCM and several other methods (K-means, kernel K-means and spectral clustering) that are not model-based. The new method outperforms the alternative methods in most cases and the differences are often large.