This paper proposes the soft Bayesian context tree model (Soft-BCT), which is a novel BCT model for real-valued time series. The Soft-BCT considers soft (probabilistic) splits of the context space, instead of hard (deterministic) splits of the context space as in the previous BCT for real-valued time series. A learning algorithm of the Soft-BCT is proposed based on the variational inference. The results of experiments demonstrate the superiority of the Soft-BCT compared to the previous BCT for some datasets.

Deterministic decision trees have difficulty in evaluating uncertainty especially for small samples. To solve this problem, we interpret the decision trees as stochastic models and consider prediction problems in the framework of Bayesian decision theory. Our models have three kinds of parameters: a tree shape, leaf parameters, and inner parameters. To make Bayesian optimal decisions, we have to calculate the posterior distribution of these parameters. Previously, two types of methods have been proposed. One marginalizes out the leaf parameters and samples the tree shape and the inner parameters by Metropolis-Hastings (MH) algorithms. The other marginalizes out both the leaf parameters and the tree shape based on a concept called meta-trees and approximates the posterior distribution for the inner parameters by a bagging-like method. In this paper, we propose a novel MH algorithm where the leaf parameters and the tree shape are marginalized out by using the meta-trees and only the inner parameters are sampled. Moreover, we update all the inner parameters simultaneously in each MH step. This algorithm accelerates the convergence and mixing of the Markov chain. We evaluate our algorithm on various benchmark datasets with other state-of-the-art methods. Further, our model provides a novel statistical evaluation of feature importance.

This study focuses on mixed-variable black-box optimization (MV-BBO), addressing continuous, integer, and categorical variables. Many real-world MV-BBO problems involve dependencies among these different types of variables, requiring efficient methods to optimize them simultaneously. Recently, stochastic optimization methods leveraging the mechanism of the covariance matrix adaptation evolution strategy have shown promising results in mixed-integer or mixed-category optimization. However, such methods cannot handle the three types of variables simultaneously. In this study, we propose CatCMA with Margin (CatCMAwM), a stochastic optimization method for MV-BBO that jointly optimizes continuous, integer, and categorical variables. CatCMAwM is developed by incorporating a novel integer handling into CatCMA, a mixed-category black-box optimization method employing a joint distribution of multivariate Gaussian and categorical distributions. The proposed integer handling is carefully designed by reviewing existing integer handlings and following the design principles of CatCMA. Even when applied to mixed-integer problems, it stabilizes the marginal probability and improves the convergence performance of continuous variables. Numerical experiments show that CatCMAwM effectively handles the three types of variables, outperforming state-of-the-art Bayesian optimization methods and baselines that simply incorporate existing integer handlings into CatCMA.

Black-box optimization problems often require simultaneously optimizing different types of variables, such as continuous, integer, and categorical variables. Unlike integer variables, categorical variables do not necessarily have a meaningful order, and the discretization approach of continuous variables does not work well. Although several Bayesian optimization methods can deal with mixed-category black-box optimization (MC-BBO), they suffer from a lack of scalability to high-dimensional problems and internal computational cost. This paper proposes CatCMA, a stochastic optimization method for MC-BBO problems, which employs the joint probability distribution of multivariate Gaussian and categorical distributions as the search distribution. CatCMA updates the parameters of the joint probability distribution in the natural gradient direction. CatCMA also incorporates the acceleration techniques used in the covariance matrix adaptation evolution strategy (CMA-ES) and the stochastic natural gradient method, such as step-size adaptation and learning rate adaptation. In addition, we restrict the ranges of the categorical distribution parameters by margin to prevent premature convergence and analytically derive a promising margin setting. Numerical experiments show that the performance of CatCMA is superior and more robust to problem dimensions compared to state-of-the-art Bayesian optimization algorithms.

In several real-world applications in medical and control engineering, there are unsafe solutions whose evaluations involve inherent risk. This optimization setting is known as safe optimization and formulated as a specialized type of constrained optimization problem with constraints for safety functions. Safe optimization requires performing efficient optimization without evaluating unsafe solutions. A few studies have proposed the optimization methods for safe optimization based on Bayesian optimization and the evolutionary algorithm. However, Bayesian optimization-based methods often struggle to achieve superior solutions, and the evolutionary algorithm-based method fails to effectively reduce unsafe evaluations. This study focuses on CMA-ES as an efficient evolutionary algorithm and proposes an optimization method termed safe CMA-ES. The safe CMA-ES is designed to achieve both safety and efficiency in safe optimization. The safe CMA-ES estimates the Lipschitz constants of safety functions transformed with the distribution parameters using the maximum norm of the gradient in Gaussian process regression. Subsequently, the safe CMA-ES projects the samples to the nearest point in the safe region constructed with the estimated Lipschitz constants. The numerical simulation using the benchmark functions shows that the safe CMA-ES successfully performs optimization, suppressing the unsafe evaluations, while the existing methods struggle to significantly reduce the unsafe evaluations.

This paper deals with the problem of soft guessing under log-loss distortion (logarithmic loss) that was recently investigated by [Wu and Joudeh, IEEE ISIT, pp. 466–471, 2023]. We extend this problem to soft guessing allowing errors, i.e., at each step, a guesser decides whether to stop the guess or not with some probability and if the guesser stops guessing, then the guesser declares an error. We show that the minimal expected value of the cost of guessing under the constraint of the error probability is characterized by the smooth Rényi entropy. Furthermore, we carry out an asymptotic analysis for a stationary and memoryless source.

This paper deals with variable-length lossy source coding. We consider a single-shot approach to source coding in which the source to be compressed is a random variable$X$. The performance criterion of codeword lengths is a cumulant generating function of codeword lengths and the performance criterion of a distortion measure is a logarithmic loss distortion. We derive an upper bound of the cumulant generating function of codeword lengths under the condition that the logarithmic loss distortion is less than or equal to$D$, where$D\geq 0$is a given distortion level. The upper bound is characterized by the Rényi entropy of the random variable$[\frac{X}{\lfloor\exp(D)\rfloor}]$. Numerical examples show that there are cases where the new upper bound is tighter than the upper bound derived in a previous study.

We study the classic problem of prediction with expert advice under bandit feedback. Our model assumes that one action, corresponding to the learner's abstention from play, has no reward or loss on every trial. We propose the CBA (Confidence-rated Bandits with Abstentions) algorithm, which exploits this assumption to obtain reward bounds that can significantly improve those of the classical Exp4 algorithm. Our problem can be construed as the aggregation of confidence-rated predictors, with the learner having the option to abstain from play. We are the first to achieve bounds on the expected cumulative reward for general confidence-rated predictors. In the special case of specialists, we achieve a novel reward bound, significantly improving previous bounds of SpecialistExp (treating abstention as another action). We discuss how CBA can be applied to the problem of adversarial contextual bandits with the option of abstaining from selecting any action. We are able to leverage a wide range of inductive biases, outperforming previous approaches both theoretically and in preliminary experimental analysis. Additionally, we achieve a reduction in runtime from quadratic to almost linear in the number of contexts for the specific case of metric space contexts.

Motivated by the high prediction performance of convolutional neural networks (CNNs), several works have applied them to tabular datasets. As CNNs are built to accept images, several transformations of tabular data have been proposed to obtain images. However, existing methods transform the tabular data into images prior to CNN training, which fails to take the prediction error into account. Additionally, they employ all features from the tables, including unimportant ones, to produce the images. Moreover, the created images might not become human-interpretable because they do not consider the interpretability of images as a metric. To overcome these problems, we propose a hard attention-based converter combined with a convolutional neural network (HACNet), consisting of an attention-based table-to-image converter and a CNN-based predictor. HACNet trains its components simultaneously by minimizing CNN prediction loss and mean squared error (MSE) between created and template images. Minimizing this MSE loss allows us to visually distinguish the created images with different labels. The attention-based converter selects exactly one feature for each pixel in the image via its hard attention mechanism with Gumbel-Softmax, enabling feature selection. We experimentally show that HACNet produces human-interpretable images, reduces used features, and achieves prediction performances comparative with existing methods on several benchmark datasets.

This study targets the mixed-integer black-box optimization (MI-BBO) problem where continuous and integer variables should be optimized simultaneously. The covariance matrix adaptation evolution strategy (CMA-ES), our focus in this study, is a population-based stochastic search method that samples solution candidates from a multivariate Gaussian distribution (MGD), which shows excellent performance in continuous black-box optimization. The parameters of MGD, mean and (co)variance, are updated based on the evaluation value of candidate solutions in the CMA-ES. If the CMA-ES is applied to the MI-BBO with straightforward discretization, however, the variance corresponding to the integer variables becomes much smaller than the granularity of the discretization before reaching the optimal solution, which leads to the stagnation of the optimization. In particular, when binary variables are included in the problem, this stagnation more likely occurs because the granularity of the discretization becomes wider, and the existing integer handling for the CMA-ES does not address this stagnation. To overcome these limitations, we propose a simple integer handling for the CMA-ES based on lower-bounding the marginal probabilities associated with the generation of integer variables in the MGD. The numerical experiments on the MI-BBO benchmark problems demonstrate the efficiency and robustness of the proposed method. Furthermore, to demonstrate the generality of the idea of the proposed method, in addition to the single-objective optimization case, we incorporate it into multi-objective CMA-ES and verify its performance on bi-objective mixed-integer benchmark problems.

The covariance matrix adaptation evolution strategy (CMA-ES) is an efficient continuous black-box optimization method. The CMA-ES possesses many attractive features, including invariance properties and a well-tuned default hyperparameter setting. Moreover, several components to specialize the CMA-ES have been proposed, such as noise handling and constraint handling. To utilize these advantages in mixed-integer optimization problems, the CMA-ES with margin has been proposed. The CMA-ES with margin prevents the premature convergence of discrete variables by the margin correction, in which the distribution parameters are modified to leave the generation probability for changing the discrete variable. The margin correction has been applied to (μ/μw,Λ)-CMA-ES, while this paper introduces the margin correction into (1+1)-CMA-ES, an elitist version of CMA-ES. The (1+1)-CMA-ES is often advantageous for unimodal functions and can be computationally less expensive. To tackle the performance deterioration on mixed-integer optimization, we use the discretized elitist solution as the mean of the sampling distribution and modify the margin correction not to move the elitist solution. The numerical simulation using benchmark functions on mixed-integer, integer, and binary domains shows that (1+1)-CMA-ES with margin outperforms the CMA-ES with margin and is better than or comparable with several specialized methods to a particular search domain.

This paper develops an approximation to the (effective) $p$-resistance and applies it to multi-class clustering. Spectral methods based on the graph Laplacian and its generalization to the graph $p$-Laplacian have been a backbone of non-euclidean clustering techniques. The advantage of the $p$-Laplacian is that the parameter $p$ induces a controllable bias on cluster structure. The drawback of $p$-Laplacian eigenvector based methods is that the third and higher eigenvectors are difficult to compute. Thus, instead, we are motivated to use the $p$-resistance induced by the $p$-Laplacian for clustering. For $p$-resistance, small $p$ biases towards clusters with high internal connectivity while large $p$ biases towards clusters of small “extent,” that is a preference for smaller shortest-path distances between vertices in the cluster. However, the $p$-resistance is expensive to compute. We overcome this by developing an approximation to the $p$-resistance. We prove upper and lower bounds on this approximation and observe that it is exact when the graph is a tree. We also provide theoretical justification for the use of $p$-resistance for clustering. Finally, we provide experiments comparing our approximated $p$-resistance clustering to other $p$-Laplacian based methods.

In recent years, Bayesian counterparts of the context tree weighting method are studied for many tasks. All these tasks require a hyperparameter setting of the prior distribution for context tree models. Therefore, we provide a framework for statistically learning these hyperparameters from data. Specifically, we consider a hierarchical Bayesian model that assumes hyperprior distributions behind the hyperparameters and learn them using an empirical variational Bayesian (EVB) method. This is the first study to propose an EVB method on the Bayesian context trees. The derived algorithm has a suggestive form that consists of subroutines partially optimal to each local probabilistic model.

Abstract For hypergraph clustering, various methods have been proposed to define hypergraph p -Laplacians in the literature. This work proposes a general framework for an abstract class of hypergraph p -Laplacians from a differential-geometric view. This class includes previously proposed hypergraph p -Laplacians and also includes previously unstudied novel generalizations. For this abstract class, we extend current spectral theory by providing an extension of nodal domain theory for the eigenvectors of our hypergraph p -Laplacian. We use this nodal domain theory to provide bounds on the eigenvalues via a higher-order Cheeger inequality. Following our extension of spectral theory, we propose a novel hypergraph partitioning algorithm for our generalized p -Laplacian. Our empirical study shows that our algorithm outperforms spectral methods based on existing p -Laplacians.

This paper investigates the problem of variable-length source coding with the criteria of the normalized cumulant generating function of codeword lengths and the excess distortion probability. We analyze the non-asymptotic fundamental limit of the normalized cumulant generating function of codeword lengths under the constraint that the excess distortion probability is allowed up to$\epsilon \in [0,1)$. Our non-asymptotic achievability and converse bounds are characterized by the quantity related to the Rényi entropy.

We propose a theoretical framework of multi-way similarity to model real-valued data into hypergraphs for clustering via spectral embedding. For graph cut based spectral clustering, it is common to model real-valued data into graph by modeling pairwise similarities using kernel function. This is because the kernel function has a theoretical connection to the graph cut. For problems where using multi-way similarities are more suitable than pairwise ones, it is natural to model as a hypergraph, which is generalization of a graph. However, although the hypergraph cut is well-studied, there is not yet established a hypergraph cut based framework to model multi-way similarity. In this paper, we formulate multi-way similarities by exploiting the theoretical foundation of kernel function. We show a theoretical connection between our formulation and hypergraph cut in two ways, generalizing both weighted kernel k-means and the heat kernel, by which we justify our formulation. We also provide a fast algorithm for spectral clustering. Our algorithm empirically shows better performance than existing graph and other heuristic modeling methods.

This study targets the mixed-integer black-box optimization (MI-BBO) problem where continuous and integer variables should be optimized simultaneously. The CMA-ES, our focus in this study, is a population-based stochastic search method that samples solution candidates from a multivariate Gaussian distribution (MGD), which shows excellent performance in continuous BBO. The parameters of MGD, mean and (co)variance, are updated based on the evaluation value of candidate solutions in the CMA-ES. If the CMA-ES is applied to the MI-BBO with straightforward discretization, however, the variance corresponding to the integer variables becomes much smaller than the granularity of the discretization before reaching the optimal solution, which leads to the stagnation of the optimization. In particular, when binary variables are included in the problem, this stagnation more likely occurs because the granularity of the discretization becomes wider, and the existing modification to the CMA-ES does not address this stagnation. To overcome these limitations, we propose a simple modification of the CMA-ES based on lower-bounding the marginal probabilities associated with the generation of integer variables in the MGD. The numerical experiments on the MI-BBO benchmark problems demonstrate the efficiency and robustness of the proposed method.

The hierarchical and recursive expressive capability of rooted trees is applicable to represent statistical models in various areas, such as data compression, image processing, and machine learning. On the other hand, such hierarchical expressive capability causes a problem in tree selection to avoid overfitting. One unified approach to solve this is a Bayesian approach, on which the rooted tree is regarded as a random variable and a direct loss function can be assumed on the selected model or the predicted value for a new data point. However, all the previous studies on this approach are based on the probability distribution on full trees, to the best of our knowledge. In this paper, we propose a generalized probability distribution for any rooted trees in which only the maximum number of child nodes and the maximum depth are fixed. Furthermore, we derive recursive methods to evaluate the characteristics of the probability distribution without any approximations.

For the problem of parameter estimation in a Bayesian setting, information-theoretic lower bounds of the Bayes risk have been investigated. Previous studies have proven the lower bound of the Bayes risk in a different manner and characterized the lower bound via different quantities such as the mutual information, Sibson’s α-mutual information, and Csiszár’s f-informativity. In this paper, we introduce an inequality called a "meta-bound for lower bounds of the Bayes risk" and show that the previous results can be derived from this bound.

Suppose that we have two training sequences generated by parametrized distributions$P_{\theta}$and$P_{\varepsilon^{*}}$, where$\theta$* and$\xi^{*}$are unknown true parameters. Given training sequences, we study the problem of classifying whether a test sequence was generated according to$P_{\theta}$* or$P_{\xi^{*}}$. This problem can be thought of as a hypothesis testing problem and our aim is to analyze the weighted sum of type-I and type-II error probabilities. Utilizing the analysis of the codeword lengths of the Bayes code, our previous study derived more refined bounds on the error probability than known previously. However, our previous study had the following deficiencies: i) the prior distributions of$\theta$and$\xi$are the same; ii) the prior distributions of two hypotheses are uniform; iii) no numerical calculation at finite blocklength. This study solves these problems. We remove the restrictions i) and ii) and derive more general results than obtained previously. To deal with problem iii), we perform a numerical calculation for a concrete model.

The context tree source is a source model in which the occurrence probability of symbols is determined from a finite past sequence, and is a broader class of sources that includes i.i.d. and Markov sources. This paper proposes a source model such that its subsequence is generated from a different context tree model. The Bayes code for such sources requires weighting of the posterior probability distributions for the change patterns of the context tree source and all possible context tree models. Therefore, the challenge is how to reduce this exponential order computational complexity. In this paper, we assume a special class of prior probability distribution of change patterns and context tree models, and propose an efficient Bayes coding algorithm whose computational complexity is the polynomial order. A full version of this paper is accessible at: https://arxiv.org/abs/2105.05163

Suppose that we have two training sequences generated by parametrized distributions ${P_{\theta _1^{\ast}}}$ and ${P_{\theta _2^{\ast}}}$, where $\theta _1^{\ast}$ and $\theta _2^{\ast}$ are unknown. Given training sequences, we study the problem of classifying whether a test sequence was generated according to ${P_{\theta _1^{\ast}}}$ or ${P_{\theta _2^{\ast}}}$. This problem can be thought of as a hypothesis testing problem and the weighted sum of type-I and type-II error probabilities is analyzed. To prove the results, we utilize the analysis of the codeword lengths of the Bayes code. It is shown that upper and lower bounds of the probability of error are characterized by the terms containing the Chernoff information, the dimension of a parameter space, and the ratio of the length between the training sequences and the test sequence. Further, we generalize the part of the preceding results to multiple hypotheses setup.

High sensitivity of neural architecture search (NAS) methods against their input such as step-size (i.e., learning rate) and search space prevents practitioners from applying them out-of-the-box to their own problems, albeit its purpose is to automate a part of tuning process. Aiming at a fast, robust, and widely-applicable NAS, we develop a generic optimization framework for NAS. We turn a coupled optimization of connection weights and neural architecture into a differentiable optimization by means of stochastic relaxation. It accepts arbitrary search space (widely-applicable) and enables to employ a gradient-based simultaneous optimization of weights and architecture (fast). We propose a stochastic natural gradient method with an adaptive step-size mechanism built upon our theoretical investigation (robust). Despite its simplicity and no problem-dependent parameter tuning, our method exhibited near state-of-the-art performances with low computational budgets both on image classification and inpainting tasks.

This paper investigates the problem of guessing subject to distortion, which was introduced by Arikan and Merhav. While the primary concern of the previous study was asymptotic analysis, our primary concern is non-asymptotic analysis. We prove non-asymptotic achievability and converse bounds of the moment of the number of guesses without side information (resp. with side information) by using a quantity based on the Rényi entropy (resp. the Arimoto-Rényi conditional entropy). Also, we introduce an error probability and show similar results. Further, from our bounds, we derive a single-letter characterization of the asymptotic exponent of guessing moment for a stationary memoryless source.

The graph Laplacian plays key roles in information processing of relational data, and has analogies with the Laplacian in differential geometry. In this paper, we generalize the analogy between graph Laplacian and differential geometry to the hypergraph setting, and propose a novel hypergraph p-Laplacian. Unlike the existing two-node graph Laplacians, this generalization makes it possible to analyze hypergraphs, where the edges are allowed to connect any number of nodes. Moreover, we propose a semi-supervised learning method based on the proposed hypergraph p-Laplacian, and formalize them as the analogue to the Dirichlet problem, which often appears in physics. We further explore theoretical connections to normalized hypergraph cut on a hypergraph, and propose normalized cut corresponding to hypergraph p-Laplacian. The proposed p-Laplacian is shown to outperform standard hypergraph Laplacians in the experiment on a hypergraph semi-supervised learning and normalized cut setting.

This paper considers the problem of variable-length lossy source coding. The performance criteria are the excess distortion probability and the cumulant generating function of codeword lengths. We derive a non-asymptotic fundamental limit of the cumulant generating function of codeword lengths allowing positive excess distortion probability. It is shown that the achievability and converse bounds are characterized by the Rényi entropy-based quantity. In the proof of the achievability result, the explicit code construction is provided. Further, we investigate an asymptotic single-letter characterization of the fundamental limit for a stationary memoryless source. A full version of this paper is accessible at: http://arxiv.org/abs/1801.02496

This paper shows some new results for the problem of variable-length lossy source coding. We deal with the case where both the excess distortion probability and the overflow probability of codeword lengths are less than or equal to positive constants. Our previous study for the problem of variable-length (noiseless) lossy source coding has derived the general formula of the infimum of the thresholds on the overflow probability by using the quantity based on the smooth max entropy. This study extends this result in two directions. First, we derive the single-letter characterization of the infimum of the thresholds on the overflow probability for stationary memoryless sources. Second, for the problem of variable-length noisy lossy source coding, also known as the problem of remote lossy source coding, we establish the general nonasymptotic formula on the converse bound by using the new quantity based on the smooth max entropy.

This paper considers the problem of variable-length intrinsic randomness. We propose the average variational distance as the performance criterion from the viewpoint of a dual relationship with the problem formulation of variable-length resolvability. Previous study has derived the general formula of the ϵ-variable-length resolvability. We derive the general formula of the ϵ-variable-length intrinsic randomness. Namely, we characterize the supremum of the mean length under the constraint that the value of the average variational distance is smaller than or equal to a constant ϵ. Our result clarifies a dual relationship between the general formula of ϵ-variable-length resolvability and that of ϵ-variable-length intrinsic randomness. We also derive a lower bound of the quantity characterizing our general formula.

This paper investigates the problem of variable-length lossy source coding. We deal with the case where both the excess distortion probability and the overflow probability of codeword length are less than or equal to positive constants. The infimum of the thresholds on the overflow probability is characterized by a smooth max entropy-based quantity. Both non-asymptotic and asymptotic cases are analyzed. To show the achievability results, we do not utilize the random coding argument but give an explicit code construction.

In recent years, camera identification methods have attracted attention in the field of digital forensics. The existing camera identification methods use features, such as the Exif header data and image noise, that indicate the characteristics of the camera. Of them, photo-response non-uniformity (PRNU) noise contains the unique features of an image sensor and is different for each individual camera. A camera identification method using the PRNU noise should have high identification ability, and a camera identification method using the pairwise magnitude relations of the clustered PRNU noise was previously proposed. In general, identification accuracy is estimated from test data sets, such as the Dresden image database. However, identification accuracy can be evaluated only with respect to the range of images within a database in the conventional evaluation method. A more detailed accuracy evaluation method is required for practical use. Furthermore, studies have not yet reported a false acceptance rate (FAR) evaluation method for the clustered PRNU pair-based camera identification capable of guaranteeing a low FAR (e.g., FAR = 10-9). In this paper, we proposed a new pixel clustering method that guarantees An FAR for camera identification using pairs of clustered PRNU noise, and evaluate its FAR based on a probability calculation of a mathematical model. In addition, we investigate the appropriate cluster size by using the Shapiro-Wilk test for an FAR evaluation. We show that it is possible to reliably calculate the FAR of a clustered PRNU noise pair-based camera identification method by using the proposed evaluation method. To demonstrate the validity of our calculations, we compare the actual identification result with the result of the proposed calculation. In this case, we used 16958 query images from the Dresden image database, which is a benchmark data set. The results of our evaluation indicate that this identification method maintains a false rejection rate of less than 5% (10%) for 5 (8) of the 10 tested cameras even for FAR = 10-9.

This paper considers universal lossless variable-length source coding problem and deals with one of the fundamental limits and pointwise asymptotics of the Bayes code for stationary ergodic finite order Markov sources. As investigation of the fundamental limits, we show upper and lower bounds of the minimum rate such that the probability which exceeds it is less than ε ∈ (0, 1). Furthermore, we prove that the codeword length of the Bayes code satisfies the asymptotic normality (pointwise √n asymptotics) and the law of the iterated logarithm (pointwise √n log log n asymptotics), where n represents length of a source sequence and “log” is the natural logarithm.

In social networking services (SNSs), persistent topics are extremely rare and valuable. In this paper, we propose an algorithm for the detection of persistent topics in SNSs based on Topic Graph. A topic graph is a subgraph of the ordinary social network graph that consists of the users who shared a certain topic up to some time point. Based on the assumption that the time-evolutions of the topic graphs associated with a persistent and non-persistent topics are different, we propose to detect persistent topics by performing anomaly detection on the feature values extracted from the time-evolution of the topic graph. For anomaly detection, we use principal component analysis to capture the subspace spanned by normal (non-persistent) topics. We demonstrate our technique on a real data set we gathered from Twitter and show that it performs significantly better than a base-line method based on power law curve fitting and the linear influence model.