In this paper, we examine the Rényi entropy rate of stationary ergodic processes. For a special class of stationary ergodic processes, we prove that the Rényi entropy rate always exists and can be approximated by its defining sequence at most polynomially; moreover, using the Markov approximation method, we show that the Rényi entropy rate can be exponentially approximated by that of the Markov approximating sequence, as the Markov order goes to infinity. For the general case, by constructing a counterexample, we disprove the conjecture that the Rényi entropy rate of a general stationary ergodic process always converges to its Shannon entropy rate as$\alpha $goes to 1.

The data-hungry problem, characterized by insufficiency and low-quality of data, poses obstacles for deep learning models. Transfer learning has been a feasible way to transfer knowledge from high-quality external data of source domains to limited data of target domains, which follows a domain-level knowledge transfer to learn a shared posterior distribution. However, they are usually built on strong assumptions, e.g., the domain invariant posterior distribution, which is usually unsatisfied and may introduce noises, resulting in poor generalization ability on target domains. Inspired by Graph Neural Networks (GNNs) that aggregate information from neighboring nodes, we redefine the paradigm as learning a knowledge-enhanced posterior distribution for target domains, namely Knowledge Bridge Learning (KBL). KBL first learns the scope of knowledge transfer by constructing a Bridged-Graph that connects knowledgeable samples to each target sample and then performs sample-wise knowledge transfer via GNNs.KBL is free from strong assumptions and is robust to noises in the source data. Guided by KBL, we propose the Bridged-GNN including an Adaptive Knowledge Retrieval module to build Bridged-Graph and a Graph Knowledge Transfer module. Comprehensive experiments on both un-relational and relational data-hungry scenarios demonstrate the significant improvements of Bridged-GNN compared with SOTA methods

Graphs consisting of vocal nodes ("the vocal minority") and silent nodes ("the silent majority"), namely VS-Graph, are ubiquitous in the real world. The vocal nodes tend to have abundant features and labels. In contrast, silent nodes only have incomplete features and rare labels, e.g., the description and political tendency of politicians (vocal) are abundant while not for ordinary civilians (silent) on the twitter’s social network. Predicting the silent majority remains a crucial yet challenging problem. However, most existing Graph Neural Networks (GNNs) assume that all nodes belong to the same domain, without considering the missing features and distribution-shift between domains, leading to poor ability to deal with VS-Graph. To combat the above challenges, we propose Knowledge Transferable Graph Neural Network (KTGNN), which models distribution-shifts during message passing and learns representation by transferring knowledge from vocal nodes to silent nodes. Specifically, we design the domain-adapted "feature completion and message passing mechanism" for node representation learning while preserving domain difference. And a knowledge transferable classifier based on KL-divergence is followed. Comprehensive experiments on real-world scenarios (i.e., company financial risk assessment and political elections) demonstrate the superior performance of our method. Our source code has been open-sourced1.

In this paper, we focus our attention on the Rényi entropy rate of hidden Markov processes under certain positivity assumptions. The existence of the Rényi entropy rate for such processes is established. Furthermore, we show that, with some extra “fast-forgetting” assumptions, the Rényi entropy rate of the approximating Markov processes exponentially converges to that of the original hidden Markov process, as the Markov order goes to infinity.

In practice, signals may be interfered by hostile jamming or illegal transmission and it is a very challenging task to determine the modulation formats of mixed signals. To tackle this problem, we propose a three-step algorithm called PFS algorithm. In the first step, principal component analysis (PCA) is conducted to suppress the noise. In the second step, the mixed signals are separated via fast independent component analysis (FICA), which transforms the received signals into the components that are maximally independent of each other. In the third step, high-order cumulants (HOCs) and support vector machines (SVMs) are adopted to determine the modulation format of the signal. The numerical experiments show that the PFS algorithm has a superior performance compared to other existing methods.