Regularization in modern machine learning is crucial, and it can take various forms in algorithmic design: training set, model family, error function, regularization terms, and optimizations. In particular, the learning rate, which can be interpreted as a temperature-like parameter within the statistical mechanics of learning, plays a crucial role in neural network training. Indeed, many widely adopted training strategies basically just define the decay of the learning rate over time. This process can be interpreted as decreasing a temperature, using either a global learning rate (for the entire model) or a learning rate that varies for each parameter. This paper proposes TempBalance, a straightforward yet effective layer-wise learning rate method. TempBalance is based on Heavy-Tailed Self-Regularization (HT-SR) Theory, an approach which characterizes the implicit self-regularization of different layers in trained models. We demonstrate the efficacy of using HT-SR-motivated metrics to guide the scheduling and balancing of temperature across all network layers during model training, resulting in improved performance during testing. We implement TempBalance on CIFAR10, CIFAR100, SVHN, and TinyImageNet datasets using ResNets, VGGs and WideResNets with various depths and widths. Our results show that TempBalance significantly outperforms ordinary SGD and carefully-tuned spectral norm regularization. We also show that TempBalance outperforms a number of state-of-the-art optimizers and learning rate schedulers.

In this paper, we address the problem of channel allocation for femtocells that share the use of regular macrocell spectrum. The femto basestation (FBS) scheduling problem is formulated in the form of restless multiarmed bandit (RMAB) framework. Our goal is to choose the arms/channels that maximize the total expected discounted reward over infinite horizon while minimizing the induced interference due to channel sharing with macrocell. Without direct observation of true channel state, we use the available macrocell user feedback known as channel quality indicator (CQI). In general, the RMAB problem is P-SPACE hard. We propose a heuristic low complexity indexing policy referred as approximated Whittle index to rank available channels for FBS. Although finding a closed form channel ranking solution typically involve dynamic programming, we show that based on the partial channel information within CQI, there exists a closed form for the channel index. Moreover, we demonstrate the performance advantage of the proposed indexing policy over a myopic policy.

MOTIVATION: Long non-coding RNAs (lncRNAs) have been implicated in the regulation of diverse biological functions. The number of newly identified lncRNAs has increased dramatically in recent years but their expression and function have not yet been described from most diseases. To elucidate lncRNA function in human disease, we have developed a novel network based method (NLCFA) integrating correlations between lncRNA, protein coding genes and noncoding miRNAs. We have also integrated target gene associations and protein-protein interactions and designed our model to provide information on the combined influence of mRNAs, lncRNAs and miRNAs on cellular signal transduction networks. RESULTS: We have generated lncRNA expression profiles from the CD34+ haematopoietic stem and progenitor cells (HSPCs) from patients with Myelodysplastic syndromes (MDS) and healthy donors. We report, for the first time, aberrantly expressed lncRNAs in MDS and further prioritize biologically relevant lncRNAs using the NLCFA. Taken together, our data suggests that aberrant levels of specific lncRNAs are intimately involved in network modules that control multiple cancer-associated signalling pathways and cellular processes. Importantly, our method can be applied to prioritize aberrantly expressed lncRNAs for functional validation in other diseases and biological contexts. AVAILABILITY AND IMPLEMENTATION: The method is implemented in R language and Matlab. CONTACT: [email protected]. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

BACKGROUND: Endometrial cancers (ECs) are one of the most common types of malignant tumor in females. Substantial efforts had been made to identify significantly mutated genes (SMGs) in ECs and use them as biomarkers for the classification of histological subtypes and the prediction of clinical outcomes. However, the impact of non-significantly mutated genes (non-SMGs), which may also play important roles in the prognosis of EC patients, has not been extensively studied. Therefore, it is essential for the discovery of biomarkers in ECs to further investigate the non-SMGs that were highly associated with clinical outcomes. RESULTS: For the 9681 non-SMGs reported by the mutation annotation pipeline, there were 1053, 1273 and 395 non-SMGs differentially expressed between the patient groups divided by the clinical endpoints of histological grade, histological type as well as the International Federation of Gynecology and Obstetrics (FIGO) stage of ECs, respectively. In the gene set enrichment analysis, the cancer-related pathways, namely neuroactive ligand-receptor interaction signaling pathway, cAMP signaling pathway and calcium signaling pathway, were significantly enriched with the differentially expressed non-SMGs for all the three endpoints. We further identified 23, 19 and 24 non-SMGs, which were highly associated with histological grade, histological type and FIGO stage, respectively, from the differentially expressed non-SMGs by using the variable combination population analysis (VCPA) approach and found that 69.6% (16/23), 78.9% (15/19) and 66.7% (16/24) of the identified non-SMGs had been previously reported to be correlated with cancers. In addition, the averaged areas under the receiver operating characteristic curve (AUCs) achieved by the predictive models with identified non-SMGs as predictors in predicting histological type, histological grade, and FIGO stage were 0.993, 0.961 and 0.832, respectively, which were superior to those achieved by the models with SMGs as features (averaged AUCs = 0.928, 0.864 and 0.535, resp.). CONCLUSIONS: Besides the SMGs, the non-SMGs reported in the mutation annotation analysis may also involve the crucial genes that were highly associated with clinical outcomes. Combining the mutation status with the gene expression profiles can efficiently identify the cancer-related non-SMGs as predictors for cancer prognostic prediction and provide more supplemental candidates for the discovery of biomarkers.

MOTIVATION: MicroRNAs (miRNAs) are short non-coding RNAs that play important roles in post-transcriptional regulations as well as other important biological processes. Recently, accumulating evidences indicate that miRNAs are extensively involved in cancer. However, it is a big challenge to identify which miRNAs are related to which cancer considering the complex processes involved in tumors, where one miRNA may target hundreds or even thousands of genes and one gene may regulate multiple miRNAs. Despite integrative analysis of matched gene and miRNA expression data can help identify cancer-associated miRNAs, such kind of data is not commonly available. On the other hand, there are huge amount of gene expression data that are publicly accessible. It will significantly improve the efficiency of characterizing miRNA's function in cancer if we can identify cancer miRNAs directly from gene expression data. RESULTS: We present a novel computational framework to identify the cancer-related miRNAs based solely on gene expression profiles without requiring either miRNA expression data or the matched gene and miRNA expression data. The results on multiple cancer datasets show that our proposed method can effectively identify cancer-related miRNAs with higher precision compared with other popular approaches. Furthermore, some of our novel predictions are validated by both differentially expressed miRNAs and evidences from literature, implying the predictive power of our proposed method. In addition, we construct a cancer-miRNA-pathway network, which can help explain how miRNAs are involved in cancer. AVAILABILITY AND IMPLEMENTATION: The R code and data files for the proposed method are available at http://comp-sysbio.org/miR_Path/ CONTACT: [email protected] SUPPLEMENTARY INFORMATION: supplementary data are available at Bioinformatics online.

We consider a large-scale cyber network with N components. Each component is either in a healthy state or an abnormal state. To model scenarios where attacks to the network may not follow a stochastic process and the attackers may adapt to the actions of the intrusion detection system (IDS) in an arbitrary and unknown way, we adopt a non-stochastic model in which the attack process at each component can be any unknown deterministic sequence. Due to resource constraints, the IDS can only choose K (K <; N) components to probe at each time. An abnormal component incurs a cost per unit time (depending on the criticality of the component) until it is probed and fixed. The objective is a dynamic probing strategy under the performance measure of regret, defined as the performance loss compared to that of a genie who knows the entire attack processes a priori and probes optimally (under certain constraints) based on this knowledge. We propose a policy that achieves sublinear regret order, thus offers the same time averaged performance as that of the omniscient genie.

MOTIVATION: Reconstruction of gene regulatory networks (GRNs) is of utmost interest to biologists and is vital for understanding the complex regulatory mechanisms within the cell. Despite various methods developed for reconstruction of GRNs from gene expression profiles, they are notorious for high false positive rate owing to the noise inherited in the data, especially for the dataset with a large number of genes but a small number of samples. RESULTS: In this work, we present a novel method, namely NARROMI, to improve the accuracy of GRN inference by combining ordinary differential equation-based recursive optimization (RO) and information theory-based mutual information (MI). In the proposed algorithm, the noisy regulations with low pairwise correlations are first removed by using MI, and the redundant regulations from indirect regulators are further excluded by RO to improve the accuracy of inferred GRNs. In particular, the RO step can help to determine regulatory directions without prior knowledge of regulators. The results on benchmark datasets from Dialogue for Reverse Engineering Assessments and Methods challenge and experimentally determined GRN of Escherichia coli show that NARROMI significantly outperforms other popular methods in terms of false positive rates and accuracy. AVAILABILITY: All the source data and code are available at: http://csb.shu.edu.cn/narromi.htm.

We consider the restless multiarmed bandit problem with unknown dynamics in which a player chooses one out ofNarms to play at each time. The reward state of each arm transits according to an unknown Markovian rule when it is played and evolves according to an arbitrary unknown random process when it is passive. The performance of an arm selection policy is measured by regret, defined as the reward loss with respect to the case where the player knows which arm is the most rewarding and always plays the best arm. We construct a policy with an interleaving exploration and exploitation epoch structure that achieves a regret with logarithmic order. We further extend the problem to a decentralized setting where multiple distributed players share the arms without information exchange. Under both an exogenous restless model and an endogenous restless model, we show that a decentralized extension of the proposed policy preserves the logarithmic regret order as in the centralized setting. The results apply to adaptive learning in various dynamic systems and communication networks, as well as financial investment.

We consider a large-scale cyber network with N components. Each component is either in a healthy state (0) or an abnormal state (1). Due to intrusions, the state of each component transits from 0 to 1 over time according to an arbitrary stochastic process. At each time, a subset of K (K <; N) components are probed and those observed in abnormal states are fixed. The objective is to design a dynamic probing strategy that minimizes the long-term network cost incurred at all abnormal components. We formulate the problem as a Restless Multi-Armed Bandit (RMAB) process. We show that this class of RMAB is indexable and Whittle index can be obtained in closed-form. For homogeneous networks, we show that Whittle index policy achieves the optimal performance with a simple structure that does not require any prior knowledge on the intrusion processes.

BACKGROUND: Cancers, a group of multifactorial complex diseases, are generally caused by mutation of multiple genes or dysregulation of pathways. Identifying biomarkers that can characterize cancers would help to understand and diagnose cancers. Traditional computational methods that detect genes differentially expressed between cancer and normal samples fail to work due to small sample size and independent assumption among genes. On the other hand, genes work in concert to perform their functions. Therefore, it is expected that dysregulated pathways will serve as better biomarkers compared with single genes. RESULTS: In this paper, we propose a novel approach to identify dysregulated pathways in cancer based on a pathway interaction network. Our contribution is three-fold. Firstly, we present a new method to construct pathway interaction network based on gene expression, protein-protein interactions and cellular pathways. Secondly, the identification of dysregulated pathways in cancer is treated as a feature selection problem, which is biologically reasonable and easy to interpret. Thirdly, the dysregulated pathways are identified as subnetworks from the pathway interaction networks, where the subnetworks characterize very well the functional dependency or crosstalk between pathways. The benchmarking results on several distinct cancer datasets demonstrate that our method can obtain more reliable and accurate results compared with existing state of the art methods. Further functional analysis and independent literature evidence also confirm that our identified potential pathogenic pathways are biologically reasonable, indicating the effectiveness of our method. CONCLUSIONS: Dysregulated pathways can serve as better biomarkers compared with single genes. In this work, by utilizing pathway interaction networks and gene expression data, we propose a novel approach that effectively identifies dysregulated pathways, which can not only be used as biomarkers to diagnose cancers but also serve as potential drug targets in the future.

We consider dynamic spectrum access where distributed secondary users search for spectrum opportunities without knowing the primary traffic statistics. In each slot, a secondary transmitter chooses one channel to sense and subsequently transmit if the channel is sensed as idle. Sensing is imperfect, i.e., an idle channel may be sensed as busy and vice versa. Without centralized control, each secondary user needs to independently identify the channels that offer the most opportunities while avoiding collisions with both primary and other secondary users. We address the problem within a cooperative game framework, where the objective is to maximize the throughput of the secondary network under a constraint on the collision with the primary system. The performance of a decentralized channel access policy is measured by the system regret, defined as the expected total performance loss with respect to the optimal performance in the ideal scenario where the traffic load of the primary system on each channel is known to all secondary users and collisions among secondary users are eliminated through centralized scheduling. By exploring the rich communication structure of the problem, we show that the optimal system regret has the same logarithmic order as in the centralized counterpart with perfect sensing. A decentralized policy is constructed to achieve the logarithmic order of the system regret. In a broader context, this work addresses imperfect reward observation in decentralized multi-armed bandit problems.

We consider the restless multi-armed bandit (RMAB) problem with unknown dynamics. At each time, a player chooses K out of N (N >; K) arms to play The state of each arm determines the reward when the arm is played and transits according to Markovian rules no matter the arm is engaged or passive. The Markovian dynamics of the arms are unknown to the player. The objective is to maximize the long-term expected reward by designing an optimal arm selection policy. The performance of a policy is measured by regret, defined as the reward loss with respect to the case where the player knows which K arms are the most rewarding and always plays these K best arms. We construct a policy, referred to as Restless Upper Confidence Bound (RUCB), that achieves a regret with logarithmic order of time when an arbitrary nontrivial bound on certain system parameters is known. When no knowledge about the system is available, we extend the RUCB policy to achieve a regret arbitrarily close to the logarithmic order. In both cases, the system achieves the maximum average reward offered by the K best arms. Potential applications of these results include cognitive radio networks, opportunistic communications in unknown fading environments, and financial investment.

We consider an opportunistic channel access problem over multiple primary bands. We exploit primary ACK/NAK packets overhearing to overlay secondary communications on top of permanently busy primary channels. Through the theory of multi-armed restless bandit processes (MARB), we prove optimality of the myopic policy with a simple structure that does not require channel state-transition probabilities knowledge at the secondary users under certain conditions.

We consider a cognitive radio network with distributed multiple secondary users, where each user independently searches for spectrum opportunities in multiple channels without exchanging information with others. The occupancy of each channel is modeled as an i.i.d. Bernoulli process with unknown mean. Users choosing the same channel collide, and none or only one receives reward depending on the collision model. This problem can be formulated as a decentralized multi-armed bandit problem. We measure the performance of a decentralized policy by the system regret, defined as the total reward loss with respect to the optimal performance under the perfect scenario where all channel parameters are known to all users and collisions among secondary users are eliminated through perfect scheduling. We show that the minimum system regret grows with time at the same logarithmic order as in the centralized counterpart, where users exchange observations and make decisions jointly. We propose a basic policy structure that ensures a Time Division Fair Sharing (TDFS) of the channels. Based on this basic TDFS structure, decentralized policies can be constructed to achieve this optimal order while ensuring fairness among users. Furthermore, we show that the proposed TDFS policy belongs to a general class of decentralized polices, for which a uniform performance benchmark is established. All results hold for general stochastic processes beyond Bernoulli and thus find a wide area of potential applications including multi-channel communication systems, multi-agent systems, web search and advertising, and social networks.

In this paper, we consider a class of restless multiarmed bandit processes (RMABs) that arises in dynamic multichannel access, user/server scheduling, and optimal activation in multiagent systems. For this class of RMABs, we establish the indexability and obtain Whittle index in closed form for both discounted and average reward criteria. These results lead to a direct implementation of Whittle index policy with remarkably low complexity. When arms are stochastically identical, we show that Whittle index policy is optimal under certain conditions. Furthermore, it has a semiuniversal structure that obviates the need to know the Markov transition probabilities. The optimality and the semiuniversal structure result from the equivalence between Whittle index policy and the myopic policy established in this work. For nonidentical arms, we develop efficient algorithms for computing a performance upper bound given by Lagrangian relaxation. The tightness of the upper bound and the near-optimal performance of Whittle index policy are illustrated with simulation examples.

We aim to characterize the maximum link throughput of a multi-channel opportunistic communication system. The states of these channels evolve as independent and identically distributed Markov processes (the Gilbert-Elliot channel model). A user, with limited sensing and access capability, chooses one channel to sense and access in each slot and collects a reward determined by the state of the chosen channel. Such a problem arises in cognitive radio networks for spectrum overlay, opportunistic transmissions in fading environments, and resource-constrained jamming and anti-jamming. The objective of this paper is to characterize the optimal performance of such systems. The problem can be generally formulated as obtaining the maximum expected long-term reward of a partially observable Markov decision process or a restless multi-armed bandit process, for which analytical characterizations are rare. Exploiting the structure and optimality of the myopic channel selection policy established recently, we obtain a closed-form expression of the maximum link throughput for two-channel systems and lower and upper bounds when there are more than two channels. These results allow us to study the rate at which the optimal performance of an opportunistic system increases with the number of channels and to obtain the limiting performance as the number of channels approaches to infinity.

We consider a multi-channel opportunistic communication system where the states of these channels evolve as independent and statistically identical Markov chains (the Gilbert- Elliot channel model). A user chooses one channel to sense and access in each slot and collects a reward determined by the state of the chosen channel. The problem is to design a sensing policy for channel selection to maximize the average reward, which can be formulated as a multi-arm restless bandit process. In this paper, we study the structure, optimality, and performance of the myopic sensing policy. We show that the myopic sensing policy has a simple robust structure that reduces channel selection to a round-robin procedure and obviates the need for knowing the channel transition probabilities. The optimality of this simple policy is established for the two-channel case and conjectured for the general case based on numerical results. The performance of the myopic sensing policy is analyzed, which, based on the optimality of myopic sensing, characterizes the maximum throughput of a multi-channel opportunistic communication system and its scaling behavior with respect to the number of channels. These results apply to cognitive radio networks, opportunistic transmission in fading environments, downlink scheduling in centralized networks, and resource-constrained jamming and anti-jamming.

With the growing competition in the global marketplace, the call center has become an essential part for many companies. The outbound technology of call center plays a very important role in customer relationship management (CRM) and marketing. The call center of the large service companies usually has the characteristics of wide geographical distribution, high volume of calls, high efficiency and centralized data processing. With the establishment of IP network infrastructure in many large companies, a solution of a distributed IP-based predictive dialing system is discussed, which includes the architectural design, network topological structure and system functions. In the end, the benefit of developing a predictive dialing system using the virtual call center is discussed