Categorical relationships between objects are encoded as overlapped neural representations in the brain, where the more similar the objects are, the larger the correlations between their evoked neuronal responses. These representation correlations, however, inevitably incur interference when memories are retrieved. Here, we propose that neural feedback, which is widely observed in the brain but whose function remains largely unknown, contributes to disentangle neural correlations to improve information retrieval. We study a hierarchical neural network storing the hierarchical categorical information of objects, and information retrieval goes from rough-to-fine, aided by the push-pull neural feedback. We elucidate that the push and the pull components of the feedback suppress the interferences due to the representation correlations between objects from different and the same categories, respectively. Our model reproduces the push-pull phenomenon observed in neural data and sheds light on our understanding of the role of feedback in neural information processing.

Oftentimes the tasks that we regularly perform require our brain to integrate information from multiple sensory inputs. Experiments on human subjects have shown that the human brain combines multiple modalities in an optimal way that is predicted by the Bayesian inference. However, there have been few studies regarding whether deep neural networks can achieve this feat. Hence, we explore the capability of a modular multilayer neural network to integrate multiple sources of information. We designed a task in which two cameras, one left and one right, are monitoring a rotating chair. Our network comprises two modules, each processing a camera input, connected by an integrator layer. We find that the network successfully combines the information from the two cameras in a nearly Bayes-optimal manner, even when it is only trained with single-camera inputs. Further works can be done to study the capacity for integrating information from more than two inputs.

Experimental data has revealed that in addition to feedforward connections, there exist abundant feedback connections in a neural pathway. Although the importance of feedback in neural information processing has been widely recognized in the field, the detailed mechanism of how it works remains largely unknown. Here, we investigate the role of feedback in hierarchical information retrieval. Specifically, we consider a hierarchical network storing the hierarchical categorical information of objects, and information retrieval goes from rough to fine, aided by dynamical push-pull feedback from higher to lower layers. We elucidate that the push (positive) and pull (negative) feedbacks suppress the interferences due to neural correlations between different and the same categories, respectively, and their joint effect improves retrieval performance significantly. Our model agrees with the push-pull phenomenon observed in neural data and sheds light on our understanding of the role of feedback in neural information processing.

Experiments reveal that in the dorsal medial superior temporal (MSTd) and the ventral intraparietal (VIP) areas, where visual and vestibular cues are integrated to infer heading direction, there are two types of neurons with roughly the same number. One is “congruent” cells, whose preferred heading directions are similar in response to visual and vestibular cues; and the other is “opposite” cells, whose preferred heading directions are nearly “opposite” (with an offset of 180 degree) in response to visual vs. vestibular cues. Congruent neurons are known to be responsible for cue integration, but the computational role of opposite neurons remains largely unknown. Here, we propose that opposite neurons may serve to encode the disparity information between cues necessary for multisensory segregation. We build a computational model composed of two reciprocally coupled modules, MSTd and VIP, and each module consists of groups of congruent and opposite neurons. In the model, congruent neurons in two modules are reciprocally connected with each other in the congruent manner, whereas opposite neurons are reciprocally connected in the opposite manner. Mimicking the experimental protocol, our model reproduces the characteristics of congruent and opposite neurons, and demonstrates that in each module, the sisters of congruent and opposite neurons can jointly achieve optimal multisensory information integration and segregation. This study sheds light on our understanding of how the brain implements optimal multisensory integration and segregation concurrently in a distributed manner.

Time delay is pervasive in neural information processing. To achieve real-time tracking, it is critical to compensate the transmission and processing delays in a neural system. In the present study we show that dynamical synapses with short-term depression can enhance the mobility of a continuous attractor network to the extent that the system tracks time-varying stimuli in a timely manner. The state of the network can either track the instantaneous position of a moving stimulus perfectly (with zero-lag) or lead it with an effectively constant time, in agreement with experiments on the head-direction systems in rodents. The parameter regions for delayed, perfect and anticipative tracking correspond to network states that are static, ready-to-move and spontaneously moving, respectively, demonstrating the strong correlation between tracking performance and the intrinsic dynamics of the network. We also find that when the speed of the stimulus coincides with the natural speed of the network state, the delay becomes effectively independent of the stimulus amplitude.

Experimental data have revealed that neuronal connection efficacy exhibits two forms of short-term plasticity: short-term depression (STD) and short-term facilitation (STF). They have time constants residing between fast neural signaling and rapid learning and may serve as substrates for neural systems manipulating temporal information on relevant timescales. This study investigates the impact of STD and STF on the dynamics of continuous attractor neural networks and their potential roles in neural information processing. We find that STD endows the network with slow-decaying plateau behaviors: the network that is initially being stimulated to an active state decays to a silent state very slowly on the timescale of STD rather than on that of neuralsignaling. This provides a mechanism for neural systems to hold sensory memory easily and shut off persistent activities gracefully. With STF, we find that the network can hold a memory trace of external inputs in the facilitated neuronal interactions, which provides a way to stabilize the network response to noisy inputs, leading to improved accuracy in population decoding. Furthermore, we find that STD increases the mobility of the network states. The increased mobility enhances the tracking performance of the network in response to time-varying stimuli, leading to anticipative neural responses. In general, we find that STD and STP tend to have opposite effects on network dynamics and complementary computational advantages, suggesting that the brain may employ a strategy of weighting them differentially depending on the computational purpose.

Neuronal connection weights exhibit short-term depression (STD). The present study investigates the impact of STD on the dynamics of a continuous attractor neural network (CANN) and its potential roles in neural information processing. We find that the network with STD can generate both static and traveling bumps, and STD enhances the performance of the network in tracking external inputs. In particular, we find that STD endows the network with slow-decaying plateau behaviors, namely, the network being initially stimulated to an active state will decay to silence very slowly in the time scale of STD rather than that of neural signaling. We argue that this provides a mechanism for neural systems to hold short-term memory easily and shut off persistent activities naturally.

Understanding how the dynamics of a neural network is shaped by the network structure and, consequently, how the network structure facilitates the functions implemented by the neural system is at the core of using mathematical models to elucidate brain functions. This study investigates the tracking dynamics of continuous attractor neural networks (CANNs). Due to the translational invariance of neuronal recurrent interactions, CANNs can hold a continuous family of stationary states. They form a continuous manifold in which the neural system is neutrally stable. We systematically explore how this property facilitates the tracking performance of a CANN, which is believed to have clear correspondence with brain functions. By using the wave functions of the quantum harmonic oscillator as the basis, we demonstrate how the dynamics of a CANN is decomposed into different motion modes, corresponding to distortions in the amplitude, position, width, or skewness of the network state. We then develop a perturbation approach that utilizes the dominating movement of the network's stationary states in the state space. This method allows us to approximate the network dynamics up to an arbitrary accuracy depending on the order of perturbation used. We quantify the distortions of a gaussian bump during tracking and study their effects on tracking performance. Results are obtained on the maximum speed for a moving stimulus to be trackable and the reaction time for the network to catch up with an abrupt change in the stimulus.

We study a model of transportation networks with nonlinear elements which represent local shortage of resources. Frustration arises from competition among the nodes to become satisfied. When the initial resources are uniform, algorithmically hard regimes emerge when the average availability of resources increases. These regimes are characterized by discrete fractions of satisfied nodes, resembling the Devil's staircase. Behavior similar to those in the vertex cover or close packing problems are found. When initial resources are bimodally distributed, such algorithmically hard regimes also emerge when the fraction of rich nodes increases.

Continuous attractor neural networks (CANNs) are emerging as promising models for describing the encoding of continuous stimuli in neural systems. Due to the translational invariance of their neuronal interactions, CANNs can hold a continuous family of neutrally stable states. In this study, we systematically explore how neutral stability of a CANN facilitates its tracking performance, a capacity believed to have wide applications in brain functions. We develop a perturbative approach that utilizes the dominant movement of the network stationary states in the state space. We quantify the distortions of the bump shape during tracking, and study their effects on the tracking performance. Results are obtained on the maximum speed for a moving stimulus to be trackable, and the reaction time to catch up an abrupt change in stimulus.

This paper addresses the issue of call admission control with the necessary quality-of-service (QoS) guarantee for an integrated service mobile cellular network. Specifically, we extend the limited fractional guarded channel (LFGC) scheme to incorporate the multiple traffic types, and derive the set of parameters that leads to the optimal call admission control. The key challenges are the state-dependent nature of the admission control, and the increasing complexity of the state space in a multi-services environment. We propose a novel control mechanism that only uses a few parameters (2-5) to characterize the optimal control planes, and use the simulated annealing technique to obtain the optimal control parameters.

The problem of resource allocation in sparse graphs with real variables is studied using methods of statistical physics. An efficient distributed algorithm is devised on the basis of insight gained from the analysis and is examined using numerical simulations, showing excellent performance and full agreement with the theoretical results.

This paper investigates the problem of call admission control in multi-service wireless cellular networks. Prior studies on call admission control have largely adopted a static approach based on a variation of guard channel policy. Given the highly dynamic traffic behavior in multi-service wireless cellular networks, we believe that a dynamic approach is essential in providing the required service guarantee and the necessary service differentiation for different types of traffic. In this paper, we focus on the connection level performance, and introduce a stable dynamic call admission (SDCA) policy that can effectively deal with the mixed type of traffic. The main objectives of the proposed SDCA are: 1) it has to enforce the QoS guarantee on the dropping probabilities of all traffic; 2) it needs to provide fairness among different class of traffic; and 3) it should maintain stability over a broad range of traffic rates. Its performance is well supported by simulation results.

We consider multi-agent systems whose agents compete for resources by striving to be in the minority group. The agents adapt to the environment by reinforcement learning of the preferences of the policies they hold. Diversity of preferences of policies is introduced by adding random bi- ases to the initial cumulative payoffs of their policies. We explain and provide evidence that agent cooperation becomes increasingly important when diversity increases. Analyses of these mechanisms yield excellent agreement with simulations over nine decades of data.

Abstract We present a preliminary empirical test of a maximum likelihood approach to using relevance data for training information retrieval (IR) parameters. Similar to language models, our method uses explicitly hypothesized distributions for documents and queries, but we add to this an explicitly hypothesized distribution for relevance judgments. The method unifies document‐oriented and query‐oriented views. Performance is better than the Rocchio heuristic for document and/or query modification. The maximum likelihood methodology also motivates a heuristic estimate of the MLE optimization. The method can be used to test competing hypotheses regarding the processes of authors' term selection, searchers' term selection, and assessors' relevancy judgments.

This paper addresses the issue of call admission control with the necessary QoS guarantee for a voice/data integrated wireless cellular network. Specifically, we propose a new call admission control policy called dual-threshold call admission control (DT-CAC) policy. It uses two thresholds, one to reserve channels for narrow-band voice handoff calls, and the other for wideband data handoff traffic. Its unique features are: (1) it adopts the complete sharing (CS) approach that maximizes the channel efficiency; (2) it derives the optimal set of thresholds that satisfies the QoS target on handoff dropping probabilities for both voice and data traffic, while at the same time minimizing the maximal new call blocking probability (voice or data). Its performance is well supported by simulation results.

We study an explicit parametric model of documents, queries, and rel- evancy assessment for Information Retrieval (IR). Mean-field methods are applied to analyze the model and derive efficient practical algorithms to estimate the parameters in the problem. The hyperparameters are es- timated by a fast approximate leave-one-out cross-validation procedure based on the cavity method. The algorithm is further evaluated on several benchmark databases by comparing with standard algorithms in IR.

Call admission control is one of the key elements in ensuring the quality of service in mobile wireless networks. The traditional trunk reservation policy and its numerous variants give preferential treatment to the handoff calls over new arrivals by reserving a number of radio channels exclusively for handoffs. Such schemes, however, cannot adapt to changes in traffic pattern due to the static nature. This paper introduces a novel stable dynamic call admission control mechanism (SDCA), which can maximize the radio channel utilization subject to a predetermined bound on the call dropping probability. The novelties of the proposed mechanism are: (1) it is adaptive to wide range of system parameters and traffic conditions due to its dynamic nature; (2) the control is stable under overloading traffic conditions, thus can effectively deal with sudden traffic surges; (3) the admission policy is stochastic, thus spreading new arrivals evenly over a control period, and resulting in more effective and accurate control; and (4) the model takes into account the effects of limited channel capacity and time dependence on the call dropping probability, and the influences from nearest and next-nearest neighboring cells, which greatly improve the control precision. In addition, we introduce local control algorithms based on strictly local estimations of the needed traffic parameters, without requiring the status information exchange among different cells, which makes it very appealing in actual implementation. Most of the computational complexities lie in off-line precalculations, except for the nonlinear equation of the acceptance ratio, in which a coarse-grain numerical integration is shown to be sufficient for stochastic control. Extensive simulation results show that our scheme steadily satisfies the hard constraint on call dropping probability while maintaining a high channel throughput.

We propose a method for the fast estimation of hyperparameters in large networks, based on the linear response relation in the cav(cid:173) ity method, and an empirical measurement of the Green's func(cid:173) tion. Simulation results show that it is efficient and precise, when compared with cross-validation and other techniques which require matrix inversion.

We introduce stagewise processing in error-correcting codes and image restoration, by extracting information from the former stage and using it selectively to improve the performance of the latter one. Both mean-field analysis using the cavity method and sim(cid:173) ulations show that it has the advantage of being robust against uncertainties in hyperparameter estimation.

An important issue in neural computing concerns the description of learning dynamics with macroscopic dynamical variables. Recent progress on online learning only addresses the often unrealistic case of an infinite training set. We introduce a new framework to model batch learning of restricted sets of examples, widely applicable to any learning cost function, and fully taking into account the temporal correlations introduced by the re-cycling of the examples. Here we illustrate the technique using the Adaline rule learning random of teacher-generated examples.

Call admission control is one of the key elements in ensuring the QoS in mobile wireless networks supporting multimedia applications. This paper introduces a new distributed and dynamic call admission control mechanism (SDCA), whose objective is to maximize the radio channel utilization subject to a pre-determined threshold of the call dropping probability. The novelties of the proposed mechanism are that we have taken into account the effects of limited channel capacity and time dependence on the call dropping probability, and the influences from nearest and next-nearest neighboring cells. New calls are spread evenly over a control period, which leads to more effective and stable control. Simulations show that our scheme steadily satisfies the hard constraint on call dropping probability while maintaining a high channel throughput when compared with previous proposals.

Overload control of call processors in telecom networks is used to protect the network of call processing computers from excessive load during traffic peaks, and involves techniques of predictive control with limited local information. Here we propose a neural-network algorithm, in which a group of neural controllers are trained using examples generated by a globally optimal control method. Simulations show that the neural controllers have better performance than local control algorithms in both the throughput and the response to traffic upsurges. Compared with the centralized control algorithm, the neural control significantly decreases the computational time for making decisions and can be implemented in real time.