In discrete systems, Shannon entropy is well known to be characterized by the Shannon-Khinchin axioms. Recently, this set of axioms was generalized for Tsallis entropy, one-parameter generalization of Shannon entropy. In continuous systems, Shannon differential entropy has been introduced as a natural extension of the above Shannon entropy without using an axiomatic approach. We derive the generalized entropy function as a solution of the functional equation determined by the generalized Shannon additivity, one of the most important axiom of the generalized Shannon-Khinchin axioms for Tsallis entropy. This generalized entropy function naturally introduces Tsallis differential entropy and two Tsallis divergences. In particular, one (Csisza¿r type) of the divergences has almost the same form as the ¿-divergence in information geometry and the other the Bregman type divergence. Our results reveal that the generalized Shannon additivity representing a branch structure of a rooted tree plays an essential role in the determination of these entropies.

We prove that the generalized Shannon additivity determines a lower bound of average description length for the q-generalized Z3-ary code tree. To clarify our main result, at first it is shown that the original Shannon additivity determines a lower bound of average code length of a Z3-ary code tree. As its generalization, we present our main result mentioned above. The generalized Shannon additivity is one of the generalized Shannon-Khinchin axioms for Tsallis entropy, i.e. one-parameter generalization of Shannon entropy. This reveals that Tsallis entropy is a lower bound of average description length for the q-generalized Z3-ary code tree.

For a unified description of power-law behaviors such as chaos, fractal and scale-free network, Tsallis entropy has been applied to the generalization of the traditional Boltzmann-Gibbs statistics as a fundamental information measure. Tsallis entropy Sqis an one-parameter generalization of Shannon entropy S1in the sense that limqrarr1Sq= Si. The generalized Boltzmann-Gibbs statistics by means of Tsallis entropy is nowadays called Tsallis statistics. The main approach in Tsallis statistics has been the maximum entropy principle, but there have been missing some fundamental mathematical formulae such as law of error, q-Stirling's formula and q-multinomial coefficient. Recently, we have succeeded in proving law of error in Tsallis statistics using the q-product uniquely determined by Tsallis entropy. Along the same lines as the proof, we present q-Stirling's formula, q-multinomial coefficient and a conjecture on the q-central limit theorem in Tsallis statistics

In order to theoretically explain the ubiquitous existence of power-law behavior such as chaos and fractals in nature, Tsallis entropy has been successfully applied to the generalization of the traditional Boltzmann-Gibbs statistics, the fundamental information measure of which is Shannon entropy. Tsallis entropy S/sub q/ is a one-parameter generalization of Shannon entropy S/sub 1/ in the sense that lim/sub q/spl rarr/1/S/sub q/=S/sub 1/. The generalized statistics using Tsallis entropy are referred to as Tsallis statistics. In order to present the law of error in Tsallis statistics as a generalization of Gauss' law of error and prove it mathematically, we apply the new multiplication operation determined by q-logarithm and q-exponential, the fundamental functions in Tsallis statistics, to the definition of the likelihood function in Gauss' law of error. The present maximum-likelihood principle (MLP) leads us to determine the so-called q-Gaussian distribution, which coincides with one of the Tsallis distributions derived from the maximum entropy principle for Tsallis entropy under the second moment constraint.

Tsallis entropy, one-parameter generalization of Shannon entropy, has been often discussed in statistical physics as a new information measure. This new information measure has provided many satisfactory physical interpretations in nonextensive systems exhibiting chaos or fractal. We present the generalized Shannon-Khinchin axioms to nonextensive systems and prove the uniqueness theorem rigorously. Our results show that Tsallis entropy is the simplest among all nonextensive entropies. By the detailed comparisons of our axioms with the previously presented two sets of axioms, we reveal the peculiarity of pseudoadditivity as an axiom. In this correspondence, the most fundamental basis for Tsallis entropy as information measure is established in the information-theoretic framework.

It is known that electroencephalogram (EEG) is broadly scale-invariant over several orders of frequencies. Applying the renormalization group method to the neural network model, we obtain the self-similar solution from which the scaling exponent -1 of the spectrum fall-off. This paper permits one to study the self-similar behavior and helps in understanding the basic ingredients underlying 1/f spectra of EEG.

This paper presents the new derivation of the storage capacity of neural networks with binary weights w/sub i//spl isin/{0,1},{-1,+1}. Our approaches are based on introducing a new parameter "d" (minimum distance between input patterns), not a usual parameter "p" (number of input patterns). Taking a new parameter "d" to characterize the input patterns, some results on the information theory can be applied to the computation of the storage capacity of neural networks with binary weights. This approach succeed to obtain almost the same storage capacities as those by the replica method in statistical physics.