We present a path starting from generalized information measures to blockchain applications. In the middle of these two endpoints, we derive the alpha-EM algorithm and the traditional log-EM algorithm simultaneously from a sole divergence. Thus, there are three subjects. The first part discusses the relationship between the Bregman divergence and the probabilistic alpha-divergence. There, a new alpha-divergence is derived from the Bregman divergence by skewing the probability space. In the second part, both the alpha-EM algorithm and the traditional log-EM algorithm are derived simultaneously from this single divergence. The third part is on the applications to the blockchain. The EM families are effective in statistical machine learning that can be incorporated in miners of blockchains for evaluating the data sent from clients. Any miner appreciates the speed in order to contest with other miners. The alpha-EM matches this purpose because of its faster speed than the log-EM. As a concrete application, we present the wine sommelier evaluation. This formulation using the multinomial mixture distribution can be applicable to a wider class of IoT data that fly on the network.

Brain waves, or electroencephalograms (EEGs), are applicable to user verification. We devise a two-factor system so that impersonators who hold identification numbers in fraudulence are detectable. In the first step, a subject either authentic or false tries to input a digit of a ten-key in the personal identification number by a P300 speller. The P300 speller is a brain-computer interface that detects positive voltage jump when a subject identifies specific digits on a display visually. By considering the performance of the P300 speller, we allow an error of one digit out of the four digits. On the other hand, we keep suspicion even for the case of perfect four digits because of the possibility of impersonation by a stolen case. Following the P300 spelling, we apply a verification of subjects by brain waves. Averaging of detected P300 waveforms after band-pass filtering takes the role of feature extraction. Then, a support vector machine applied to the averaged waveforms decides whether the subject is authentic or false. Thus, the total system does not entail the complexity of multimodality. For this system, we measured average error rates for 20 subjects. Experiments showed the false rejection rate of 3.9% at the false acceptance rate of 0% for the 4-digit number case. These pair values are successfully low even by using brain waves that usually contain many artifacts. Additionally, experiments on a diabetes patient before and after an insulin injection are also conducted. The result shows that the appropriate injection control maintains no difference from ordinary subjects. In concluding remarks, we consider methods to increase subjects and digits for applications in a larger society.

Video data captured and uploaded under volatile conditions is accumulating into a flood of unstructured content that is hard to manage. In this paper, we present a set of algorithms that generate numeric or soft labels to structure automatically and produce video similarity rankings. Exemplar frames are extracted from each video during the labeling process. We propose five types of learning algorithms based on the following three classes: affinity propagation, k-means or harmonic competition, and pairwise nearest-neighbor. For redundancy reduction, we also provide an algorithm that prunes excessive exemplars. The five learning algorithms produced creditable order-aware exemplar sets for the target videos. Because the content and lengths of the videos differ in terms of temporal order, we provide new methods to analyze the similarities between exemplar sets. The m-distance similarity measure is the core concept used for the global and local alignments performed on the obtained exemplar sets. Based on this comparison mechanism, we identified high-precision recall curves for all five methods. In terms of learning speed, the k-means and pairwise nearest-neighbor classes are recommendable. To facilitate similar-video retrieval, we developed a graphical user interface that accepts videos downloaded from the Web. By replacing a procedure in the software, the proposed similar-video retrieval system can accommodate more elaborate frame comparison features.

Fast estimation algorithms for Hidden Markov models (HMMs) for given data are presented. These algorithms start from the alpha-EM algorithm which includes the traditional log-EM as its proper subset. Since existing or traditional HMMs are the outcome of the log-EM, it had been expected that the alpha-HMM would exist. In this paper, it is shown that this foresight is true by using methods of the iteration index shift and likelihood ratio expansion. In each iteration, new update equations utilize one-step past terms which are computed and stored during the previous maximization step. Therefore, iteration speedup directly appears as that of CPU time. Since the new method is theoretically based on the alpha-EM, all of its properties are inherited. There are eight types of alpha-HMMs derived. They are discrete, continuous, semi-continuous and discrete-continuous alpha-HMMs, and both for single and multiple sequences. Using the properties of the alpha-EM algorithm, the speedup property is theoretically analyzed. Experimental results including real world data are given.

New algorithms for joint learning of independent component analysis and graphical high-order correlation (GC-ICA: Graphically Correlated ICA) are presented. The presented method has a fixed point style or of the FastICA, however, it comprises independent but correlated subparts. Correlations by teacher signals are also allowed. In spite of such inclusion of the dependency, the presented algorithm shows fast convergence. The converged set of bases has reduced indeterminacy on the ordering. This is equivalent to a self-organization of bases. This method can be used to analyze multiple images simultaneously. Examples are given on images from 3D- stereo videos shots. The correlation of bases on left and right eye views is shown for the first time here. Further speedup using the strategy of the RapidICA is possible.

Heterogeneous bio-signals including human motions, brain NIRS and neural spike trains are utilized for operating biped humanoids. The Bayesian network comprising Hidden Markov Models and Support Vector Machines is designed for the signal integration. By this method, the system complexity is reduced so that that total operation is within the scope of PCs. The designed system is capable of transducing original sensory meaning to another. This leads to prosthesis, rehabilitation and gaming. In addition to the supervised mode, the humanoid can act autonomously for its own designed tasks.

A fast learning algorithm for Hidden Markov Models is derived starting from convex divergence optimization. This method utilizes the alpha-logarithm as a surrogate function for the traditional logarithm to process the likelihood ratio. This enables the utilization of a stronger curvature than the logarithm. This paper's method includes the ordinary Baum-Welch re-estimation algorithm as a proper subset. The presented algorithm shows fast learning by utilizing time-shifted information during the progress of iterations. The computational complexity of this algorithm, which directly affects the CPU time, remains almost the same as the logarithmic one since only stored results are utilized for the speedup. Software implementation and speed are examined in the test data. The results showed that the presented method is creditable.

A faster algorithm of ICA is devised. This method utilizes multi-step past information with the existing fixed-point method. The use of past information comes from the idea of surrogate optimization. The speed and implemented software are checked for both simulated and read data. The presented ICA, named Rapid ICA, is applied to image-to-image retrieval without indices. Images judged to be similar reflected user's opinion well.

New methods for joint compression and Image-to-image retrieval (12I retrieval) are presented. The novelty exists in the usage of computationally learned image bases besides color distributions. The bases are obtained by the Principal Component Analysis and/or the Independent Component Analysis. On the image compression, PCA and ICA outperform the JPEG's DCT This superiority holds even if the bases and superposition coefficients are quantized and encoded. On the 12I retrieval, the precision-recall curve is used to measure the performance. It is found that adding the basis information always increases the baseline ability of the color information. Besides the retrieval evaluation, a unified image format called RIM (Retrieval-aware IMage format) for effective packing of codewords including bases is specified. Furthermore, an image search viewer called Wisvi (Waseda Image Search Viewer) is developed and exploited. A beta-version of all source codes can be down-loaded from a web site given in the text.

Similar-image retrieval systems are presented and evaluated. The new systems directly use image bases via ICA (independent component analysis) and PCA (principal component analysis). These bases can extract source image's information which is viable to define similarity measures. But, the indeterminacy on amplitude and permutation exists. In this paper, similarity measures which can absorb such indeterminacy are presented. Then, carefully designed opinion tests are carried out to compare the new systems' ability with existing ones. The compatibility of color spaces such as RGB, YIQ, and HSV is also examined. By these massive tests, {ICA, HSV} is judged the best. The resulting system is thus proved to be highly competent at the similar-image retrieval.

Independent component analysis (ICA) is applied to image coding. There, new design methods for ICA bases are presented. The new feature of this learning algorithm includes the weak guidance, or decreasing supervisory information. The weak guidance reduces the permutation indeterminacy which is unavoidable in usual ICA algorithms. In view of the image compression, this effect corresponds to the generation of image bases honoring the space frequency's neighborhood and 2-D ordering. Following the presentation of this learning algorithm, experiments are performed to obtain serviceable ICA bases. Finally, image compression and restoration are demonstrated to show the eligibility for "image. IPEG." Other applications such as image retrieval are also commented upon.

A new likelihood maximization algorithm called the /spl alpha/-EM algorithm (/spl alpha/-expectation-maximization algorithm) is presented. This algorithm outperforms the traditional or logarithmic EM algorithm in terms of convergence speed for an appropriate range of the design parameter /spl alpha/. The log-EM algorithm is a special case corresponding to /spl alpha/=-1. The main idea behind the /spl alpha/-EM algorithm is to search for an effective surrogate function or a minorizer for the maximization of the observed data's likelihood ratio. The surrogate function adopted in this paper is based upon the /spl alpha/-logarithm which is related to the convex divergence. The convergence speed of the /spl alpha/-EM algorithm is theoretically analyzed through /spl alpha/-dependent update matrices and illustrated by numerical simulations. Finally, general guidelines for using the /spl alpha/-logarithmic methods are given. The choice of alternative surrogate functions is also discussed.

The /spl alpha/-EM algorithm is a super-class of the traditional expectation-maximization (EM) algorithm. This algorithm is derived by computing the likelihood ratio of incomplete data through an extended logarithm; namely, the /spl alpha/-logarithm. The case of /spl alpha/=-1 corresponds to the logarithm. The number /spl alpha/ adjusts eigenvalues of update matrices by reflecting the optimization function's second-order properties with respect to the estimation parameter. This property shows merits on speedup of convergence. In the paper, a derivation of the algorithm is given first. Then, convergence and speedup properties are discussed. Finally, the applicability of the /spl alpha/-FM algorithm and examples are shown.

The /spl alpha/-logarithm extends the logarithm as the special case of /spl alpha/=-1. Usage of /spl alpha/-related information measures based upon this extended logarithm is expected to be effective to speedup of convergence, i.e., on the improvement of learning aptitude. In this paper, two typical cases are investigated. One is the /spl alpha/-EM algorithm (/spl alpha/-expectation-maximization algorithm) which is derived from the /spl alpha/-log-likelihood ratio. The other is the /spl alpha/-ICA (/spl alpha/-independent component analysis) which is formulated as minimizing the /spl alpha/-mutual information. In the derivation of both algorithms, the /spl alpha/-divergence plays the main role. For the /spl alpha/-EM algorithm, the reason for the speedup is explained using Hessian and Jacobian matrices for learning. For the /spl alpha/-ICA learning, methods of exploiting the past and future information are presented. Examples are shown on single-loop /spl alpha/-EM and sample-based /spl alpha/-ICA. In all cases, effective speedups are observed. Thus, this paper's examples together with formerly reported ones are evidences that the speed improvement by the /spl alpha/-logarithm is a general property beyond individual problems.

The /spl alpha/-EM algorithm is a proper extension of the traditional log-EM algorithm. This new algorithm is based on the /spl alpha/-logarithm, while the traditional one uses the logarithm. The case of /spl alpha/=-1 corresponds to the log-EM algorithm. Since the speed of the /spl alpha/-EM algorithm was reported for learning problems, this paper shows that closed-form E-steps can be obtained for a wide class of problems. There is a set of common techniques. That is, a cookbooks for the /spl alpha/-EM algorithm is presented. The recipes include unsupervised neural networks, supervised neural networks for various gating, hidden Markov models and Markov random fields for moving object segmentation. Reasoning for the speedup is also given.

Multiple descent cost competition is a composition of learning phases for minimizing a given measure of total performance, i.e., cost. If these phases are heterogeneous toward each other, the total learning algorithm shows a variety of extraordinary abilities; especially in regards to multimedia information processing. In the first phase of descent cost learning, elements of source data are grouped. Simultaneously, a weight vector for minimal learning, (i.e., a winner), is found. Then, the winner and its partners are updated for further cost reduction. Therefore, two classes of self-organizing feature maps are generated. One is called a grouping feature map, which partitions the source data. The other is an ordinary weight vector feature map. The grouping feature map, together with the winners, retains most of the source data information. This feature map is able to assist in a high quality approximation of the original data. Traditional weight vector feature maps lack this ability. Another important capacity of the grouping feature map is that it can change its shape. Thus, the grouping pattern can accept external directions in order to metamorphose. In the text, the total algorithm of the multiple descent cost competition is explained first. In that section, image processing concepts are introduced in order to assist in the description of this algorithm. Then, a still image is first data-compressed (DC). Next, a restored image is morphed using the grouping feature map by receiving directions given by an external intelligence. Next, an interpolation of frames is applied in order to complete animation coding (AC). Thus, multiple descent cost competition bridges "DC to AC." Examples of multimedia processing on virtual digital movies are given.

Harmonic competition is a learning strategy based upon winner-take-all or winner-take-quota with respect to a composite of heterogeneous subcosts. This learning is unsupervised and organizes itself. The subcosts may conflict with each other. Thus, the total learning system realizes a self-organizing multiple criteria optimization. The subcosts are combined additively and multiplicatively using adjusting parameters. For such a total cost, a general successive learning algorithm is derived first. Then, specific problems in the Euclidian space are addressed. Vector quantization with various constraints and traveling salesperson problems are selected as test problems. The former is a typical class of problems where the number of neurons is less than that of the data. The latter is an opposite case. Duality exists in these two classes. In both cases, the combination parameters of the subcosts show wide dynamic ranges in the course of learning. It is possible, however, to decide the parameter control from the structure of the total cost. This method finds a preferred solution from the Pareto optimal set of the multiple object optimization. Controlled mutations motivated by genetic algorithms are proved to be effective in finding near-optimal solutions.

Novel general algorithms for multiple-descent cost-competitive learning are presented. These algorithms self-organize neural networks and possess the following features: optimal grouping of applied vector inputs, product form of neurons, neural topologies, excitatory and inhibitory connections, fair competitive bias, oblivion, winner-take-quota rule, stochastic update, and applicability to a wide class of costs. Both batch and successive training algorithms are given. Each type has its own merits. However, these two classes are equivalent, since a problem solved in the batch mode can be computed successively, and vice versa. The algorithms cover a class of combinatorial optimizations besides traditional standard pattern set design

The author discusses algorithms of competitive self-organization and their application to a typical combinatorial problem, the traveling salesman problem. The main feature of the proposed algorithm is the sophisticated use of excitatory/inhibitory intralayer connections of neurons combined with a judicious selection of neural network topology. Such properties contribute to obtaining excellent approximate solutions. Five hundred sets of 30-city solutions are compared with those obtained by a pure simulated annealing method. From this comparison, it is found that a considerable number of the solutions obtained by this self-organization method are highly likely to be the optimal tours. Successive training algorithms are mainly used; however, applications of batch training algorithms are also discussed. Implications for multisalesman problems are discussed

Algorithms for vector quantization of variable region data are given. The design iteration is proved to converge. An important issue here is the optimization step of the region shape with respect to the vector quantization codebook. Thus, the presented design method is a nontrivial extention of ordinary vector quantizer design which contains the classical Lloyd-Max algorithm. First, the main algorithm is given without introducing any physical entity. Therefore, the method is applicable to any data including speech and image as long as the quantization distortion is defined. In the speech coding case, which is the main body of this paper, the region shape optimization is interpreted as the epoch interval adjustment. The selection of the adjusted epochs with respect to the vector quantization codebook considerably reduces the quantizing distortion. This enables very-low-rate speech compression. Then, the image coding case is formulated and some convergence problem is discussed.

Joint time-spectral vector quantization to achieve much lower bit rates on the average than ordinary spectral vector quantizers is discussed. The principle and used quantities are closed within the theory of LPC so that theoretical clearness and the circumvention of unnecessary system complexity are promoted. The presented theory and method lead to an architecture for a pipelined inverse filter array for VLSI.

Two speech compression systems based on codebooks of inverse filters produced by off-line linear predictive coding (LPC) and vector quantization (VQ) techniques are considered. The first system is a pitch excited vocoder that is a variation on a speech coding system based upon vector quantization. The encoder selects an LPC reverse filter from a finite codebook that best "matches" an observed frame of sampled speech. This filter is in turn used to determine the voicing and digitized pitch information. Unlike LPC systems, the digitization is performed in a single step on the data rather than separate modeling and digitization steps. The second system is a tree encoding system that uses the filter selected by an inverse filter matching vocoder to "color" a tree that is then searched for a minimum distortion path for the original sampled speech waveform. This system can be viewed as a hybrid between an adaptive predictive coder and a universal tree encoder. The two systems are described, simulated, and compared with other similar systems.

A low-rate (about one bit per sample) waveform coder for speech compression is designed using techniques from universal source coding, fake process tree encoding, and linear predictive coding (LPC). The system does not require on-line adaptation or LPC analysis, yet it yields a fidelity that compares well with the best existing adaptive-waveform coder of the same rate.