Underwater images are actively utilized for ocean observation and various applications related to marine science. They, however, suffer from image quality degradation due to underwater imaging which impedes in-depth image analysis. While deep models are effectively applied to correct degraded images, underwater image enhancement (UIE) specifically poses two issues regarding severe color distortion and scarce training data. To cope with those two issues, we propose a novel framework of multi-channel enhanced Glow, dubbed as MC-Glow, which effectively harnesses a deep model to enhance underwater images. Our deep model built upon invertible Glow structure is equipped with a multi-channel processing for resolving severe color degradation. In contrast to standard processing of full colors, the multi-channel branch encodes RGB colors separately to suppress interference among distorted color channels. Besides, we explore the possibility of leveraging the white balance dataset for pre-training the deep model. The pre-training dataset is composed of easy-to-access photos unlike underwater images and endows the model with effective image enhancement by mitigating scarcity of underwater images. Experimental results on UIE tasks using two benchmark datasets demonstrate that the proposed MC-Glow produces competitive performance with the other UIE approaches. Code is available at https://github.com/tkswalk/2025_Signal-Processing/tree/main .

A non-negative matrix factorization (NMF) is effectively applied to analyze data in an unsupervised way. Though non-negative factors are endowed with favorable interpretability, such as part-based representation, NMF lacks flexibility, being only applicable to data composed of non-negative values. In this paper, we propose a novel approach to enhance both flexibility and interpretability of matrix factorization. While NMF approximates a matrix in an additive form of non-negative factors, the proposed method disentangles the matrix into additive and subtractive parts by exploiting non-negative factors with a center factor akin to the average of data. Thereby, the disentanglement flexibly deals with any real-valued data beyond non-negative ones while rendering clear functionality to the factors. In the experiments on several factorization tasks using real-world data, the proposed method provides effective factorization to embed well interpretability especially into factors for analyzing intrinsic characteristics of the data.

Few-shot learning (FSL) is a challenging task in machine learning, demanding a model to render discriminative classification by using only a few labeled samples. In the literature of FSL, deep models are trained in a manner of metric learning to provide metric in a feature space which is well generalizable to classify samples of novel classes; in the space, even a few amount of labeled training examples can construct a decent classifier. In this paper, we propose a novel FSL loss based on geometric mean to embed effective metric into deep features. In contrast to the other losses such as utilizing arithmetic mean in softmax-based formulation, the proposed method leverages geometric mean to aggregate pair-wise relationships among samples for enhancing discriminative metric across class categories. The proposed loss is not only formulated in a simple form but also is thoroughly analyzed in theoretical ways to reveal its favorable characteristics which are favorable for learning feature metric in FSL. In the experiments on few-shot image classification tasks, the method produces competitive performance in comparison to the other losses.

Transformer models developed in NLP make a great impact on computer vision fields, producing promising performance on various tasks. While multi-head attention, a characteristic mechanism of the transformer, attracts keen research interest such as for reducing computation cost, we analyze the transformer model from a viewpoint of feature transformation based on a distribution of input feature to-kens. The analysis inspires us to derive a novel transformation method from mean-shift update which is an effective gradient ascent to seek a local mode of distinctive repre-sentation on the token distribution. We also present an efficient projection approach to reduce parameter size of linear projections constituting the proposed multi-head feature transformation. In the experiments on ImageNet-lK dataset, the proposed methods embedded into various network models exhibit favorable performance improvement in place of the transformer module. Codes are available at https://github.com/tk1980/MSFtransformer.

This paper proposes a time-series data embedding technique that preserves curvature and orientation, with a focus on visualizing temporal manifold-valued data. Manifold-valued data provide pair-wise local distances on which the proposed method is built. First, we introduce a simpler form of our method, conformal folding embedding (CFE), as an interpretable straightforward algorithm to perfectly preserve the angles between adjacent velocity vectors, maintaining local geometric structure while forgoing global structure. Then we introduce the general formulation, dubbed local distance correlation embedding (LDCE), that maximizes distance correlation between input manifoldvalued data and the embedded ones while preserving both global distance structure and local geometric structure. Although the two algorithms are different in formulation, we show their theoretical connection by proving that CFE is a special case of LDCE. We empirically showcase the effectiveness of LDCE in preserving curvature/orientation by visualizing simulated data. The method is also applied to analyze the temporal information encoded at a population level in the inferior temporal cortex of monkeys.

As 2D-CNNs are growing in image recognition literature, 3D-CNNs are enthusiastically applied to video action recognition. While spatio-temporal (3D) convolution successfully stems from spatial (2D) convolution, it is still unclear how the convolution works for encoding temporal motion patterns in 3D-CNNs. In this paper, we shed light on the mechanism of feature extraction through analyzing the spatio-temporal filters from a temporal viewpoint. The analysis not only describes characteristics of the two action datasets, Something-Something-v2 (SSv2) and Kinetics-400, but also reveals how temporal dynamics are characterized through stacked spatio-temporal convolutions. Based on the analysis, we propose methods to improve temporal feature extraction, covering temporal filter representation and temporal data augmentation. The proposed method contributes to enlarging temporal receptive field of 3D-CNN without touching its fundamental architecture, thus keeping the computation cost. In the experiments on action classification using SSv2 and Kinetics-400, it produces favorable performance improvement of 3D-CNNs.

A natural image frequently contains multiple classification targets, accordingly providing multiple class labels rather than a single label per image. While the single-label classification is effectively addressed by applying a softmax cross-entropy loss, the multi-label task is tackled mainly in a binary cross-entropy (BCE) framework. In contrast to the softmax loss, the BCE loss involves issues regarding imbalance as multiple classes are decomposed into a bunch of binary classifications; recent works improve the BCE loss to cope with the issue by means of weighting. In this paper, we propose a multi-label loss by bridging a gap between the softmax loss and the multi-label scenario. The proposed loss function is formulated on the basis of relative comparison among classes which also enables us to further improve discriminative power of features by enhancing classification margin. The loss function is so flexible as to be applicable to a multi-label setting in two ways for discriminating classes as well as samples. In the experiments on multi-label classification, the proposed method exhibits competitive performance to the other multi-label losses, and it also provides transferrable features on single-label ImageNet training. Codes are available at https://github.com/tk1980/TwowayMultiLabelLoss.

Non-negative matrix factorization (NMF) is widely applied to analyze pattern data in an unsupervised manner. It imposes hard non-negativity constraints on factors to extract intrinsic characteristics from an input matrix, though demanding complicated optimization techniques which hinder the general applicability. Toward flexible formulation, we propose weakly non-negative factorization. In contrast to the strict non-negative approach, our method permits factors to contain small amount of negative values. The relaxation theoretically leads to an efficient factorization formulation which can be implemented by means of off-the-shelf techniques used in a deep learning literature. Thus, the method is flexibly applicable to versatile factorization tasks, such as deep NMF and structured NMF. In the experiments on the NMF-related tasks, we demonstrate that the weak non-negativity produces effective factors similarly to NMF and the method exhibits favorable performance in comparison to the other approaches.

In this paper, we reveal the discriminant capacity of orthogonal data projection onto the generalized difference subspace (GDS), both theoretically and experimentally. In our previous work, we demonstrated that the GDS projection works as a quasi-orthogonalization of class subspaces, which is an effective feature extraction for subspace based classifiers. Here, we further show that GDS projection also works as a discriminant feature extraction through a similar mechanism to the Fisher discriminant analysis (FDA). A direct proof of the connection between GDS projection and FDA is difficult due to the significant difference in their formulations. To circumvent the complication, we first introduce geometrical Fisher discriminant analysis (gFDA) based on a simplified Fisher criterion. It is derived from a heuristic yet practically plausible assumption: the direction of the sample mean vector of a class is largely aligned to the first principal component vector of the class, given that the principal component analysis (PCA) is applied without data centering. gFDA works stably even under few samples, bypassing the small sample size (SSS) problem of FDA. We then prove that gFDA is equivalent to GDS projection with a small correction term. This equivalence ensures GDS projection to inherit the discriminant ability from FDA via gFDA. Furthermore, we discuss two useful extensions of these methods, 1) a nonlinear extension by kernel trick, 2) a combination with CNN features. The equivalence and the effectiveness of the extensions have been verified through extensive experiments on the extended Yale B+, CMU face database, ALOI, ETH80, MNIST, and CIFAR10, mainly focusing on image recognition under small samples.

Knowledge distillation (KD) transfers knowledge of a teacher model to improve performance of a student model which is usually equipped with lower capacity. In the KD framework, however, it is unclear what kind of knowledge is effective and how it is transferred. This paper analyzes a KD process to explore the key factors. In a KD formulation, softmax temperature entangles three main components of student and teacher probabilities and a weight for KD, making it hard to analyze contributions of those factors separately. We disentangle those components so as to further analyze especially the temperature and improve the components respectively. Based on the analysis about temperature and uniformity of the teacher probability, we propose a method, called extractive distillation, for extracting effective knowledge from the teacher model. The extractive KD touches only teacher knowledge, thus being applicable to various KD methods. In the experiments on image classification tasks using Cifar-100 and TinyImageNet datasets, we demonstrate that the proposed method outperforms the other KD methods and analyze feature representation to show its effectiveness in the framework of transfer learning.

Deep convolutional neural networks (CNNs) leverage large-scale training dataset to produce remarkable performance on various image classification tasks. It, however, is difficult to effectively train the CNNs on some realistic learning situations such as regarding class imbalance, small-scale and label noises. Regularizing CNNs works well on learning with such deteriorated training datasets by mitigating overfitting issues. In this work, we propose a method to effectively impose regularization on feature representation learning. By focusing on the angle between a feature and a classifier which is embedded in cosine similarity at the classification layer, we formulate a novel similarity beyond the cosine based on von Mises-Fisher distribution of directional statistics. In contrast to the cosine similarity, our similarity is compact while having heavy tail, which contributes to regularizing intra-class feature distribution to improve generalization performance. Through the experiments on some realistic learning situations such as of imbalance, small-scale and noisy labels, we demonstrate the effectiveness of the proposed method for training CNNs, in comparison to the other regularization methods. Codes are available at https://github.com/tk1980/tvMF.

In the supervised learning framework, a softmax cross-entropy loss is commonly applied to train deep neural networks for high-performance classification. It, however, demands large amount of annotated data and fails to learn the discriminative networks on a smaller amount of data. In this paper, we propose a novel loss measure to train the networks such that discriminative feature representation can be learned even on the smaller-scale dataset. By means of feature grouping, we effectively expose non-discriminative feature components to representation learning and formulate two types of group softmax losses to cope with the grouped features. The proposed method encourages discriminative representation across all feature components, and from a theoretical viewpoint it renders adversarial training which works for alleviating over-fitting especially on scarce training data. The experimental results on image classification tasks demonstrate that the proposed loss favorably improves performance of CNNs on various-scale data.

Stochastic gradient descent (SGD) is successfully applied to train deep convolutional neural networks (CNNs) on various computer vision tasks. Since fixed step-size Sgd converges to so-called error plateau, it is applied in combination with decaying learning rate to reach a favorable optimum. In this paper, we propose a simple yet effective optimization method to improve Sgd with a phase-wise decay of learning rate. Through analyzing both a loss surface around the error plateau and a structure of the Sgd optimization process, the proposed method is formulated to improve convergence as well as initialization at each training phase by efficiently aggregating the CNN parameters along the optimization sequence. The method keeps the simplicity of Sgd while touching the Sgd procedure only a few times during training. The experimental results on image classification tasks thoroughly validate the effectiveness of the proposed method in comparison to the other methods.

As the field of deep learning has been rapidly growing, the optimization methods (optimizers) gain keen attention for efficiently training neural networks. While SGD exhibits practically favorable performance on various tasks, adaptive methods, such as ADAM, are also formulated to equip the gradient-based updating with adaptive scaling in a sophisticated way. In this paper, we propose a novel optimizer to integrate those two approaches of the adaptive method and SGD through assigning stochastic confidence weights to the gradient-based updating. We define statistical uncertainty of the gradients which is implicitly embedded in the adaptive scaling of ADAM, and then based on the uncertainty, naturally incorporate stochasticity into the optimizer as a bridge between SGD and ADAM. Thereby, the proposed optimizer, SCWSGD, endows the parameter updating with two types of stochasticity regarding multiplicative scaling for the gradient and mini-batch sampling to compute the gradient, for improving generalization performance. In the experiments on image classification using various CNNs, the proposed optimizer produces favorable performance in comparison to the other optimizers.

In deep convolutional neural networks (CNNs), local pooling operation is a key building block to effectively downsize feature maps for reducing computation cost as well as increasing robustness against input variation. There are several types of pooling operation, such as average/max-pooling, from which one has to be manually selected for building CNNs. The optimal pooling type would be dependent on characteristics of features in CNNs and classification tasks, making it hard to find out the proper pooling module in advance. In this paper, we propose a flexible pooling method which adaptively tunes the pooling functionality based on input features without manually fixing it beforehand. In the proposed method, the parameterized pooling form is derived from a probabilistic perspective to flexibly represent various types of pooling and then the parameters are estimated by means of global statistics in the input feature map. Thus, the proposed local pooling guided by global features effectively works in the CNNs trained in an end-to-end manner. The experimental results on image classification tasks demonstrate the effectiveness of the proposed pooling method in various deep CNNs.

Convolutional neural networks (CNNs) contain local pooling to effectively downsize feature maps for increasing computation efficiency as well as robustness to input variations. The local pooling methods are generally formulated in a form of convex combination of local neuron activations for retaining the characteristics of an input feature map in a manner similar to image downscaling. In this paper, to improve performance of CNNs, we propose a novel local pooling method based on the Gaussian-based probabilistic model over local neuron activations for flexibly pooling (extracting) features, in contrast to the previous model restricting the output within the convex hull of local neurons. In the proposed method, the local neuron activations are aggregated into the statistics of mean and standard deviation in a Gaussian distribution, and then on the basis of those statistics, we construct the probabilistic model suitable for the pooling in accordance with the knowledge about local pooling in CNNs. Through the probabilistic model equipped with trainable parameters, the proposed method naturally integrates two schemes of adaptively training the pooling form based on input feature maps and stochastically performing the pooling throughout the end-to-end learning. The experimental results on image classification demonstrate that the proposed method favorably improves performance of various CNNs in comparison with the other pooling methods.

Deep convolutional neural network (ConvNet) is a promising approach for high-performance image classification. The behavior of ConvNet is analyzed mainly based on the neuron activations, such as by visualizing them. In this paper, in contrast to the activations, we focus on filters which are main components of ConvNets. Through analyzing two types of filters at convolution and fully-connected (FC) layers, respectively, on various pre-trained ConvNets, we present the methods to efficiently reformulate the filters, contributing to improving both memory size and classification performance of the ConvNets. They render the filter bases formulated in a parameter-free form as well as the efficient representation for the FC layer. The experimental results on image classification show that the methods are favorably applied to improve various ConvNets, including ResNet, trained on ImageNet with exhibiting high transferability on the other datasets.

A deep neural network is one of the promising approach to produce state-of-the-art performance on various fields such as pattern recognition and signal processing. While the network architecture is intensively studied, as to the network components, non-linear activation functions are the main subject of research in the literature. Most of the activation functions, such as a rectified linear unit (ReLU), operate on each of feature channels in an element-wise manner and thus can be regarded as extracting occurrence characteristics from the input feature map. In this paper, we propose a co-occurrence activation unit to work across feature channels by extending the element-wise activation function. In contrast to the original co-occurrence formulation applied to hand-crafted feature extraction methods, the proposed co-occurrence unit is trainable by a gradient-based optimization through back-propagation learning and exploits the co-occurrence relationships among the feature channels. The experimental results on image classification datasets show that the proposed co-occurrence activation unit embedded into various types of ConvNets favorably improve classification performance.

In action recognition, local motion descriptors contribute to effectively representing video sequences where target actions appear in localized spatio-temporal regions. For robust recognition, those fundamental descriptors are required to be invariant against horizontal (mirror) flipping in video frames which frequently occurs due to changes of camera viewpoints and action directions, deteriorating classification performance. In this paper, we propose methods to render flip invariance to the local motion descriptors by two approaches. One method leverages local motion flows to ensure the invariance on input patches where the descriptors are computed. The other derives a invariant form theoretically from the flipping transformation applied to hand-crafted descriptors. The method is also extended so as to deal with ConvNet descriptors through learning the invariant form based on data. The experimental results on human action classification show that the proposed methods favorably improve performance both of the handcrafted and the ConvNet descriptors.

Feature matching is a fundamental process in a variety of computer vision tasks. Beyond the standard L2metric, various methods to measure similarity between features have been proposed mainly on the assumption that the features are defined in a histogram form. On the other hand, in a field of image quality assessment, SSIM [27] produces effective similarity between images, taking the place of L2metric. In this paper, we propose a feature similarity measurement method based on the SSIM. Unlike the previous methods, the proposed method is built on not a histogram form but a tensor structure of a feature array extracted such as on spatial grids, in order to construct effective SSIM-based similarity measure of high robustness which is a key requirement in feature matching. In addition, we provide the explicit feature map such that the proposed similarity metric is embedded as a dot product. It contributes to significant speedup in similarity measurement as well as to feature transformation toward an effective vector form to which linear classifiers are directly applicable. In the experiments on various tasks, the proposed method exhibits favorable performance in both feature matching and classification.

A histogram is an effective form for extracting various types of features and has been attracting keen attention in pattern recognition fields. In visual recognition, however, the histogram features suffer from smoothing due to the processes of quantizing continuous input patterns into discrete codes and pooling them. Such smoothing degrades discriminative power by blurring the distinctive features. In this paper, we propose a novel method to get rid of such blurriness and reveal the essential discriminative information from the histogram features. We first model the blurring process based on the graph Laplacian of discrete codes and then formulate a method for deblurring histogram features in a computationally efficient form which works as post-processing just after feature extraction. The proposed method is also shown to be related to unsharp masking of an image restoration technique. In the experiments on image classification using BoW histogram features, the proposed deblurring method favorably improves performance of the histogram features.

Filter banks on a frequency domain are widely applied and studied mainly for MFCC and its variant methods on speech recognition tasks. In recent years, other types of acoustic features which are derived from image classification literature have attracted attentions for the tasks regarding environmental sounds. For those features, the filter banks can also be employed mainly to effectively reduce feature dimensionality along the frequency. The filter banks have been designed according to human auditory process and are not necessarily optimal from the viewpoint of distinguishing actual acoustic data. We propose a method to build a filter bank from scratch in a data-driven manner based on the natural properties of filter banks without parametrically modeling them, which thereby more flexibly describes intrinsic characteristics of data. Those filters are optimized by incorporating discriminative criterion so as to provide effective features of high performance even with the smaller-sized filter bank. In the experiments on acoustic scene classification, the proposed method exhibits favorable performance especially on lower dimensional features.

Image quality assessment gains a greater interest due to development of digital imaging and storage. In that field, structural similarity (SSIM) index has been shown to favorably agree with human perceptual assessment, significantly outperforming the method of mean squared error, i.e., L2distance. The similarity measure function in SSIM which compares a target (distorted) image with its reference (original) image is handcrafted in a simple form via a top-down approach based on the human visual system. It, however, might lack optimality without directly considering the relationships between image data and the perceptual assessment (scores). In this paper, we propose a method to construct an image similarity measure based on actual data. The proposed method optimizes a similarity measure function by exploiting annotated data in a bottom-up and data-driven manner, while retaining the favorable property of structural similarity in SSIM. The non-linear similarity function is optimized as the global optimum of high generalization power. In addition, the proposed method is simply formulated and thus applicable to the family of SSIM, especially to FSIM which has been recently proposed exhibiting superior performance to SSIM. The experimental results on image quality assessment demonstrate the effectiveness of the proposed method compared to the other methods.

In contrast to category-level or cluster-level classifiers, exemplar SVM [17] is successfully applied to classifying (or detecting) a target object as well as transferring instance-level annotations. The method, however, is formulated in a highly biased classification problem where only one positive sample is contrasted with a substantial number of negative samples, which makes it difficult to properly determine the regularization parameters balancing two types of costs derived from positive and negative samples. In this paper, we present two novel viewpoints toward exemplar SVM in addition to the original definition. From these proposed viewpoints, we can give light on an intrinsic structure of exemplar SVM, reducing two parameters into only one as well as providing clear intuition on the parameter, in order to free us from exhaustive parameter tuning. We can also clarify how the classifier geometrically works so as to produce homogeneous classification scores of multiple exemplar SVMs which are comparable to each other without calibration. In addition, we propose a novel feature transformation method based on those viewpoints which contributes to general classification tasks. In the experiments on object detection and image classification, the proposed methods regarding exemplar SVM exhibit favorable performance.

Semantic labelling of acoustic scenes has recently emerged as active topic covering a wide range of applications, e.g. surveillance and audio-based information retrieval. In this paper, we present an effective approach for acoustic scene classification through characterizing both background sound textures and acoustic events. The work takes inspiration from the psychoacoustic definition of acoustic scenes, that is, "skeleton of (acoustic) events on a bed of (sound) texture". In detail, we firstly employ distinct models to exploit sound textures and events in acoustic scenes, individually. Subsequently, based on fact that the perceptual importance of two parts will vary with respect to different scene categories, we develop favourable class-conditional fusion scheme to aggregate two-channel information. To validate proposed approach, we conduct extensive experiments on Rouen dataset which includes 19 categories of daily acoustic scenes with 3026 real-world recordings, and the proposed approach outperforms state-of-the-art methods by a large margin.

Histogram-based features have significantly contributed to recent development of image classifications, such as by SIFT local descriptors. In this paper, we propose a method to efficiently transform those histogram features for improving the classification performance. The (L1-normalized) histogram feature is regarded as a probability mass function, which is modeled by Dirichlet distribution. Based on the probabilistic modeling, we induce the Dirichlet Fisher kernel for transforming the histogram feature vector. The method works on the individual histogram feature to enhance the discriminative power at a low computational cost. On the other hand, in the bag-of-feature (BoF) frame- work, the Dirichlet mixture model can be extended to Gaussian mixture by transforming histogram-based local descriptors, e.g., SIFT, and thereby we propose the method of Dirichlet-derived GMM Fisher kernel. In the experiments on diverse image classification tasks including recognition of subordinate objects and material textures, the pro- posed methods improve the performance of the histogram- based features and BoF-based Fisher kernel, being favor- ably competitive with the state-of-the-arts.

Classification of environmental sounds is a fundamental procedure for a wide range of real-world applications. In this paper, we propose a novel acoustic feature extraction method for classifying the environmental sounds. The proposed method is motivated from the image processing technique, local binary pattern (LBP), and works on a spectrogram which forms two-dimensional (time-frequency) data like an image. Since the spectrogram contains noisy pixel values, for improving classification performance, it is crucial to extract the features which are robust to the fluctuations in pixel values. We effectively incorporate the local statistics, mean and standard deviation on local pixels, to establish robust LBP. In addition, we provide the technique of L2-Hellinger normalization which is efficiently applied to the proposed features so as to further enhance the discriminative power while increasing the robustness. In the experiments on environmental sound classification using RWCP dataset that contains 105 sound categories, the proposed method produces the superior performance (98.62%) compared to the other methods, exhibiting significant improvements over the standard LBP method as well as robustness to noise and low computation time.

In this paper, we present a novel method for environmental sound classification in non-stationary noise environment. The proposed method mainly consists of three stages: noise source separation and acoustic feature extraction and multi-class classification. At first stage, we employ probabilistic latent component analysis (PLCA) to perform time-varying noise separation. To alleviate the artifacts introduced by source separation, a series of spectral weightings is applied to enhance reliability of audio spectra. At feature extraction stage, we extract acoustic subspace to effectively characterize temporal-spectral patterns of denoised sound spectrogram. Subsequently, regularized kernel Fisher discriminant analysis (KFDA) is adopted to conduct multi-class sound classification through exploiting class conditional distributions based on extracted acoustic subspaces (features). The proposed method is evaluated with Real World Computing Partnership (RWCP) sound scene database and experimental results demonstrate its superior performance compared to other methods.

The partial pattern matching is fundamental for pattern recognition to compare the pair of input patterns by exploiting the common features shared by those patterns while excluding the irrelevant ones. In this paper, for the pattern matching, we propose a novel method of smoothly structured sparse canonical correlation analysis, called S3CCA. The proposed method works on the feature matrix composed of a (local) feature dimension and an array dimension. In the framework of CCA, the method provides map weights along the array dimension to depict the parts that exhibit the common/similar features across the pair of feature matrices. By introducing the appropriate regularization into CCA, the map weights are optimized so as to be both smooth and localized, i.e., structured sparse. Thereby, the common features are effectively detected by the smooth and well-localized weights to improve the matching performance. In the experiments on pattern matching as well as classification based on the matching, the proposed method produces the favorable performance compared to the other methods.

Abstract: Prior information has been effectively exploited mainly using probabilistic models. In this paper, by focus-ing on the bias embedded in the classifier, we propose a novel method to discriminatively learn the prior bias based on the extra prior information assigned to the samples other than the class category, e.g., the 2-D position where the local image feature is extracted. The proposed method is formulated in the framework of maximum margin to adaptively optimize the biases, improving the classification performance. We also present the computationally efficient optimization approach that makes the method even faster than the standard SVM of the same size. The experimental results on patch labeling in the on-board camera images demonstrate the favorable performance of the proposed method in terms of both classification accuracy and computation time. 1

A novel algorithm for tracking by shape with deforming prediction is proposed. The algorithm is based on the similarity of the predicted and actual object shape. Second order approximation for feature point movement by Taylor expansion is adopted for shape prediction, and the similarity is measured by using chamfer matching of the predicted and the actual shape. Chamfer matching is also used to detect the feature point movements to predict the object deformation. The proposed algorithm is applied to the tracking of a skier and showed a good tracking and shape prediction performance.

Image classification methods have been significantly developed in the last decade. Most methods stem from bag-of-features (BoF) approach and it is recently extended to a vector aggregation model, such as using Fisher kernels. In this paper, we propose a novel feature extraction method for image classification. Following the BoF approach, a plenty of local descriptors are first extracted in an image and the proposed method is built upon the probability density function (p.d.f) formed by those descriptors. Since the p.d.f essentially represents the image, we extract the features from the p.d.f by means of the gradients on the p.d.f. The gradients, especially their orientations, effectively characterize the shape of the p.d.f from the geometrical viewpoint. We construct the features by the histogram of the oriented p.d.f gradients via orientation coding followed by aggregation of the orientation codes. The proposed image features, imposing no specific assumption on the targets, are so general as to be applicable to any kinds of tasks regarding image classifications. In the experiments on object recognition and scene classification using various datasets, the proposed method exhibits superior performances compared to the other existing methods.

Sparsity induced in the optimized weights effectively works for factorization with robustness to noises and for classification with feature selection. For enhancing the sparsity, L1regularization is introduced into the objective cost function to be minimized. In general, however, Lp(p1, though Lpregularized problem is difficult to be effectively optimized. In this paper, we propose a method to efficiently optimize the Lpregularized problem. The method reduces the Lpproblem into L1regularized one via transforming target variables by the mapping based on Lp, and optimizes it by using orthant-wise approach. In the proposed method, the Lpproblem is directly optimized for computational efficiency without reformulating it into iteratively reweighting scheme. The proposed method is generally applicable to various problems with Lpregularization, such as factorization and classification. In the experiments on the classification using logistic regression and factorization based on least squares, the proposed method produces favorable sparse results.

In this paper, we propose an effective discriminant subspace learning framework to recognize the environmental sounds. Firstly, Gabor transform is adopted to characterize the time-frequency distributions of environmental sounds. We further encode the prominent time-frequency patterns with low rank representation by extracting the subspace from Gabor spectrogram. Unlike conventional sound recognition schemes that are mostly based on acoustic feature vectors, we treat the acoustic subspaces (matrixes) as basic elements for recognition, retaining rich temporal-spectral contextual information. At recognition stage, we employ kernel Fisher discriminant analysis to effectively exploit the class conditional distributions of environmental sounds which are favorable for performing multi-class classification. With a well developed kernel function, the proposed approach achieved superior recognition performance on RWCP sound scene database, compared with the existing methods.

We propose a method to extract image features based on effective higher-order cooccurrences. The proposed method constructs the co-clusters to discriminatively quantize joint primitive quantitative data, such as pair-wise pixel intensities, unlike the standard co-occurrence methods that utilize simple clusters trained in an unsupervised manner for quantizing point-wise data. The discriminative co-clusters effectively exploit the co-occurrence characteristics even by a fewer number of cluster components, resulting in low-dimensional co-occurrence features. By taking advantage of those discriminative co-clusters, the co-occurrence features can be extended to the higher-order co-occurrence features of feasible dimensionality. The higher-order co-occurrence captures richer information in image textures by extracting relationships in multiplets more than only doublets (pairs). In the experiments on image classifications for cancer cells and pedestrians, the proposed method exhibits favorable performances compared to the other methods, even to the standard co-occurrence based methods.

In this paper, we propose a novel method of label propagation for one-class learning. For binary (positive/negative) classification, the proposed method simultaneously measures the pair-wise similarity between samples and the negativity at every sample based on a cone-based model of local neighborhoods. Relying only on positive labeled samples as in one-class learning, the method estimates the labels of unlabeled samples via label propagation using the similarities and the negativities in the framework of semi-supervised learning. In the proposed method, unlike standard label propagation methods, it is not necessary to prepare negative labeled samples since the measured negativity works as an alternative of such labeling negative samples. In experiments on target detection in still images and motion images, the proposed method exhibits the favorable performances compared to the other methods.

In this paper, we propose a method to extract features from three-dimensional acceleration signals. The proposed method is based on the (auto-)correlation matrix of Fourier transform features, naturally containing the correlations between the frequencies as well as the ordinary power spectrum for each frequency. The proposed features are inherently invariant to both rotational variations and temporal shift (delay), whereas the other methods employ ad hoc preprocessing to increase robustness to those variations. Thereby, we can favorably apply the proposed method to analyze 3-D acceleration signals regardless of the orientations of the accelerometer. In the experiment on gait identification using an accelerometer embedded in a cellular phone, the proposed method outperformed the other methods.

In biology, peptide ion detection from mass spectra is important for identifying proteins. Many methods have been proposed for detecting peptide ion peaks, some of which use wavelet transform. In these methods, however, the co-occurrence pattern of peptide ions and those isotopes is not directly considered. In this paper, we propose a novel method for detecting peptide ion peaks from a mass spectrum by using a weighted auto-correlation. The weight functions derived from Maxwell-Boltzmann distribution and the sine function are introduced to the proposed auto correlations to effectively represent the peptide ion co-occurrence patterns. The multi-scaled auto-correlation features extracted with those weight functions are compressed by using principal component analysis. Experiments on raw mass spectra show that the proposed method achieves the favorable performances and is capable of automatically detecting peptide ion peaks.

In the cancer detection from stained biopsy images, it is important to extract histologically discriminative characteristics. For this purpose, we propose a novel method to extract statistical and morphological features. At the first stage, we estimate cell component memberships at each pixel by applying an expectation maximization (EM) algorithm to the color information. Next we calculate the local co-occurrence of the memberships as image features. And then, linear discriminant analysis (LDA) is applied to those features for final decision of whether cancer or not, with enhancing the discrimination. In the experiments on real biopsy images of cancers, the resulting detection accuracy is superior to the other methods.

We propose cone-restricted kernel subspace methods for pattern classification. A cone is mathematically defined in a manner similar to a linear subspace with a nonnegativity constraint. Since the angles between vectors (i.e., inner products) are fundamental to the cone, kernel tricks can be directly applied. The proposed methods approximate the distribution of sample patterns by using the cone in kernel feature space via kernel tricks, and the classification is more accurate than that of the kernel subspace method. Due to the nonlinearity of kernel functions, even a single cone in the kernel feature space can can cope with multi-modal distributions in the original input space. In the experimental results on person detection and motion detection, the proposed methods exhibit the favorable performances.

The goal of image segmentation is to partition an image into regions that are internally homogeneous and heterogeneous with respect to neighbouring regions. Recently, a link shifting based pyramidal segmentation method was proposed to resolve existing problems with elongated regions. In this paper, we propose further improvements by replacing pixel intensities at the base level with pixel level higher order local auto-correlation (HLAC) feature vectors over greyscale, RGB, and CIV channels. Thereby, rich texture-like information is incorporated into segmentation. We propose a normalized distance formula between HLAC vectors, where each component contributes with physically same unit. The new algorithms were tested on a set of Landsat images over forested areas, and compared with a non-HLAC variant and several other existing segmentation algorithms. A significant improvement in segmentation quality was achieved compared to non-HLAC variants, and it also gave better results than other existing algorithms on most examples.

We propose a method of clustering sample vectors on a hypersphere. Sample vectors are normalized in many cases, especially when applying kernel functions, and thus lie on a (unit) hypersphere. Considering the constraint of the hypersphere, the proposed method utilizes the von Mises-Fisher distribution in the framework of mean shift. It is also extended to the kernel-based clustering method via kernel tricks to cope with complex distributions. The algorithms of the proposed methods are based on simple matrix calculations. In the experiments, including a practical motion clustering task, the proposed methods produce favorable clustering results.

We propose a bag-of-hierarchical-co-occurrence features method incorporating hierarchical structures for image classification. Local co-occurrences of visual words effectively characterize the spatial alignment of objects' components. The visual words are hierarchically constructed in the feature space, which helps us to extract higher-level words and to avoid quantization error in assigning the words to descriptors. For extracting descriptors, we employ two types of features hierarchically: narrow (local) descriptors, like SIFT, and broad descriptors based on co-occurrence features. The proposed method thus captures the co-occurrences of both small and large components. We conduct an experiment on image classification by applying the method to the Caltech 101 dataset and show the favorable performance of the proposed method.

In this paper, we propose a method for extracting color image features, called color index local auto-correlations. Pixel color is quantized and described sparsely in a manner similar to the color indexing of color histograms. In addition, by utilizing spatial auto-correlations of the color indexes, the characteristics of color texture can be extracted more effectively than ordinary histogram-based methods. The proposed method has variants in terms of the color space and basic colors used for indexing colors. These various settings are comprehensively compared in the experiments of image retrieval and image classification, and are shown to exhibit favorable results compared to the other conventional methods.

Kernel-based methods, e.g., support vector machine (SVM), produce high classification performances. However, the computation becomes time-consuming as the number of the vectors supporting the classifier increases. In this paper, we propose a method for reducing the computational cost of classification by kernel-based methods while retaining the high performance. By using linear algebra of a kernel Gram matrix of the support vectors (SVs) at low computational cost, the method efficiently prunes the redundant SVs which are unnecessary for constructing the classifier. The pruning is based on the evaluation of the performance of the classifier formed by the reduced SVs in SVM. In the experiment of classification using SVM for various datasets, the feasibility of the evaluation criterion and the effectiveness of the proposed method are demonstrated.

We propose methods for segmenting a motion sequence into motion primitives, taking into account temporal constraints (continuity along the time axis). In the proposed methods, dynamic programming (DP) is used on a motion feature sequence to allow for the effects of these constraints on the results of the segmentation. The methods do not require such a running window along the time axis, as is typical for the usual methods, and thus they can be applied to the segmentation of transient motions. The results of comparative experiments using several motion features and segmentation methods on weightlifting motion data demonstrate the effectiveness of the proposed methods.

In pattern recognition, feature vectors are occasionally subject to non-negative constraints. This characteristic can be expressed by a cone in feature vector space. In this paper, we propose cone-restricted subspace methods. The proposed methods admit the scaling and additivity of vectors as well as ordinary subspace methods; in addition, vectors can be strictly classified at the boundary of the cone. Some experimental results for face and person detection demonstrate the effectiveness of the proposed methods.

Image matting and segmentation, which are used to extract a foreground object from the background, are primary techniques for digital image and video editing. In digital matting, the transparency of the foreground object is considered, while segmentation performs a rigid extraction of the object. Recently, several algorithms for matting and segmentation problems have been proposed and have provided high-quality results. In this paper, we propose a unified formulation for image matting in the framework of the method of quantification IV based on a review of the previous studies from this framework. Our method also utilizes discriminative information provided by a user (a few strokes drawn by the user). The experimental results show a favorable matting performance.