Two main concepts studied in machine learning theory are generalization gap (difference between train and test error) and excess risk (difference between test error and the minimum possible error). While information-theoretic tools have been used extensively to study the generalization gap of learning algorithms, the information-theoretic nature of excess risk has not yet been fully investigated. In this paper, some steps are taken toward this goal. We consider the frequentist problem of minimax excess risk as a zero-sum game between the algorithm designer and the world. Then, we argue that it is desirable to modify this game in a way that the order of play can be swapped. We then prove that, under some regularity conditions, if the world and designer can play randomly the duality gap is zero and the order of play can be changed. In this case, a Bayesian problem surfaces in the dual representation. This makes it possible to utilize recent information-theoretic results on minimum excess risk in Bayesian learning to provide bounds on the minimax excess risk. We demonstrate the applicability of the results by providing information theoretic insight on two important classes of problems: classification when the hypothesis space has finite VC-dimension, and regularized least squares.

Visual target tracking is one of the most sought-after yet challenging research topics in computer vision. Given the ill-posed nature of the problem and its popularity in a broad range of real-world scenarios, a number of large-scale benchmark datasets have been established, on which considerable methods have been developed and demonstrated with significant progress in recent years – predominantly by recentdeep learning(DL)-based methods. This survey aims to systematically investigate the current DL-based visual tracking methods, benchmark datasets, and evaluation metrics. It also extensively evaluates and analyzes the leading visual tracking methods. First, the fundamental characteristics, primary motivations, and contributions of DL-based methods are summarized from nine key aspects of: network architecture, network exploitation, network training for visual tracking, network objective, network output, exploitation of correlation filter advantages, aerial-view tracking, long-term tracking, and online tracking. Second, popular visual tracking benchmarks and their respective properties are compared, and their evaluation metrics are summarized. Third, the state-of-the-art DL-based methods are comprehensively examined on a set of well-established benchmarks of OTB2013, OTB2015, VOT2018, LaSOT, UAV123, UAVDT, and VisDrone2019. Finally, by conducting critical analyses of these state-of-the-art trackers quantitatively and qualitatively, their pros and cons under various common scenarios are investigated. It may serve as a gentle use guide for practitioners to weigh when and under what conditions to choose which method(s). It also facilitates a discussion on ongoing issues and sheds light on promising research directions.

In parametric Bayesian learning, a prior is assumed on the parameter $W$ which determines the distribution of samples. In this setting, Minimum Excess Risk (MER) is defined as the difference between the minimum expected loss achievable when learning from data and the minimum expected loss that could be achieved if $W$ was observed. In this paper, we build upon and extend the recent results of (Xu & Raginsky, 2020) to analyze the MER in Bayesian learning and derive information-theoretic bounds on it. We formulate the problem as a (constrained) rate-distortion optimization and show how the solution can be bounded above and below by two other rate-distortion functions that are easier to study. The lower bound represents the minimum possible excess risk achievable by \emph{any} process using $R$ bits of information from the parameter $W$. For the upper bound, the optimization is further constrained to use $R$ bits from the training set, a setting which relates MER to information-theoretic bounds on the generalization gap in frequentist learning. We derive information-theoretic bounds on the difference between these upper and lower bounds and show that they can provide order-wise tight rates for MER under certain conditions. This analysis gives more insight into the information-theoretic nature of Bayesian learning as well as providing novel bounds.

Obtaining generalization bounds for learning algorithms is one of the main subjects studied in theoretical machine learning. In recent years, information-theoretic bounds on generalization have gained the attention of researchers. This approach provides an insight into learning algorithms by considering the mutual information between the model and the training set. In this paper, a probabilistic graphical representation of this approach is adopted and two general techniques to improve the bounds are introduced, namely conditioning and processing. In conditioning, a random variable in the graph is considered as given, while in processing a random variable is substituted with one of its children. These techniques can be used to improve the bounds by either sharpening them or increasing their applicability. It is demonstrated that the proposed framework provides a simple and unified way to explain a variety of recent tightening results. New improved bounds derived by utilizing these techniques are also proposed.

Hand gesture recognition (HGR) from sequences of depth maps is a challenging computer vision task because of the low inter-class and high intra-class variability, different execution rates of each gesture, and the high articulated nature of the human hand. In this paper, a multilevel temporal sampling (MTS) method is first proposed that is based on the motion energy of keyframes of depth sequences. As a result, long, middle, and short sequences are generated that contain the relevant gesture information. The MTS results in increasing the intra-class similarity while raising the inter-class dissimilarities. The weighted depth motion map (WDMM) is then proposed to extract the spatiotemporal information from generated summarized sequences by an accumulated weighted absolute difference of consecutive frames. The histogram of gradient and local binary pattern are exploited to extract features from WDMM. The obtained results define the current state-of-the-art on three public benchmark datasets of: MSR Gesture 3D, SKIG, and MSR Action 3D, for 3D HGR. We also achieve competitive results on NTU action dataset.

The interest in action and gesture recognition has grown considerably in the last years. In this paper, we present a survey on current deep learning methodologies for action and gesture recognition in image sequences. We introduce a taxonomy that summarizes important aspects of deep learning for approaching both tasks. We review the details of the proposed architectures, fusion strategies, main datasets, and competitions. We summarize and discuss the main works proposed so far with particular interest on how they treat the temporal dimension of data, discussing their main features and identify opportunities and challenges for future research.

Although there is an increasing interest in employing the depth data in computer vision applications, the spatial resolution of depth maps is still limited compared with typical visible‐light images. A novel method is proposed to synthetically improve the spatial resolution of a single depth image. It integrates the higher‐order terms into the Markov random field (MRF) formulation of example‐based methods in order to improve the representational power of those methods. The inference is performed by approximately minimising the higher‐order multi‐label MRF energies. In addition, to improve the efficiency of the inference algorithm, a hierarchical scheme on the number of MRF states is proposed. First, a large number of states are used to obtain an initial labelling by solving the minimisation problem of inference for only the first‐order energies. Then, the problem is solved for the higher‐order energies in a smaller number of states. Performance comparisons show that proposed method improves the results of first‐order approaches that are based on simple four‐connected MRF graph structure, both qualitatively and quantitatively.

Voluminous amount of videos in today’s world has made the subject of objective (or semi-objective) classification of videos to be very popular. Among the various descriptors used for video classification, SIFT and LIFT can lead to highly accurate classifiers. But, SIFT descriptor does not consider video motion and LIFT is time-consuming. In this paper, a robust descriptor for semi-supervised classification based on video content is proposed. It holds the benefits of LIFT and SIFT descriptors and overcomes their shortcomings to some extent. For extracting this descriptor, the SIFT descriptor is first used and the motion of the extracted keypoints are then employed to improve the accuracy of the subsequent classification stage. As SIFT descriptor is scale invariant, the proposed method is also robust toward zooming. Also, using the global motion of keypoints in videos helps to neglect the local motions caused during video capturing by the cameraman. In comparison to other works that consider the motion and mobility of videos, the proposed descriptor requires less computations. Obtained results on the TRECVIT 2006 dataset show that the proposed method achieves more accurate results in comparison with SIFT in content-based video classifications by about 15 percent.

Although the ability to estimate the face pose and recognise its identity are common human abilities, they are still a challenge in computer vision context. In this study, the authors aim to overcome these difficulties by learning a non‐linear tensor‐based model based on multi‐linear decomposition. Proposed model maps the high‐dimensional image space into low‐dimensional pose manifold. For preserving the actual distance along the manifold shape, a graph‐based distance measure is proposed. Also, to compensate for the limited number of training poses, mirrored images are added to training ones to improve the recognition accuracy. For performance evaluation of the proposed method, experiments are run on three famous face databases using three different manifold shapes and two different distance measures. Eight training data modes are chosen such that the influential parameters are studied comprehensively. The obtained results confirm the effectiveness of proposed model in achieving high accuracy in pose estimation and multi‐view face recognition, even with different training poses for different identities.

One major challenge encountered in face recognition is how to handle the wide pose variation and in‐depth rotations of head. A multiview face recognition method is proposed in this study that addresses this challenge based on multilinear decomposition approach and pose subspace. In order to preserve the pose manifold geometry among different individuals in pose subspace, a pose‐biased distance measure is proposed. In addition, as one of the impediments in manifold‐based methods is the lack of sufficient data, a new half‐ellipsoid‐based pose generation method is presented. For performance evaluation of the proposed multiview face recognition method, three different experiments are run on three famous face datasets. The obtained recognition accuracy and the cumulative match characteristic curves confirm the effectiveness of the proposed method in wide pose variation, even with limited number of training poses.

A Bayesian framework for 3-D human pose estimation from monocular images based on sparse representation (SR) is introduced. Our probabilistic approach aims at simultaneously learning two overcomplete dictionaries (one for the visual input space and the other for the pose space) with a shared sparse representation. Existing SR-based pose estimation approaches only offer a point estimation of the dictionary and the sparse codes. Therefore, they might be unreliable when the number of training examples is small. Our Bayesian framework estimates a posterior distribution for the sparse codes and the dictionaries from labeled training data. Hence, it is robust to overfitting on small-size training data. Experimental results on various human activities show that the proposed method is superior to the state-of-the-art pose estimation algorithms.

Semantic video analysis and automatic concept extraction play an important role in several applications; including content-based search engines, video indexing, and video summarization. As the Bayesian network is a powerful tool for learning complex patterns, a novel Bayesian network-based method is proposed for automatic event detection and summarization in soccer videos. The proposed method includes efficient algorithms for shot boundary detection, shot view classification, mid-level visual feature extraction, and construction of the related Bayesian network. The method contains of three main stages. In the first stage, the shot boundaries are detected. Using the hidden Markov model, the video is segmented into large and meaningful semantic units, called play-break sequences. In the next stage, several features are extracted from each of these units. Finally, in the last stage, in order to achieve high level semantic features (events and concepts), the Bayesian network is used. The basic part of the method is constructing the Bayesian network, for which the structure is estimated using the Chow-Liu tree. The joint distributions of random variables of the network are modeled by applying the Farlie-Gumbel-Morgenstern family of Copulas. The performance of the proposed method is evaluated on a dataset with about 9 h of soccer videos. The method is capable of detecting seven different events in soccer videos; namely, goal, card, goal attempt, corner, foul, offside, and nonhighlights. Experimental results show the effectiveness and robustness of the proposed method on detecting these events.

New methods based on local spatio-temporal features have exhibited significant performance in action recognition. In these methods, feature selection plays an important role to achieve a superior performance. Actions are represented by local spatio-temporal features extracted from action videos. Ac tion representations are then classified by applying a classifier (such as k-nearest neighbor or SVM). In this paper, we have proposed two feature weighting methods to better discriminate similar actions. We have proposed a definition of feature discrimination power to be used in the feature selection process. Our proposed weighting schemes have greatly improved the final categorization accuracy on the well-known KTH and Weizmann datasets.

A new and general framework-called modified polynomial regression (MPR)-is introduced in this letter, which detects the changes that occurred in remote sensing images. It is an improvement of the conventional polynomial regression (CPR) method. Most change detection (CD) methods, including CPR, do not consider the spatial relations among image pixels. To improve CPR, our proposed framework incorporates the spatial information into the CD process by using linear spatial-oriented image operators. It is proved that MPR preserves the affine invariance property of CPR. A realization of MPR is proposed, which employs the image derivatives to account for spatiality. Experimental results show the superiority of the proposed method over the CPR method and three other difference-based CD methods, namely, simple differencing, linear chronochrome CD, and multivariate alteration detection.

Clustering-based image segmentation is a well-known multispectral image segmentation method. However, as it inherently does not account for the spatial relation among image pixels, it often results in inhomogeneous segmented regions. The recently proposed membership-connectedness (MC)-based segmentation method considers the local and global spatial relations besides the fuzzy clustering stage to improve segmentation accuracy. However, the inherent spatial and intraclass redundancies in multispectral images might decrease the accuracy and efficiency of the method. This letter addresses these two problems and proposes a segmentation method that is based on the MC method, watershed transform, and the proposed size-weighted fuzzy clustering method. The conducted experiments demonstrate the strength of the proposed algorithm in segmenting small objects, which plays an important role in remote-sensing image segmentation applications.

This paper introduces a new dimension reduction (DR) method, calledminimum change rate deviation(MCRD), which is applicable to the DR of remote sensing images. As the main shortcoming of the well-knownprincipal component analysis(PCA) method is that it does not consider the spatial relation among image points, our proposed approach takes into account the spatial relation among neighboring image pixels while preserving all useful properties of PCA. These include uncorrelatedness property in resulted components and the decrease of error with the increasing of the number of selected components. Our proposed method can be considered as a generalization of PCA and, under certain conditions, reduces to it. The proposed MCRD method employs linear spatial operators to consider the spatiality of images. The superiority of MCRD over conventional PCA is demonstrated both mathematically and experimentally. It is shown that MCRD, with an acceptable speed, outperforms PCA in retaining the required information for classification purposes. Moreover, as thelocally linear embedding(LLE) method also employs the spatial relations in its DR process, the performances of MCRD and LLE are compared, and the superiority of the proposed method in both classification accuracy and computational cost is shown.

In this paper, we propose a novel algorithm that combines color and texture information of skin with cellular learning automata to segment skin-like regions in color images. First, the presence of skin colors in an image is detected, using a committee structure, to make decision from several explicit boundary skin models. Detected skin-color regions are then fed to a color texture extractor that extracts the texture features of skin regions via their color statistical properties and maps them to a skin probability map. Cellular learning automatons use this map to make decision on skin-like regions. The proposed algorithm has demonstrated true positive rate of about 83.4% and false positive rate of about 11.3% on the Compaq skin database. Experimental results show the effectiveness of the proposed algorithm.

Video transcoding comprises the necessary operations for the conversion of a compressed video stream from one syntax to another one for inter-network communications, without the need of any further decoding and re-encoding process. The two most recent and popular standard video coders are H.263 and H.264. In this paper, we present tools to enable low complexity intra transcoding from H.263 to H.264 using a novel fast intra prediction mode decision algorithm that does not require extra computation. It is shown that due to basic similarities of H.263 and H.264 it is possible to reuse predictor side information, so this side information is used to simplify the mode and direction decision for intra prediction. Also, as it is shown the proposed algorithm is improved by using temporal correlation between the intra predicted block in the current frame and the motion-compensated block in its reference. This novel transcoding algorithm reduces the computational complexity, for mode and direction estimation, while maintaining similar PSNR and bitrate.

This paper proposes two enhanced cross-diamond-hexagonal search algorithms to solve the motion-estimation problem in video coding. These algorithms differ from each other by their second step search only, and both of them employ cross-shaped pattern in first step. Proposed method is an improvement over CDHS, which eliminates some checking points of CDHS algorithm. Experimental results show that the proposed methods perform faster than the diamond search (DS) and CDHS, whereas similar quality is preserved.

Fuzzy logic has proved to be a powerful tool to represent imprecise and irregular patterns. This paper presents a novel fuzzy approach for recognizing online Persian (Farsi) handwriting which is also useful for multi-writer environments. In this approach, the representation of handwriting parameters is accomplished by fuzzy linguistic modeling. The representative features are selected to describe the shape of tokens. Fuzzy linguistic terms provide robustness against handwriting variations. The purposed method was run on a database of Persian isolated handwritten characters and achieved a relatively high recognition rate.

We present the most general form for error control coding using finite field multirate filters. This method shows how different types of codes can easily be generated by multirate filters and filter banks. In all previous works, codes and syndromes were generated using prefilters. We present simple multirate structures for encoding and generating syndromes. We show that all kinds of arbitrary rate K/L, circulant linear codes can be generated by these structures. Then we claim that a similar simple structure exists for syndrome generation in all presented cases.

The conventional implementation of multi-dimensional wavelet transform (e.g. 3D wavelet) requires either a high amount of "in access" memory or a continual access to slow memory of a processor which makes it infeasible for most applications. In this paper, we propose a novel algorithm for computation of an nD discrete wavelet transform (DWT) based on a lifting scheme. In addition to the benefits of the lifting scheme (which causes a major reduction in computational complexity and performs the total computations in the time domain), our real-time approach computes the coefficients for all kinds of 1/sup st/ and 2/sup nd/ generation wavelets with short delay and optimized utilization of the slow and fast memories of a processor.

A novel compression algorithm for fingerprint images is introduced. Using wavelet packets and lattice vector quantization , a new vector quantization scheme based on an accurate model for the distribution of the wavelet coefficients is presented. The model is based on the generalized Gaussian distribution. We also discuss a new method for determining the largest radius of the lattice used and its scaling factor , for both uniform and piecewise-uniform pyramidal lattices. The proposed algorithms aim at achieving the best rate-distortion function by adapting to the characteristics of the subimages. In the proposed optimization algorithm, no assumptions about the lattice parameters are made, and no training and multi-quantizing are required. We also show that the wedge region problem encountered with sharply distributed random sources is resolved in the proposed algorithm. The proposed algorithms adapt to variability in input images and to specified bit rates. Compared to other available image compression algorithms, the proposed algorithms result in higher quality reconstructed images for identical bit rates.

A novel compression algorithm for fingerprint images is introduced. Using wavelet packets and lattice vector quantisation, a new vector quantisation scheme based on an accurate model for the distribution of the wavelet coefficients is presented. In the new algorithm, no assumptions are made about the lattice parameters and no training and multi-quantising are required. The proposed algorithms achieve the best rate-distortion performance by adapting to the statistical characteristics of the source image in each sub-image. Compared to other available image compression algorithms, the proposed algorithms result in higher quality reconstructed images for identical bit rates.

A new compression algorithm for fingerprint images is introduced. Using lattice vector quantization (LVQ), a technique for determining the largest radius of the lattice and its scaling factor is presented. The design is based on obtaining the smallest possible expected total distortion (ETD) measure, using a given bit budget, while using the smallest codebook size. In the proposed piecewise-uniform pyramid LVQ, the wedge problem encountered with the pyramidal lattice point shells is resolved. At very low bit rates, for the coefficients with high-frequency content, the positive-negative mean (PNM) method is proposed to improve the resolution of the reconstructed image. The proposed algorithm results in a high compression ratio and a high reconstructed image quality with a low computational load compared to other existing algorithms.