Convolutional Neural Networks (CNNs) are very effective for many pattern recognition tasks. However, training deep CNNs needs extensive computation and large training data. In this paper we propose Bank of Filter-Trees (BFT) as a transfer learning mechanism for improving efficiency of learning CNNs. A filter-tree corresponding to a filter in kth convolutional layer of a CNN is a subnetwork consisting of the filter along with all its connections to filters in all preceding layers. An ensemble of such filter-trees created from many CNNs learnt on different but related tasks, forms the BFT. To learn a new CNN, we sample from the BFT to select a set of filter trees. This fixes the first few layers of the target net and only the remaining network would be learnt using training data of new task. Through simulations we demonstrate the effectiveness of this idea of BFT. This method constitutes a novel transfer learning technique where transfer is at a subnetwork level; transfer can be effected from multiple source networks, the number of weights to be learnt is same as a single CNN; and, with no finetuning of the transferred weights, the performance achieved is quite good. In all our experiments the number of filter trees sampled is kept same as the number of filters in the kth layer of the new CNN. This is not a limitation it is just to keep the number of filters freshly learnt in the subsequent layers equal to a single CNN for a fair comparison.

Predicting cascade volumes in social media communication is an important topic in furthering the use of social media for viral marketing, impact of political campaigns and in home-land security. Several techniques have been reported in the literature to estimate the cascade volumes. These algorithms use a variety of information such as Content, Structural and Temporal features, depending on their availability. Due to the spread of information infused into the algorithms the prediction accuracy has been shown in the literature to be different for different algorithms.

In this paper, we propose an image segmentation-based algorithm to perform upsampling of noisy low-resolution depth maps using information from the high-resolution color image. The depth map is initially upscaled using standard image interpolation technique, and then refined by a process based on the combination of Normalized Cuts segmentation and various smoothness priors in order to obtain a high quality highresolution depth map. Our method outperforms other existing methods when applied on both synthetic and real-world datasets.

The Euclidean Median (EM) of a set of points Ω in an Euclidean space is the point x minimizing the (weighted) sum of the Euclidean distances of x to the points in Ω. While there exits no closed-form expression for the EM, it can nevertheless be computed using iterative methods such as the Weiszfeld algorithm. The EM has classically been used as a robust estimator of centrality for multivariate data. It was recently demonstrated that the EM can be used to perform robust patch-based denoising of images by generalizing the popular Non-Local Means algorithm. In this paper, we propose a novel algorithm for computing the EM (and its box-constrained counterpart) using variable splitting and the method of augmented Lagrangian. The attractive feature of this approach is that the subproblems involved in the ADMM-based optimization of the augmented Lagrangian can be resolved using simple closed-form projections. The proposed ADMM solver is used for robust patch-based image denoising and is shown to exhibit faster convergence compared to an existing solver.

The utility of canonical correlation analysis (CCA) for domain adaptation (DA) in the context of multi-view head pose estimation is examined in this work. We consider the three problems studied in [1], where different DA approaches are explored to transfer head pose-related knowledge from an extensively labeled source dataset to a sparsely labeled target set, whose attributes are vastly different from the source. CCA is found to benefit DA for all the three problems, and the use of a covariance profile-based diagonality score (DS) also improves classification performance with respect to a nearest neighbor (NN) classifier.

An action is typically composed of different parts of the object moving in particular sequences. The presence of different motions (represented as a 1D histogram) has been used in the traditional bag-of-words (BoW) approach for recognizing actions. However the interactions among the motions also form a crucial part of an action. Different object-parts have varying degrees of interactions with the other parts during an action cycle. It is these interactions we want to quantify in order to bring in additional information about the actions. In this paper we propose a causality based approach for quantifying the interactions to aid action classification. Granger causality is used to compute the cause and effect relationships for pairs of motion trajectories of a video. A 2D histogram descriptor for the video is constructed using these pairwise measures. Our proposed method of obtaining pairwise measures for videos is also applicable for large datasets. We have conducted experiments on challenging action recognition databases such as HMDB51 and UCF50 and shown that our causality descriptor helps in encoding additional information regarding the actions and performs on par with the state-of-the art approaches. Due to the complementary nature, a further increase in performance can be observed by combining our approach with state-of-the-art approaches.

In recent years Auto stereoscopic display systems are being developed to give viewers a realistic 3D perception. Towards this end Multiview Plus Depth format is becoming increasingly popular. Using Depth Image Based Rendering algorithms additional views can be synthesized by the display device. In the past enhancement of Multiview Video Coding using depth image based 3D-warping has been explored, primarily to improve disparity estimation. Generally it is assumed that the camera array is static but this assumption is not valid for a realistic application. In this paper we propose a method to generate 3D-warped reference frames for temporally displaced frames. Temporal motion compensation is enhanced by tracking the camera motion. Our experiments show significant improvements in coding efficiency of motion vectors at similar PSNR values.

Real-time object tracking is a difficult task in unconstrained environment due to the variations in factors such as pose, size, illumination, partial occlusion and motion blur. In this paper, we propose a novel approach based on local sparse representation for robust object tracking to address the above issues. In the proposed approach, a search window, including the object and the surrounding information is used to create dictionary of overlapping patches. This context-based method is used to discriminate the confusing patches of the foreground with those in the background. Candidate patches are sparsely represented in the dictionary and foreground/background classification is done by computing the confidence map based on the distribution of sparse coefficients. Pyramidal structure of the object window that depicts object at different scales is used to create a dictionary that can handle scale changes. Object is localized by seeking the mode of the confidence map. A suitable dictionary update strategy is used to alleviate the drift problem during tracking. Numerous experiments on challenging videos demonstrate that the proposed tracker outperforms several state-of-the-art algorithms. The proposed approach tracks at a high processing speed and is suitable for real-time applications.

We propose a novel space-time descriptor for region-based tracking which is very concise and efficient. The regions represented by covariance matrices within a temporal fragment, are used to estimate this space-time descriptor which we call the Eigenprofiles(EP). EP so obtained is used in estimating the Covariance Matrix of features over spatio-temporal fragments. The Second Order Statistics of spatio-temporal fragments form our target model which can be adapted for variations across the video. The model being concise also allows the use of multiple spatially overlapping fragments to represent the target. We demonstrate good tracking results on very challenging datasets, shot under insufficient illumination conditions.

The problem of human detection is challenging, more so, when faced with adverse conditions such as occlusion and background clutter. This paper addresses the problem of human detection by representing an extracted feature of an image using a sparse linear combination of chosen dictionary atoms. The detection along with the scale finding, is done by using the coefficients obtained from sparse representation. This is of particular interest as we address the problem of scale using a scale-embedded dictionary where the conventional methods detect the object by running the detection window at all scales.

It is well known that extremely long low-density parity-check (LDPC) codes perform exceptionally well for error correction applications, short-length codes are preferable in practical applications. However, short-length LDPC codes suffer from performance degradation owing to graph-based impairments such as short cycles, trapping sets and stopping sets and so on in the bipartite graph of the LDPC matrix. In particular, performance degradation at moderate to high Eb/N0 is caused by the oscillations in bit node a posteriori probabilities induced by short cycles and trapping sets in bipartite graphs. In this study, a computationally efficient algorithm is proposed to improve the performance of short-length LDPC codes at moderate to high Eb/N0. This algorithm makes use of the information generated by the belief propagation (BP) algorithm in previous iterations before a decoding failure occurs. Using this information, a reliability-based estimation is performed on each bit node to supplement the BP algorithm. The proposed algorithm gives an appreciable coding gain as compared with BP decoding for LDPC codes of a code rate equal to or less than 1/2 rate coding. The coding gains are modest to significant in the case of optimised (for bipartite graph conditioning) regular LDPC codes, whereas the coding gains are huge in the case of unoptimised codes. Hence, this algorithm is useful for relaxing some stringent constraints on the graphical structure of the LDPC code and for developing hardware-friendly designs.

In this paper, we explore the use of LDPC codes for nonuniform sources under distributed source coding paradigm. Our analysis reveals that several capacity approaching LDPC codes indeed do approach the Slepian-Wolf bound for nonuniform sources as well. The Monte Carlo simulation results show that highly biased sources can be compressed to 0.049 bits/sample away from Slepian-Wolf bound for moderate block lengths.

Human Visual attention (HVA) is an important strategy to focus on specific information while observing and understanding visual stimuli. HVA involves making a series of fixations on select locations while performing tasks such as object recognition, scene understanding, etc. We present one of the first works that combines fixation information with automated concept detectors to (i) infer abstract image semantics, and (ii) enhance performance of object detectors.

Motivated from real world problems, like object categorization, we study a particular mixed-norm regularization for Multiple Kernel Learning (MKL). It is assumed that the given set of kernels are grouped into distinct components where each component is crucial for the learning task at hand. The formulation hence employs $l_\infty$ regularization for promoting combinations at the component level and $l_1$ regularization for promoting sparsity among kernels in each component. While previous attempts have formulated this as a non-convex problem, the formulation given here is an instance of non-smooth convex optimization problem which admits an efficient Mirror-Descent (MD) based procedure. The MD procedure optimizes over product of simplexes, which is not a well-studied case in literature. Results on real-world datasets show that the new MKL formulation is well-suited for object categorization tasks and that the MD based algorithm outperforms state-of-the-art MKL solvers like \texttt{simpleMKL} in terms of computational effort.

This article addresses the problem of how to select the optimal combination of sensors and how to determine their optimal placement in a surveillance region in order to meet the given performance requirements at a minimal cost for a multimedia surveillance system. We propose to solve this problem by obtaining a performance vector, with its elements representing the performances of subtasks, for a given input combination of sensors and their placement. Then we show that the optimal sensor selection problem can be converted into the form of Integer Linear Programming problem (ILP) by using a linear model for computing the optimal performance vector corresponding to a sensor combination. Optimal performance vector corresponding to a sensor combination refers to the performance vector corresponding to the optimal placement of a sensor combination. To demonstrate the utility of our technique, we design and build a surveillance system consisting of PTZ (Pan-Tilt-Zoom) cameras and active motion sensors for capturing faces. Finally, we show experimentally that optimal placement of sensors based on the design maximizes the system performance.

The mean shift algorithm has been proved to be efficient for tracking 2D blobs through a video sequence. Even so, this algorithm has certain inherent disadvantages. In this paper, we propose a robust tracking algorithm which overcomes the drawbacks of global color histogram based tracking. We incorporate tracking based only on reliable colors by separating the object from its background. A fast yet robust model update is employed to overcome illumination changes. This algorithm is computationally simple enough to be executed real time and was tested on several complex video sequences. The proposed technique could be easily extended to other tracking algorithms too.

Magnetic resonance imaging of dynamic events such as cognitive tasks in the brain, requires high spatial and temporal resolution. In order to increase the resolution in both domains simultaneously, parallel imaging schemes have been in existence, where multiple receiver coils are used, each of which needs to acquire only a fraction of the total available signal. In our approach, we regularly undersample dersample the signal at each of the receiver coils and the resulting aliased coil images are combined (unaliased) using the neural network framework. Data acquisition follows a variable-density sampling scheme, where lower frequencies are densely sampled, and the remaining signal is sparsely sampled. The low resolution images obtained using the densely sampled low frequencies are used to train the neural network. Reconstruction of the image is carried out by feeding the high-resolution aliased images to the trained network. The proposed approach has been applied to phantom as well as real brain MRI data sets, and results have been compared with the standard existing parallel imaging techniques. The proposed approach is found to perform better than the standard existing techniques.

This paper introduces kernels on attributed pointsets, which are sets of vectors embedded in an euclidean space. The embedding gives the notion of neighborhood, which is used to define positive semidefinite kernels on pointsets. Two novel kernels on neighborhoods are proposed, one evaluating the attribute similarity and the other evaluating shape similarity. Shape similarity function is motivated from spectral graph matching techniques. The kernels are tested on three real life applications: face recognition, photo album tagging, and shot annotation in video sequences, with encouraging results.

We propose a simple speech music discriminator that uses features based on HILN (harmonics, individual lines and noise) model. We have been able to test the strength of the feature set on a standard database of 66 files and get an accuracy of around 97%. We also have tested on sung queries and polyphonic music and have got very good results. The current algorithm is being used to discriminate between sung queries and played (using an instrument like flute) queries for a query by humming (QBH) system currently under development in the lab

This paper describes an audio retrieval system, Quebex that works on raw audio data. The system is able to retrieve songs that are rhythmically and timbrewise similar from a database of 700 songs. A new algorithm is proposed that attempts to match two songs temporally in terms of timbre, using a simple dynamic time warping algorithm. A sieving technique is used to eliminate unlikely candidates from further processing and a ranking system among the rhythm, spectral and temporal features is used to retrieve songs. Initial results have shown great promise for the approach as it retrieves remixes and dubbed songs within the top 10 retrievals

Feature track matrix factorization based methods have been attractive solutions to the structure-from-motion (Sfm) problem. Group motion of the feature points is analyzed to get the 3D information. It is well known that the factorization formulations give rise to rank deficient system of equations. Even when enough constraints exist, the extracted models are sparse due the unavailability of pixel level tracks. Pixel level tracking of 3D surfaces is a difficult problem, particularly when the surface has very little texture as in a human face. Only sparsely located feature points can be tracked and tracking errors are inevitable along rotating low texture surfaces. However, the 3D models of an object class lie in a subspace of the set of all possible 3D models. We propose a novel solution to the structure-from-motion problem which utilizes the high-resolution 3D obtained from range scanner to compute a basis for this desired subspace. Adding subspace constraints during factorization also facilitates removal of tracking noise which causes distortions outside the subspace. We demonstrate the effectiveness of our formulation by extracting dense 3D structure of a human face and comparing it with a well known structure-from-motion algorithm due to brand.

This paper addresses the problem of extracting video objects from MPEG compressed video. The only cues used for object segmentation are the motion vectors which are sparse in MPEG. A method for automatically estimating the number of objects and extracting independently moving video objects using motion vectors is presented here. First, the motion vectors are accumulated over a few frames to enhance the motion information, which are further spatially interpolated to get dense motion vectors. The final segmentation, using the dense motion vectors, is obtained by applying the expectation maximization (EM) algorithm. A block-based affine clustering method is proposed for determining the number of appropriate motion models to be used for the EM step and the segmented objects are temporally tracked to obtain the video objects. Finally, a strategy for edge refinement is proposed to extract the precise object boundaries. Illustrative examples are provided to demonstrate the efficacy of the approach. A prominent application of the proposed method is that of object-based coding, which is part of the MPEG-4 standard.

We propose a framework for orthonormal symmetric wavelet transform given a cyclic zero-phase half band filter. The scheme consists of a design for cyclic wavelet transform with real symmetric filters based on the perfect reconstruction circular convolution filter banks accompanied by an implementation in the discrete trigonometric transform domain. This is followed by a brief discussion on its performance in the context of applications such as image compression and implementation of symmetric wavelet transforms based on bandlimited wavelets.

In this paper we present a system for classifying various human actions in compressed domain video framework. We introduce the notion of quantifying the motion involved, through what we call "Motion Flow History" (MFH). The encoded motion information readily available in the compressed MPEG stream is used to construct the coarse Motion History Image (MHI) and the corresponding MFH. The features extracted from the static MHI and MFH compactly characterize the temporal and motion information of the action. Since the features are extracted from the partially decoded sparse motion data, the computational load is minimized to a great extent. The extracted features are used to train the KNN, Neural network and the Bayes classifiers for recognizing a set of seven human actions. Experimental results show that the proposed method efficiently recognizes the set of actions considered.

In this paper we present a system for classifying various human actions in compressed domain video framework. We introduce the notion of quantifying the motion involved, through what we call "motion flow history" (MFH). The encoded motion information readily available in the compressed MPEG stream is used to construct the coarse motion history image (MHI) and the corresponding MFH. The features extracted from the static MHI and MFH compactly characterize the temporal and motion information of the action. Since the features are extracted from the partially decoded sparse motion data, the computational load is minimized to a great extent. The extracted features are used to train the KNN, neural network, SVM and the Bayes classifiers for recognizing a set of seven human actions. Experimental results show that the proposed method efficiently recognizes the set of actions considered.

This paper addresses the problem of extracting video objects from MPEG compressed video. The only cues used for object segmentation are the motion vectors which are sparse in MPEG. A method for automatically estimating the number of objects and extracting independently moving video objects using motion vectors is presented here. First, the motion vectors are accumulated over few frames to enhance the motion information, which are further spatially interpolated to get a dense motion vectors. The final segmentation from the dense motion vectors is obtained by applying Expectation Maximization (EM) algorithm. A block based affine clustering method is proposed for determining the number of appropriate motion models to be used for the EM step. Finally, the segmented objects are temporally tracked to obtain the video objects. This work has been carried out in the context of the emerging MPEG-4 standard which aims at interactivity at object level.

We propose a new multiple description method for wavelet transform coded images. The partitioning of the coefficients at the encoder is combined with concealment at the decoder to produce descriptions. An optimal concealment method for wavelet coefficients is derived based on statistical characteristics. Concealment in the LL subband is treated differently and is based on its smoothness property. We have designed a coder for wireless image transmission which emphasizes the need for source and channel coding interaction based on residual redundancy. Optimal channel rate allocation is derived depending upon source significance and channel state information. Simulations for wireless transmission have shown promising results.

"Can we watermark without perturbing an image?" We present the salient results of an investigation carried out to find an answer to this question. We judge candidate watermarking schemes based on transparency, robustness, capacity and non-invertibility, together referred to as the governing factors. We refer to schemes that embed by perturbing an image as intrusive schemes, and those which claim to embed without perturbing an image as non-intrusive schemes. We discover that it is only to ensure non-invertibility that an image needs to be perturbed in order to be watermarked. We propose a watermarking framework designed such that any watermarking scheme which embeds by adhering to the rules of the framework carries with it the positive aspects of non-intrusive embedding and at the same time is non-invertible.

A technique to extract pitch information directly from audio files encoded using MPEG/Audio coding standard is described. The technique works in the compressed domain and requires the MPEG/Audio file to be decoded only partially for extracting the encoded subband samples. The paper first proposes a method for extracting wavelet coefficients from the subband samples. The pitch interval is then estimated from the time interval between two successive maxima of the wavelet coefficients. It is shown that the computational complexity for compressed domain pitch extraction is less than 7% of the computations required for decoding the subband samples and finding the pitch.

In this paper, we explore the convergence of the caching and streaming technologies for Internet multimedia. The paper describes a design for a streaming and caching architecture to be deployed on broadband networks. The basis of the work is the proposed Internet standard, Real Time Streaming Protocol (RTSP), likely to be the de-facto standard for web-based A/V caching and streaming, in the near future. The proxies are all managed by an `Intelligent Agent' or `Broker' - this has been designed as an enhanced RTSP proxy server that maintains the state information that is so essential in streaming of media data. In addition, all the caching algorithms run on the broker. Having an intelligent agent or broker ensures that the `simple' caching servers can be easily embedded into the network. However, RTSP does nor have the right model for doing broker based streaming/caching architecture. The work reported here is an attempt to contribute towards that end.

Nearest neighbor classifiers that use all the training samples for classification require large memory and demand large online testing computation. To reduce the memory requirements and the computation cost, many algorithms have been developed that perform nearest neighbor classification using only a small number of representative samples obtained from the training set. We call the classification model underlying all these algorithms as Voronoi networks (Vnets), because these algorithms discretize the feature space into Voronoi regions and assign the samples in each region to a class. In this paper we analyze the generalization capabilities of these networks by bounding the generalization error. The class of problems that can be "efficiently" solved by Vnets is characterized by the extent to which the set of points on the decision boundaries fill the feature space, thus quantifying how efficiently a problem can be solved using Vnets. We show that Vnets asymptotically converge to the Bayes classifier with arbitrarily high probability provided the number of representative samples grow slower than the square root of the number of training samples and also give the optimal growth rate of the number of representative samples. We redo the analysis for decision tree (DT) classifiers and compare them with Vnets. The bias/variance dilemma and the curse of dimensionality with respect to Vnets and DTs are also discussed.

What are the natural features of handwritten characters and how to arrive at them automatically? We apply independent components analysis on handwritten characters. Independent components analysis extracts the underlying statistically independent signals from a mixture of them. We expect strokes to be the independent components of handwritten characters. Our findings show that stroke-like features emerge as a result of the analysis confirming the above intuition. This finding is significant since it gives automatic procedures for extracting stroke-like features from multilingual character data sets. We use these features for handwritten digit recognition using a very simple classifier. The classifier is chosen to be simple so that the quality of the input feature set can be evaluated. The recognition results indicate that the features arrived at by independent component analysis are useful.

Digital watermarking is the technique in which a visible/invisible signal (watermark) is embedded in a multimedia document for copyright protection. In this paper, we propose a watermarking scheme called "dual watermarking". Dual watermark is a combination of a visible watermark and an invisible watermark.

A new technique for scene change detection using neural networks is proposed to operate on compressed video data obtained with only minimal decoding of video. In this paper various types of neural network structures have been trained and tested on different varieties of video clips. The neural networks are shown to be robust for camera activities and object movements. They gave comparatively good results over the conventional techniques. All the algorithms are tested on MPEG-1 video streams.

The use of audio to retrieve and index the associated video is a relatively new approach. In this paper the focus is on MPEG video. For indexing and retrieval one needs to segment the audio stream associated with the video in terms of gender, speech, music and the speaker. This is called "audio scene" analysis. The paper discusses techniques for such analysis in the MPEG audio compressed domain.

We propose a method to design an orthonormal two-channel cyclic filterbank (CFB) with real analysis and synthesis filters having linear phase. This is particularly significant, since it is not possible to realize regular two-channel linear phase orthogonal filterbanks (unless the filters are two-tap). The lowpass filter of the FB is obtained as a spectral factor of a zero phase Nyquist(2) filter, which is optimized. We show that the CFB design is computationally less demanding as compared to the noncyclic case, and it may be possible to get better filters. We design the CFB for a particular case and show that this filterbank is not a noncyclic orthogonal filterbank.

In a number of applications of computerized tomography, the ultimate goal is to detect and characterize objects within a cross section. Detection of edges of different contrast regions yields the required information. The problem of detecting edges from projection data is addressed. It is shown that the class of linear edge detection operators used on images can be used for detection of edges directly from projection data. This not only reduces the computational burden but also avoids the difficulties of postprocessing a reconstructed image. This is accomplished by a convolution backprojection operation. For example, with the Marr-Hildreth edge detection operator, the filtering function that is to be used on the projection data is the Radon transform of the Laplacian of the 2-D Gaussian function which is combined with the reconstruction filter. Simulation results showing the efficacy of the proposed method and a comparison with edges detected from the reconstructed image are presented.

The problem of image reconstruction from noisy projection data is considered. Adaptive noise canceling techniques are utilized to obtain noise-canceled projections in the data-preprocessing stage. Because noise encountered in computerized tomography is signal-dependent, an adaptive predictor is preferred to a fixed-parameter filter. Noise cancellation is achieved because the decorrelation time for broadband noise is less than that of narrowband signals. Both the tapped delay line structure and the lattice have been studied for implementing the adaptive linear predictor. The idea of forward-backward processing is used for circumventing the large error in the initial part of the noise-canceled projection data. The reconstruction is then completed from noise-filtered projection data using the convolution back projection method. Simulation results demonstrating the efficacy of the proposed method are presented.>

This paper presents a method of designing a programmable signal processor based on a bit parallel matrix vector matrix multiplier (linear transformer). The salient feature of this design is that the efficiency of the direct vector matrix multiplier is improved and VLSI design is made much simpler by trading off the more expensive arithematic operation (multiplication) for 'cheaper' manipulation (addition/subtraction) of the data.

A new technique for 2-D spectral estimation of a stationary random field (SRF) is investigated in this paper. This is based on the extension of the Radon transform theory to stationary random fields, proposed by Jain and Ansari [19]. Using Radon transform the 2-D estimation problem is reduced to a set of 1-D independent problems, which could then be solved using 1-D Linear Prediction (LP) or any other high resolution estimation procedure. This is unlike previous methods which obtains the 2-D Power Spectral Density (PSD) estimate by using 1-D high resolution techniques in the spirit of a separable estimator [2]. An example is provided to illustrate the performance of the new technique. Various features of this approach are highlighted.

Unambiguous reconstruction is not possible with truncated projections. In order to reduce the ambiguity in the reconstructed image, several authors have suggested that a simple 'completion' involving extrapolation of the truncated projection is sufficient. In this paper a method based on a Linear Prediction approach is proposed for obtaining the missing part in the projection. Reconstruction is then carried out using the Convolution Backprojection (CBP) method [1]. Simulation results showing the effectiveness of this method in extracting significantly more information from truncated projections are presented.