Condensed datasets offer a compact representation of larger datasets, but training models directly on them or using them to enhance model performance through knowledge distillation (KD) can result in suboptimal outcomes due to limited information. To address this, we propose a method that expands condensed datasets using model inversion, a technique for generating synthetic data based on the impressions of a pre-trained model on its training data. This approach is particularly well-suited for KD scenarios, as the teacher model is already pre-trained and retains knowledge of the original training data. By creating synthetic data that complements the condensed samples, we enrich the training set and better approximate the underlying data distribution, leading to improvements in student model accuracy during knowledge distillation. Our method demonstrates significant gains in KD accuracy compared to using condensed datasets alone and outperforms standard model inversion-based KD methods by up to 11.4% across various datasets and model architectures. Importantly, it remains effective even when using as few as one condensed sample per class, and can also enhance performance in few-shot scenarios where only limited real data samples are available.

Data-Free Knowledge Distillation (KD) allows knowledge transfer from a trained neural network (teacher) to a more compact one (student) in the absence of original training data. Existing works use a validation set to monitor the accuracy of the student over real data and report the highest performance throughout the entire process. However, validation data may not be available at distillation time either, making it infeasible to record the student snapshot that achieved the peak accuracy. Therefore, a practical data-free KD method should be robust and ideally provide monotonically increasing student accuracy during distillation. This is challenging because the student experiences knowledge degradation due to the distribution shift of the synthetic data. A straightforward approach to overcome this issue is to store and rehearse the generated samples periodically, which increases the memory footprint and creates privacy concerns. We propose to model the distribution of the previously observed synthetic samples with a generative network. In particular, we design a Variational Autoencoder (VAE) with a training objective that is customized to learn the synthetic data representations optimally. The student is rehearsed by the generative pseudo replay technique, with samples produced by the VAE. Hence knowledge degradation can be prevented without storing any samples. Experiments on image classification benchmarks show that our method optimizes the expected value of the distilled model accuracy while eliminating the large memory overhead incurred by the sample-storing methods.

With the increasing popularity of deep learning on edge devices, compressing large neural networks to meet the hardware requirements of resource-constrained devices became a significant research direction. Numerous compression methodologies are currently being used to reduce the memory sizes and energy consumption of neural networks. Knowledge distillation (KD) is among such methodologies and it functions by using data samples to transfer the knowledge captured by a large model (teacher) to a smaller one (student). However, due to various reasons, the original training data might not be accessible at the compression stage. Therefore, data-free model compression is an ongoing research problem that has been addressed by various works. In this paper, we point out that catastrophic forgetting is a problem that can potentially be observed in existing data-free distillation methods. Moreover, the sample generation strategies in some of these methods could result in a mismatch between the synthetic and real data distributions. To prevent such problems, we propose a data-free KD framework that maintains a dynamic collection of generated samples over time. Additionally, we add the constraint of matching the real data distribution in sample generation strategies that target maximum information gain. Our experiments demonstrate that we can improve the accuracy of the student models obtained via KD when compared with state-of-the-art approaches on the SVHN, Fashion MNIST and CIFAR100 datasets.

Traditional image stitching algorithms use transforms such as homography to combine different views of a scene. They usually work well when the scene is planar or when the camera is only rotated, keeping its position static. This severely limits their use in real world scenarios where an unmanned aerial vehicle (UAV) potentially hovers around and flies in an enclosed area while rotating to capture a video sequence. We utilize known scene geometry along with recorded camera trajectory to create cylindrical images captured in a given environment such as a tunnel where the camera rotates around its center. The captured images of the inner surface of the given scene are combined to create a composite panoramic image that is textured onto a 3D geometrical object in Unity graphical engine to create an immersive environment for end users.

Local structures of shadow boundaries as well as complex interactions of image regions remain largely unexploited by previous shadow detection approaches. In this paper, we present a novel learning-based framework for shadow region recovery from a single image. We exploit local structures of shadow edges by using a structured CNN learning framework. We show that using structured label information in classification can improve local consistency over pixel labels and avoid spurious labelling. We further propose and formulate shadow/bright measure to model complex interactions among image regions. The shadow and bright measures of each patch are computed from the shadow edges detected by the proposed CNN. Using the global interaction constraints on patches, we formulate a least-square optimization problem for shadow recovery that can be solved efficiently. Our shadow recovery method achieves state-of-the-art results on major shadow benchmark databases collected under various conditions.

Model-free object tracking is still challenging because of the limited prior knowledge and the unexpected variation of the target object. In this paper, we propose a feature learning algorithm for model-free multiple object tracking. First, we pre-learn generic features invariant to diverse motion transformations from auxiliary video data by using a deep network of anto-encoder. Then, we adapt the pre-learned features according to multiple target objects respectively in a multi-task learning manner. We treat the feature adaptation for each target object as one single task. We simultaneously learn the common feature shared by all target objects and the individual feature of each object. Experimental results demonstrate that our feature learning algorithm can significantly improve multiple object tracking performance.

In many intelligent surveillance systems there is a requirement to search for people of interest through archived semantic labels. Other than searching through typical appearance attributes such as clothing color and body height, information such as whether a person carries a bag or not is valuable to provide more relevant targeted search. We propose two novel and fast algorithms for sling bag and backpack detection based on the geometrical properties of bags. The advantage of the proposed algorithms is that it does not require shape information from human silhouettes therefore it can work under crowded condition. In addition, the absence of background subtraction makes the algorithms suitable for mobile platforms such as robots. The system was tested with a low resolution surveillance video dataset. Experimental results demonstrate that our method is promising.

Background subtraction is an essential technique for automatic video analysis. The main idea is to construct and update the model of the background scene. Foreground pixels are detected if they deviate from the background model to a certain extent. The model can consist of color, texture and gradient information [1]. In this paper, we focus on both color and texture information. The proposed texture feature is based on local binary pattern (LBP), while the color feature is represented by local color pattern (LCP). LBP is known to work well on texture rich regions and is invariant to subtle illumination variations, but it is inefficient on uniform regions. In view of this, color information can be incorporated to complement the texture feature. On the other hand, when the scene contrast or video quality is poor, color information may be unreliable and should be assigned lower priority than texture information. We propose a fuzzy rule-based system that adaptively adjusts the weights of the texture and color features based on the pixel's local properties. Experimental results on real scenes demonstrate the robustness of the proposed method.

Texture and color are two primitive forms of features that can be used to describe a scene. While conventional local binary pattern (LBP) texture based background subtraction performs well on texture rich regions, it fails to detect uniform foreground objects in large uniform background. As such, color information can be used to complement texture feature. In this study, we propose to incorporate local color feature based on Improved Hue, Luminance, and Saturation (IHLS) color space and introduce an adaptive scheme that automatically adjusts the weight between texture and color similarities based on the pixel's local properties: texture uniformity and color saturation. Experiments on eight challenging sequences demonstrate the effectiveness of the proposed method compared to the state-of-the-art algorithms.

Shape and motion are two most distinct cues observed from human actions. Traditionally, K-Nearest Neighbor (K-NN) classifier is used to compute crisp votes from multiple cues separately. The votes are then combined using linear weighting scheme. Usually, the weights are determined in a brute-force or trial-and-error manner. In this study, we propose a new classification framework based on sum-rule fusion of fuzzy K NN classifiers. Fuzzy K-NN classifier is capable of producing soft votes, also known as fuzzy membership values. Based on Bayes theorem, we show that the fuzzy membership values produced by the classifiers can be combined using sum-rule. In our experiment, the proposed framework consistently outperforms the conventional counterpart (K-NN with majority voting) for both Weizmann and KTH datasets. The improvement may attribute to the ability of the proposed framework to handle data ambiguity due to similar poses present in different action classes. We also show that the performance of our method compares favorably with the state-of-the-arts.

Most methods for multiple camera tracking rely on accurate calibration to associate data from multiple cameras. However, it often is not easy to have an accurate calibration in some real applications due to practical reasons. The inaccurate calibration can lead to wrong data association of objects between cameras. In this paper, we propose a method to handle the data association of objects in multiple cameras under inaccurate ground plane homography by using the RFS Bayes filter. Our method is based on modeling measurements from cameras to a random finite set. This random finite set includes the primary measurement from the object, extraneous measurements of the object, and clutter. Experimental results show the efficiency and robustness of our method through challenging cases such as occlusions, merged and split persons.

This paper presents a new keypoint-based approach to near-duplicate images detection. It consists of three steps. Firstly, the keypoints of images are extracted and then matched. Secondly, the matched keypoints are voted for estimation of affine transform based on an affine invariant ratio of normalized lengths. Finally, to further confirm the matching, the color histograms of areas formed by matched keypoints in two images are compared. This method has the advantage for handling the case when there are only a few matched keypoints. The proposed algorithm has been tested on Columbia dataset and conducted the quantitative comparison with RANdom SAmple Consensus (RANSAC) algorithm and Scale-Rotation Invariant Pattern Entropy (SR-PE) algorithm. The experiment result turns out that the proposed method compares favorably against the state-of-the-arts.

In practical surveillance applications, methods for multiple camera tracking have to deal with many challenges such as noises and errors in estimating system parameters. In this pa per, we propose a distributed multiple camera tracking system that can track multiple persons in real time under challenging conditions such as inaccuracy from calibration, noises from background and lighting changes. The system combines a network of cameras located around the environment, a central process unit and mobile devices. Experimental results show that our system has already performed well in military training.

Merging and splitting of objects cause challenges for visual tracking. This is due to observation ambiguity, object lost, and tracking errors when objects are close together. In this paper, we propose a method to combine the joint probabilistic data association (JPDA) and the particle filter to maintain tracks of objects. The results of JPDA are employed to improve the observation model in the particle filter. Based on the ability of handling missing detections and clutter of JPDA, tracks of objects can be maintained after merging or splitting. Conversely, the particle filter also improves the performance of JPDA by fusing other observations such as color and background subtraction information. Hence, our method can take advantages from both JPDA and particle filter to track objects through merging and splitting.

In this paper we focus on improving the optimized content-aware authentication scheme over lossy network and incorporating this scheme into secure scalable streaming (SSS) and secure transcoding with JPEG-2000 images. SSS encodes, encrypts and packetizes image into secure scalable packets in a manner that allows downstream transcoders to perform transcoding operations by simply truncating or discarding packets without decrypting the data. However SSS does not consider the issues of packet loss and stream authentication while the stream is transmitted over today's public and lossy networks. This paper proposes an improved optimized content-aware authentication scheme and implements a system to integrate it with SSS seamlessly. Experimental results demonstrate that the proposed scheme achieved our design goal in that the R-D curve of the authenticated image is very close to its original one where no authentication is applied.

A method to extract views of different orientations of a vehicle captured using multi cameras on roadside is proposed. We expect the use of multi views would increase classification performance in tasks such as identifying vehicle types/makes. This paper does not discuss classification work in details; it accepts the concept that with more data obtained through multi camera views, the use of distinctive orientations only would improve classifier's performance. Prior to this, we have to resolve practical issues such as identifying condition of vehicle merges and shadow. We use correlated data from multi cameras to find the most optimized cut for a merge situation. We also propose a novel approach of removing vehicle's shadow using blob reconstruction technique. Views of different vehicle orientations (in our experiment, left, rear, right) are interpreted using a 3D graph fitting on images from multi cameras.

This paper describes the design and implementation of autonomous real-time motion recognition on a Personal Digital Assistant. All previous such applications have been non real-time and required user interaction. The motivation to use a PDA is to test the viability of performing complex video processing on an embedded platform. The application was constructed using a representation and recognition technique for identifying patterns using Hu Moments. The approach is based upon temporal templates (Motion Energy and History Images) and their matching in time. The implementation was done using Intel Integrated Performance Primitives functions in order to reduce the complexity of the application. Tests were conducted using 5 different motion actions like arm waving, walking from left and right of the camera, head tilting and bending forward. Suggestions were also made on how to improve the performance of the system and possible applications.

One of the important features for human machine interaction is its ability to recognize human faces. This paper presents a novel architecture suitable for real time robotic face recognition by learning a person's face incrementally, where the Gabor features at respective feature locations of a face are used to derive a similarity measurement. A face tracking followed by a clustering technique is used to learn a person's face appearance variance when the system interacts with the person. The recognition by learning proposed in this paper is similar to the partial memory incremental learning method, where we proposed a novel approach to the learning and updating process. Experiment shows significant improvement in the face recognition performance after learning over the time and with more interaction between a person and the system.

This paper proposes a novel method for tracking persons based on the principal colors of human objects. First, an efficient human object representation method, principal color representation (PCR), is proposed. Asymmetric similarity measures are then proposed based on the principal color representation. These asymmetric similarity measures could be used to evaluate the matching between two individuals as well as visual evident of an individual in a group. An efficient algorithm for tracking persons as individuals or in groups is then described. The method has been tested using image sequences containing multiple moving persons frequently gathering and separating. Our test results have shown that proposed method has successfully tracked both persons as individuals or in groups, and is robust to illumination changes.