Out-of-Distribution (OOD) detection is essential for the trustworthiness of AI systems. Methods using prior information (i.e., subspace-based methods) have shown effective performance by extracting information geometry to detect OOD data with a more appropriate distance metric. However, these methods fail to address the geometry distorted by ill-distributed samples, due to the limitation of statically extracting information geometry from the training distribution. In this paper, we argue that the influence of ill-distributed samples can be corrected by dynamically adjusting the prior geometry in response to new data. Based on this insight, we propose a novel approach that dynamically updates the prior covariance matrix using real-time input features, refining its information. Specifically, we reduce the covariance along the direction of real-time input features and constrain adjustments to the residual space, thus preserving essential data characteristics and avoiding effects on unintended directions in the principal space. We evaluate our method on two pre-trained models for the CIFAR dataset and five pre-trained models for ImageNet-1k, including the self-supervised DINO model. Extensive experiments demonstrate that our approach significantly enhances OOD detection across various models. The code is released at https://github.com/workerbcd/ooddcc.

The generalization ability of deep learning has been extensively studied in supervised settings, yet it remains less explored in unsupervised scenarios. Recently, the Unsupervised Domain Generalization (UDG) task has been proposed to enhance the generalization of models trained with prevalent unsupervised learning techniques, such as Self-Supervised Learning (SSL). UDG confronts the challenge of distinguishing semantics from variations without category labels. Although some recent methods have employed domain labels to tackle this issue, such domain labels are often unavailable in real-world contexts. In this paper, we address these limitations by formalizing UDG as the task of learning a Minimal Sufficient Semantic Representation: a representation that (i) preserves all semantic information shared across augmented views (sufficiency), and (ii) maximally removes information irrelevant to semantics (minimality). We theoretically ground these objectives from the perspective of information theory, demonstrating that optimizing representations to achieve sufficiency and minimality directly reduces out-of-distribution risk. Practically, we implement this optimization through Minimal-Sufficient UDG (MS-UDG), a learnable model by integrating (a) an InfoNCE-based objective to achieve sufficiency; (b) two complementary components to promote minimality: a novel semantic-variation disentanglement loss and a reconstruction-based mechanism for capturing adequate variation. Empirically, MS-UDG sets a new state-of-the-art on popular unsupervised domain-generalization benchmarks, consistently outperforming existing SSL and UDG methods, without category or domain labels during representation learning.

Deep learning models often suffer from catastrophic forgetting when fine-tuned with samples of new classes. This issue becomes even more challenging when there is a domain shift between training and testing data. In this paper, we address the critical yet less explored Domain-Generalized Class-Incremental Learning (DGCIL) task. We propose a DGCIL approach designed to memorize old classes, adapt to new classes, and reliably classify objects from unseen domains. Specifically, our loss formulation maintains classification boundaries while suppressing domain-specific information for each class. Without storing old exemplars, we employ knowledge distillation and estimate the drift of old class prototypes as incremental training progresses. Our prototype representations are based on multivariate Normal distributions , with means and covariances continually adapted to reflect evolving model features, providing effective representations for old classes. We then sample pseudo-features for these old classes from the adapted Normal distributions using Cholesky decomposition . Unlike previous pseudo-feature sampling strategies that rely solely on average mean prototypes, our method captures richer semantic variations. Experiments on several benchmarks demonstrate the superior performance of our method compared to the state of the art.

Current medical image synthetic augmentation techniques rely on the intensive use of generative adversarial networks (GANs). However, the nature of GAN architecture leads to heavy computational resources to produce synthetic images and the augmentation process requires multiple stages to complete. To address these challenges, we introduce a novel generative meta curriculum learning method that trains the task-specific model (student) end-to-end with only one additional teacher model. The teacher learns to generate curriculum to feed into the student model for data augmentation and guides the student to improve performance in a meta-learning style. In contrast to the generator and discriminator in GAN, which compete with each other, the teacher and student collaborate to improve the student's performance on the target tasks. Extensive experiments on the histopathology datasets show that leveraging our framework results in significant and consistent improvements in classification performance.

The identification of bacteria species is an important aspect of microbiology, but it can be expensive, laborious and time-consuming. Using computer-aided classification can signifi-cantly reduce the cost and time of diagnosing bacteria subtypes. However, commonly used image recognition models are close-set classification models, and they assume full knowledge of the world. That is, all testing classes are known at training time. Such an assumption cannot be always valid in real-world applications. Open-set recognition models have the ability to both classify known instances and detect unknown samples of novel classes. This work aims to tackle the problem of bacteria subtyping from an open-set perspective. Our framework, OpenGram, combines a convolutional neural network classifier with a Gaussian mixtures model to adapt to open-set classification. The results demonstrate OpenGram's ability to correctly detect unknown bacteria classes that were unseen by the network during training as well as classify known bacteria classes. Our experiments showed that OpenGram was capable of accurately classifying bacteria subtypes under open-set setting at up to 99.33% F1 score and 99.70% AUC.

The Gram stain test is one of the most commonly used procedures in microbiology labs as the first step to diagnose infection. Automated interpretation of such a test can be very beneficial, but the difficulty of obtaining fully annotated Gram stain Whole Slide Images (WSIs) datasets has hindered research in the area. We aim to tackle this lack of instance-level cell labels in Gram stain WSIs for the epithelial and leukocyte cell counting task. In this paper, we demonstrate that it is possible to train a cell detector on a totally unlabelled large dataset using some labels from a different domain. We present, HybridGram, a framework with image translation and pseudo labelling modules to completely avoid manual labelling on a new dataset. We conducted our experiments using our SNPGram21 dataset which comprises 100 Gram stain WSIs from different specimen sites. The results show that HybridGram can bridge the performance gap between fully supervised and unsupervised models. Furthermore, we investigate how well our pseudo-labelling module performs in comparison to manual labeling within the same domain. The results indicate that pseudo labelling can automatically label large quantities of Gram stain WSIs with minimum accuracy loss compared to expert manual labelling, thereby vastly reducing manual annotation effort.

Deep learning networks have demonstrated competitive performance for various tasks on medical images. However, obtaining promising results requires a large amount of annotated data for supervised training, which is labor-intensive. Recently, the increasing interest in transformers has suggested their robust performance on computer vision tasks, including generative adversarial networks (GANs). In this paper, we propose a conditional GAN built on pure transformer-based architectures, named MedViTGAN, to assist in generating synthetic histopathology images for data augmentation in an end-to-end manner. The presented model adopts a conditioned training strategy by incorporating a transformer-based auxiliary classifier to facilitate the discriminative image generation process. We further introduce an adaptive hybrid loss weighting mechanism to balance multiple losses over sources and classes to stabilize the training. Extensive experiments on the histopathology datasets show that leveraging MedViTGAN generated images results in a significant and consistent improvement in classification performance.

Nucleus instance segmentation and classification in histopathological images is an essential prerequisite in pathology diagnosis/prognosis. However, nucleus annotations (e.g., segmentation and labeling) require domain experts, and annotating nuclei at pixel-level is time-consuming and labor-intensive. Moreover, nuclei from different cancer types vary in shapes and appearances. These inter-cancer variations require careful annotations for specific cancer types. Therefore, to minimize the labeling cost, we propose a novel application that considers each cancer type as an individual domain and apply domain adaptation techniques to improve the segmentation/classification performance among different cancer types. Unlike the previous studies that focus on unsupervised or weakly-supervised domain adaptation independently, we would like to discover what kinds of labeling can achieve the most cost-effective domain adaptation performance in nucleus instance segmentation and classification. Specifically, we propose a unified framework that is applicable to different level annotations: no annotations, image-level, and point-level annotations. Cyclic adaptation with pseudo labels and adversarial discriminator are utilized for unsupervised domain alignment. Image-level or point-level annotations are additionally adopted to supervise the nucleus classification and refine the pseudo labels. Experiments demonstrate the effectiveness and efficacy of the proposed framework (jointly using unsupervised and weakly supervised learning) on adapting the segmentation and classification model from one cancer type to 18 other cancer types.

Text-to-Face (TTF) synthesis is a challenging task with great potential for diverse computer vision applications. Compared to Text-to-Image (TTI) synthesis tasks, the textual description of faces can be much more complicated and detailed due to the variety of facial attributes and the parsing of high dimensional abstract natural language. In this paper, we propose a Text-to-Face model that not only produces images in high resolution (1024×1024) with text-to-image consistency, but also outputs multiple diverse faces to cover a wide range of unspecified facial features in a natural way. By fine-tuning the multi-label classifier and im age encoder, our model obtains the adjustment vectors and image embeddings which are used to transform the input noise vector sampled from the normal distribution. Afterwards, the transformed noise vector is fed into a pre-trained high-resolution image generator to produce a set of faces with the desired facial attributes. We refer to our model as TTF-HD. Experimental results show that TTF-HD generates high-quality synthesised faces from free-form text descriptions with state-of-the-art performance.

The difficulty of processing gigapixel whole slide images (WSIs) in clinical microscopy has been a long-standing barrier to implementing computer aided diagnostic systems. Since modern computing resources are unable to perform computations at this extremely large scale, current state of the art methods utilize patch-based processing to preserve the resolution of WSIs. However, these methods are often resource intensive and make significant compromises on processing time. In this paper, we demonstrate that conventional patch-based processing is redundant for certain WSI classification tasks where high resolution is only required in a minority of cases. This reflects what is observed in clinical practice; where a pathologist may screen slides using a low power objective and only switch to a high power in cases where they are uncertain about their findings. To eliminate these redundancies, we propose a method for the selective use of high resolution processing based on the confidence of predictions on downscaled WSIs --- we call this the Selective Objective Switch (SOS). Our method is validated on a novel dataset of 684 Liver-Kidney-Stomach immunofluorescence WSIs routinely used in the investigation of autoimmune liver disease. By limiting high resolution processing to cases which cannot be classified confidently at low resolution, we maintain the accuracy of patch-level analysis whilst reducing the inference time by a factor of 7.74.

This work addresses the pose estimation task on low-resolution images captured using thermal sensors which can operate in a no-light environment. Low-resolution thermal sensors have been widely adopted in various applications for cost control and privacy protection purposes. In this paper, targeting the challenging scenario of ultra-low resolution thermal imaging (32×32 pixels), we aim to estimate human poses for the purpose of monitoring health conditions and indoor events. To overcome the challenges in ultra-low resolution thermal imaging such as blurred boundaries and data scarcity, we propose a new Image-to-Image (I2I) translation architecture which can translate the original blurred thermal image into a visible light image with sharper boundaries. Then the generated visible light image can be fed into the off-the-shelf pose estimator which was well-trained in the visible domain. Experimental results suggest that the proposed framework outperforms other state-of-the-art methods in the I2I based pose estimation task for our thermal image dataset. Furthermore, we also demonstrated the merits of the proposed method on the publicly available FLIR dataset by measuring the quality of translated images.

Electrical distribution poles are important assets in electricity supply. These poles need to be maintained in good condition to ensure they protect community safety, maintain reliability of supply, and meet legislative obligations. However, maintaining such a large volumes of assets is an expensive and challenging task. To address this, recent approaches utilise imagery data captured from helicopter and/or drone inspections. Whilst reducing the cost for manual inspection, manual analysis on each image is still required. As such, several image-based automated inspection systems have been proposed. In this paper, we target two major challenges: tiny object detection and extremely imbalanced datasets, which currently hinder the wide deployment of the automatic inspection. We propose a novel two-stage zoom-in detection method to gradually focus on the object of interest. To address the imbalanced dataset problem, we propose the resampling as well as reweighting schemes to iteratively adapt the model to the large intra-class variation of major class and balance the contributions to the loss from each class. Finally, we integrate these components together and devise a novel automatic inspection framework. Extensive experiments demonstrate that our proposed approaches are effective and can boost the performance compared to the baseline methods.

The image-to-image translation task often associates with the problem of missing texture and edge information. In this paper, we proposed a framework to translate images while preserving more realistic textures and details. To this end, we disentangle the samples into shared content space and domain-specific style domain. Then, according to the blurred outlines and textures in the source domain, we introduce the classic canny edge detection algorithm to encode the boundary and edge information in the content latent space. We test the proposed method in the thermal to visible image translation scenario and the experimental results demonstrate that the proposed method outperforms several other state-of-the-art models.

This work tackles the automatic fine-grained slide quality assessment problem for digitized direct smears test using the Gram staining protocol. Automatic quality assessment can provide useful information for the pathologists and the whole digital pathology workflow. For instance, if the system found a slide to have a low staining quality, it could send a request to the automatic slide preparation system to remake the slide. If the system detects severe damage in the slides, it could notify the experts that manual microscope reading may be required. In order to address the quality assessment problem, we propose a deep neural network based framework to automatically assess the slide quality in a semantic way. Specifically, the first step of our framework is to perform dense fine-grained region classification on the whole slide and calculate the region histogram of the label distributions. Next, our framework will generate assessments of the slide quality from various perspectives: staining quality, information density, damage level and which regions are more valuable for subsequent high-magnification analysis. To make the information more accessible, we present our results in the form of a heat map and text summaries. Additionally, in order to stimulate research in this direction, we propose a novel dataset for slide quality assessment. Experiments show that the proposed framework outperforms recent related works.

Utilizing multiple descriptions/views of an object is often useful in image clustering tasks. Despite many works that have been proposed to effectively cluster multi-view data, there are still unaddressed problems such as the errors introduced by the traditional spectral-based clustering methods due to the two disjoint stages: 1) eigendecomposition and 2) the discretization of new representations. In this paper, we propose a unified clustering framework which jointly learns the two stages together as well as utilizing multiple descriptions of the data. More specifically, two learning methods from this framework are proposed: 1) through a graph construction from different views and 2) through combining multiple graphs. Furthermore, benefiting from the separability and local graph preserving properties of the proposed methods, a novel unsupervised automatic attribute discovery method is proposed. We validate the efficacy of our methods on five data sets, showing that the proposed joint learning clustering methods outperform the recent state-of-the-art methods. We also show that it is possible to derive a novel method to address the unsupervised automatic attribute discovery tasks.

In this paper, we extend the nearest convex hull classifier to Symmetric Positive Definite (SPD) manifolds. SPD manifold features have been shown to have excellent performance in various image/video classification tasks. Unfortunately, SPD manifolds naturally possess non-Euclidean geometry, so existing Euclidean machineries such as the nearest convex hull classifier cannot be used directly. To that end, we propose a novel mathematical framework, named Manifold Convex Hull (MACH), that extends the nearest convex hull classifier to SPD Manifolds. The superior performance of our nearest convex hull framework on SPD manifolds is demonstrated in several computer vision applications including object recognition, pedestrian detection and texture classification.

Can we predict the winner of Miss Universe after watching how they stride down the catwalk during the evening gown competition? Fashion gurus say they can! In our work, we study this question from the perspective of computer vision. In particular, we want to understand whether existing computer vision approaches can be used to automatically extract the qualities exhibited by the Miss Universe winners during their catwalk. This study can pave the way towards new vision-based applications for the fashion industry. To this end, we propose a novel video dataset, called the Miss Universe dataset, comprising 10 years of the evening gown competition selected between 1996-2010. We further propose two ranking-related problems: (1) Miss Universe Listwise Ranking and (2) Miss Universe Pairwise Ranking. In addition, we also develop an approach that simultaneously addresses the two proposed problems. To describe the videos we employ the recently proposed Stacked Fisher Vectors in conjunction with robust local spatio-temporal features. From our evaluation we found that although the addressed problems are extremely challenging, the proposed system is able to rank the winner in the top 3 best predicted scores for 5 out of 10 Miss Universe competitions.

Recently, various automated attribute discovery methods have been developed to discover useful visual attributes from a given set of images. Despite the progress made, most methods consider the supervised scenario which assumes the existence of labelled data. Recent results suggest that it is possible to discover attributes from a set of unlabelled data. In this work, we propose a novel unsupervised attribute discovery method utilising multi-graph approach that preserves both local neighbourhood structure as well as class separability. Whilst, the local neighbourhood structure is preserved by considering multiple similarity graphs, the class separability is achieved by incorporating the traditional clustering objective. For evaluation, we first investigate the performance of the proposed approach to address a clustering task. Then we apply our proposed method to automatically discover visual attributes and compare with various automatic attribute discovery and hashing methods. The results show that our proposed method is able to improve the performance in the clustering task. Furthermore, when evaluated using the recent meaningfulness metric, the proposed method outperforms the other unsupervised attribute discovery methods.

This contest is a joint initiative organized by the University of Salerno (Italy) and the University of Queensland (Australia) with the support of the Sullivan Nicolaides Pathology (SNP), Australia. The contest primarily aims to provide a platform for scientists and practitioners for performing research to develop Computer Aided Diagnosis (CAD) systems for pathology tests utilizing indirect immunofluorescence protocol. In particular, the contest considers the Antinuclear Antibodies (ANA) test using Human Epithelial type 2 (HEp-2) cells. The competition is divided into four tasks that address specific problems: (1) HEp-2 cell classification; (2) Patient specimen classification; (3) HEp-2 mitotic cell identification and (4) Cell segmentation.

We propose a novel supervised initialization scheme for cascaded face alignment by searching nearest neighbors based on global image descriptors. Unlike existing schemes which resort to additional large training data sets for learning features, our method does not require additional training steps; thus making our method low computational. Moreover, we found that it is sufficient to use a simple low-dimensional global image descriptor that is easy to extract. In particular, in this work we use the GIST features as our global image descriptor. The proposed initialization scheme outperforms existing initialization schemes for face alignment and improves on the state-of-the-art methods on two challenging datasets, 300-W and COFW.

Automatically recognising facial emotions has drawn increasing attention in computer vision. Facial landmark based methods are one of the most widely used approaches to perform this task. However, these approaches do not provide good performance. Thus, researchers usually tend to combine more information such as textural and audio information to increase the recognition rate. In this paper we propose a novel method, here called the landmark manifold, that shows the possibility to achieve competitive performance by facial landmark information alone. Through experiments on the well-known dataset: marked Cohn-Kanade extended facial emotion dataset (CK+), we show that with accurate facial landmarks, our simple approach is fast to run and can achieve competitive performance with enormously expensive methods.

Many automatic attribute discovery methods have been developed to extract a set of visual attributes from images for various tasks. However, despite good performance in some image classification tasks, it is difficult to evaluate whether these methods discover meaningful attributes and which one is the best to find the attributes for image descriptions. An intuitive way to evaluate this is to manually verify whether consistent identifiable visual concepts exist to distinguish between positive and negative images of an attribute. This manual checking is tedious, labor intensive and expensive and it is very hard to get quantitative comparisons between different methods. In this work, we tackle this problem by proposing an attribute meaningfulness metric, that can perform automatic evaluation on the meaningfulness of attribute sets as well as achieving quantitative comparisons. We apply our proposed metric to recent automatic attribute discovery methods and popular hashing methods on three attribute datasets. A user study is also conducted to validate the effectiveness of the metric. In our evaluation, we gleaned some insights that could be beneficial in developing automatic attribute discovery methods to generate meaningful attributes. To the best of our knowledge, this is the first work to quantitatively measure the semantic content of automatically discovered attributes.

Recent progress in video description has shown promising results by combining object/action recognition and natural language processing techniques. However, even the most simplest form of the generated sentence, the SVO triplet (Subject/Verb/Object), can be misleading for its lack of role relationship analysis. When the system detects keywords "person", "baby" and "feed", we do not want the system to generate "a person feeding a baby" when the actual screen is a scene where the baby is trying to share the food. In this paper, we explore role relationships between objects/persons and their usage in generating a more meaningful video description. More specifically, we confine ourselves on the following problem: identifying subject and object roles in two-person activities. We argue that the subject and object roles have consistent properties across different activities. To that end, we cast this problem as a domain adaptation problem. A novel Youtube SVO dataset is proposed for evaluating methods developed for this problem. The performance of the proposed method is compared against several baseline methods.

This paper introduces sparse coding and dictionary learning for symmetric positive definite (SPD) matrices, which are often used in machine learning, computer vision, and related areas. Unlike traditional sparse coding schemes that work in vector spaces, in this paper, we discuss how SPD matrices can be described by sparse combination of dictionary atoms, where the atoms are also SPD matrices. We propose to seek sparse coding by embedding the space of SPD matrices into the Hilbert spaces through two types of the Bregman matrix divergences. This not only leads to an efficient way of performing sparse coding but also an online and iterative scheme for dictionary learning. We apply the proposed methods to several computer vision tasks where images are represented by region covariance matrices. Our proposed algorithms outperform state-of-the-art methods on a wide range of classification tasks, including face recognition, action recognition, material classification, and texture categorization.

Microscope examination of Gram stained clinical specimens is used for aiding the diagnosis of patients with infectious diseases. In high volume pathology laboratories, this manual microscopy examination is considered time consuming and labour intensive. Unfortunately, despite the great benefits offered from the application of Computer Aided Diagnosis (CAD) systems, to our knowledge, the highest automation stage for Gram stained slide analysis is only at the pre-analytical process. This paper takes the first steps towards the application of computer vision to direct smear, Gram stained images. To that end, we present a novel Gram stain image dataset. In addition, we also propose a multiple covariance approach for leukocyte and epithelial cell detection in Gram stain images. Each covariance matrix represents a particular image region characterising the cell's deformed structure. As covariance matrices form points on an Symmetric Positive Definite (SPD) manifold, the traditional Euclidean-based analysis cannot be used. As such, we first map the manifold points into the Reproducing Kernel Hilbert Space (RKHS). The analysis is done via a novel kernel similarity function that allows comparison between sets of covariance matrices. The proposed approach is contrasted, in the proposed dataset, with two recent state of the art methods in pedestrian detection: Histogram Of Gradient (HOG) and the traditional single covariance matrix approach. We found that the proposed approach outperformed both of these methods.

Recent studies in computer vision have provided new solutions to real-world problems. In this paper, we focus on using computer vision methods to assist in the study of kangaroos in the wild. In order to investigate the feasibility, we built a kangaroo image dataset from collected data from several national parks across the State of Queensland. To achieve reasonable detection accuracy, we explored a multi-pose approach and proposed a framework based on the state-of-the-art Deformable Part Model (DPM). Experiments show that the proposed framework outperformed the state-of-the-art methods on the proposed dataset. Also, the proposed vision tools are able to help our field biologists in studying kangaroo related problems such as population tracking for activity analysis.

Widespread deployment of biometric systems supporting consumer transactions is starting to occur. Smart consumer devices, such as tablets and phones, have the potential to act as biometric readers authenticating user transactions. However, the use of these devices in uncontrolled environments is highly susceptible to replay attacks, where these biometric data are captured and replayed at a later time. Current approaches to counter replay attacks in this context are inadequate. In order to show this, we demonstrate a simple replay attack that is 100% effective against a recent state-of-the-art face recognition system; this system was specifically designed to robustly distinguish between live people and spoofing attempts, such as photographs. This paper proposes an approach to counter replay attacks for face recognition on smart consumer devices using a noninvasive challenge and response technique. The image on the screen creates the challenge, and the dynamic reflection from the person's face as they look at the screen forms the response. The sequence of screen images and their associated reflections digitally watermarks the video. By extracting the features from the reflection region, it is possible to determine if the reflection matches the sequence of images that were displayed on the screen. Experiments indicate that the face reflection sequences can be classified under ideal conditions with a high degree of confidence. These encouraging results may pave the way for further studies in the use of video analysis for defeating biometric replay attacks on consumer devices.

In this paper, we tackle the problem of unsupervised domain adaptation for classification. In the unsupervised scenario where no labeled samples from the target domain are provided, a popular approach consists in transforming the data such that the source and target distributions become similar. To compare the two distributions, existing approaches make use of the Maximum Mean Discrepancy (MMD). However, this does not exploit the fact that probability distributions lie on a Riemannian manifold. Here, we propose to make better use of the structure of this manifold and rely on the distance on the manifold to compare the source and target distributions. In this framework, we introduce a sample selection method and a subspace-based method for unsupervised domain adaptation, and show that both these manifold-based techniques outperform the corresponding approaches based on the MMD. Furthermore, we show that our subspace-based approach yields state-of-the-art results on a standard object recognition benchmark.

There has been an ongoing effort in improving reliability and consistency of pathology test results due to their critical role in making an accurate diagnosis. One way to do this is by applying image-based Computer Aided Diagnosis (CAD) systems. This paper proposes a comprehensive benchmarking platform comprising over 1,000 images to evaluate CAD systems for the Anti-Nuclear Antibody (ANA) test via the Indirect Immunofluorescence (IIF) protocol applied on Human Epithelial Type 2 (HEp-2) cells. While prior works in this domain have primarily focussed on classifying individual cell images derived from ANA IIF HEp-2 images, our proposed benchmarking platform goes beyond this by considering the ANA IIF HEp-2 image classification problem. Generally the existing works derive an ANA IIF HEp-2 image label from the dominant pattern of the cell images (we call this approach baseline). In this work, we argue that this approach cannot be used to achieve an acceptable performance, thus, the problem of classifying ANA IIF HEp-2 images (or ANA images in short) is still largely unexplored. To demonstrate that, we propose a simple-yet-effective CAD system which is inspired from the recent success of object bank representation in the object classification domain. We evaluate the proposed system, the baseline and a recent CAD system and show that our proposed system considerably outperforms the others.

Recently the use of image attributes as image descriptors has drawn great attention. This is because the resulting descriptors extracted using these attributes are human understandable as well as machine readable. Although the image attributes are generally semantically meaningful, they may not be discriminative. As such, prior works often consider a discriminative learning approach that could discover discriminative attributes. Nevertheless, the resulting learned attributes could lose their semantic meaning. To that end, in the present work, we study two properties of attributes: discriminative power and reliability. We then propose a novel greedy algorithm called Discriminative and Reliable Attribute Learning (DRAL) which selects a subset of attributes which maximises an objective function incorporating the two properties. We compare our proposed system to the recent state-of-the-art approach, called Direct Attribute Prediction (DAP) for the zero-shot learning task on the Animal with Attributes (AwA) dataset. The results show that our proposed approach can achieve similar performance to this state-of-the-art approach while using a significantly smaller number of attributes.

Recent advances suggest that encoding images through Symmetric Positive Definite (SPD) matrices and then interpreting such matrices as points on Riemannian manifolds can lead to increased classification performance. Taking into account manifold geometry is typically done via (1) embedding the manifolds in tangent spaces, or (2) embedding into Reproducing Kernel Hilbert Spaces (RKHS). While embedding into tangent spaces allows the use of existing Euclidean-based learning algorithms, manifold shape is only approximated which can cause loss of discriminatory information. The RKHS approach retains more of the manifold structure, but may require non-trivial effort to kernelise Euclidean-based learning algorithms. In contrast to the above approaches, in this paper we offer a novel solution that allows SPD matrices to be used with unmodified Euclidean-based learning algorithms, with the true manifold shape well-preserved. Specifically, we propose to project SPD matrices using a set of random projection hyperplanes over RKHS into a random projection space, which leads to representing each matrix as a vector of projection coefficients. Experiments on face recognition, person re-identification and texture classification show that the proposed approach outperforms several recent methods, such as Tensor Sparse Coding, Histogram Plus Epitome, Riemannian Locality Preserving Projection and Relational Divergence Classification.

Traditional nearest points methods use all the samples in an image set to construct a single convex or affine hull model for classification. However, strong artificial features and noisy data may be generated from combinations of training samples when significant intra-class variations and/or noise occur in the image set. Existing multi-model approaches extract local models by clustering each image set individually only once, with fixed clusters used for matching with various image sets. This may not be optimal for discrimination, as undesirable environmental conditions (eg. illumination and pose variations) may result in the two closest clusters representing different characteristics of an object (eg. frontal face being compared to non-frontal face). To address the above problem, we propose a novel approach to enhance nearest points based methods by integrating affine/convex hull classification with an adapted multi-model approach. We first extract multiple local convex hulls from a query image set via maximum margin clustering to diminish the artificial variations and constrain the noise in local convex hulls. We then propose adaptive reference clustering (ARC) to constrain the clustering of each gallery image set by forcing the clusters to have resemblance to the clusters in the query image set. By applying ARC, noisy clusters in the query set can be discarded. Experiments on Honda, MoBo and ETH-80 datasets show that the proposed method outperforms single model approaches and other recent techniques, such as Sparse Approximated Nearest Points, Mutual Subspace Method and Manifold Discriminant Analysis.

A robust visual tracking system requires an object appearance model that is able to handle occlusion, pose, and illumination variations in the video stream. This can be difficult to accomplish when the model is trained using only a single image. In this paper, we first propose a tracking approach based on affine subspaces (constructed from several images) which are able to accommodate the above-mentioned variations. We use affine subspaces not only to represent the object, but also the candidate areas that the object may occupy. We furthermore propose a novel approach to measure affine subspace-to-subspace distance via the use of non-Euclidean geometry of Grassmann manifolds. The tracking problem is then considered as an inference task in a Markov Chain Monte Carlo framework via particle filtering. Quantitative evaluation on challenging video sequences indicates that the proposed approach obtains considerably better performance than several recent state-of-the-art methods such as Tracking-Learning-Detection and MILtrack.

Recently, there has been a growing interest in developing Computer Aided Diagnostic (CAD) systems for improving the reliability and consistency of pathology test results. This paper describes a novel CAD system for the Anti-Nuclear Antibody (ANA) test via Indirect Immunofluorescence protocol on Human Epithelial Type 2 (HEp-2) cells. While prior works have primarily focused on classifying cell images extracted from ANA specimen images, this work takes a further step by focussing on the specimen image classification problem itself. Our system is able to efficiently classify specimen images as well as producing meaningful descriptions of ANA pattern class which helps physicians to understand the differences between various ANA patterns. We achieve this goal by designing a specimen-level image descriptor that: (1) is highly discriminative; (2) has small descriptor length and (3) is semantically meaningful at the cell level. In our work, a specimen image descriptor is represented by its overall cell attribute descriptors. As such, we propose two max-margin based learning schemes to discover cell attributes whilst still maintaining the discrimination of the specimen image descriptor. Our learning schemes differ from the existing discriminative attribute learning approaches as they primarily focus on discovering image-level attributes. Comparative evaluations were undertaken to contrast the proposed approach to various state-of-the-art approaches on a novel HEp-2 cell dataset which was specifically proposed for the specimen-level classification. Finally, we showcase the ability of the proposed approach to provide textual descriptions to explain ANA patterns.

Low-dimensional representations are key to the success of many video classification algorithms. However, the commonly-used dimensionality reduction techniques fail to account for the fact that only part of the signal is shared across all the videos in one class. As a consequence, the resulting representations contain instance-specific information, which introduces noise in the classification process. In this paper, we introduce non-linear stationary subspace analysis: a method that overcomes this issue by explicitly separating the stationary parts of the video signal (i.e., the parts shared across all videos in one class), from its non-stationary parts (i.e., the parts specific to individual videos). Our method also encourages the new representation to be discriminative, thus accounting for the underlying classification problem. We demonstrate the effectiveness of our approach on dynamic texture recognition, scene classification and action recognition.

Existing multi-model approaches for image set classification extract local models by clustering each image set individually only once, with fixed clusters used for matching with other image sets. However, this may result in the two closest clusters to represent different characteristics of an object, due to different undesirable environmental conditions (such as variations in illumination and pose). To address this problem, we propose to constrain the clustering of each query image set by forcing the clusters to have resemblance to the clusters in the gallery image sets. We first define a Frobenius norm distance between subspaces over Grassmann manifolds based on reconstruction error. We then extract local linear subspaces from a gallery image set via sparse representation. For each local linear subspace, we adaptively construct the corresponding closest subspace from the samples of a probe image set by joint sparse representation. We show that by minimising the sparse representation reconstruction error, we approach the nearest point on a Grassmann manifold. Experiments on Honda, ETH-80 and Cambridge-Gesture datasets show that the proposed method consistently outperforms several other recent techniques, such as Affine Hull based Image Set Distance (AHISD), Sparse Approximated Nearest Points(SANP) and Manifold Discriminant Analysis (MDA).

Domain-invariant representations are key to addressing the domain shift problem where the training and test examples follow different distributions. Existing techniques that have attempted to match the distributions of the source and target domains typically compare these distributions in the original feature space. This space, however, may not be directly suitable for such a comparison, since some of the features may have been distorted by the domain shift, or may be domain specific. In this paper, we introduce a Domain Invariant Projection approach: An unsupervised domain adaptation method that overcomes this issue by extracting the information that is invariant across the source and target domains. More specifically, we learn a projection of the data to a low-dimensional latent space where the distance between the empirical distributions of the source and target examples is minimized. We demonstrate the effectiveness of our approach on the task of visual object recognition and show that it outperforms state-of-the-art methods on a standard domain adaptation benchmark dataset.

Recent advances in computer vision and machine learning suggest that a wide range of problems can be addressed more appropriately by considering non-Euclidean geometry. In this paper we explore sparse dictionary learning over the space of linear subspaces, which form Riemannian structures known as Grassmann manifolds. To this end, we propose to embed Grassmann manifolds into the space of symmetric matrices by an isometric mapping, which enables us to devise a closed-form solution for updating a Grassmann dictionary, atom by atom. Furthermore, to handle non-linearity in data, we propose a kernelised version of the dictionary learning algorithm. Experiments on several classification tasks (face recognition, action recognition, dynamic texture classification) show that the proposed approach achieves considerable improvements in discrimination accuracy, in comparison to state-of-the-art methods such as kernelised Affine Hull Method and graph-embedding Grassmann discriminant analysis.

Low-dimensional representations are key to the success of many video classification algorithms. However, the commonly-used dimensionality reduction techniques fail to account for the fact that only part of the signal is shared across all the videos in one class. As a consequence, the resulting representations contain instance-specific information, which introduces noise in the classification process. In this paper, we introduce Non-Linear Stationary Subspace Analysis: A method that overcomes this issue by explicitly separating the stationary parts of the video signal (i.e., the parts shared across all videos in one class), from its non-stationary parts (i.e., specific to individual videos). We demonstrate the effectiveness of our approach on action recognition, dynamic texture classification and scene recognition.

We propose a new action and gesture recognition method based on spatio-temporal covariance descriptors and a weighted Riemannian locality preserving projection approach that takes into account the curved space formed by the descriptors. The weighted projection is then exploited during boosting to create a final multiclass classification algorithm that employs the most useful spatio-temporal regions. We also show how the descriptors can be computed quickly through the use of integral video representations. Experiments on the UCF sport, CK+ facial expression and Cambridge hand gesture datasets indicate superior performance of the proposed method compared to several recent state-of-the-art techniques. The proposed method is robust and does not require additional processing of the videos, such as foreground detection, interest-point detection or tracking.

The Anti-Nuclear Antibody (ANA) clinical pathology test is commonly used to identify the existence of various diseases. A hallmark method for identifying the presence of ANAs is the Indirect Immunofluorescence method on Human Epithelial (HEp-2) cells, due to its high sensitivity and the large range of antigens that can be detected. However, the method suffers from numerous shortcomings, such as being subjective as well as time and labour intensive. Computer Aided Diagnostic (CAD) systems have been developed to address these problems, which automatically classify a HEp-2 cell image into one of its known patterns (eg., speckled, homogeneous). Most of the existing CAD systems use handpicked features to represent a HEp-2 cell image, which may only work in limited scenarios. In this paper, we propose a cell classification system comprised of a dual-region codebook-based descriptor, combined with the Nearest Convex Hull Classifier. We evaluate the performance of several variants of the descriptor on two publicly available datasets: ICPR HEp-2 cell classification contest dataset and the new SNPHEp-2 dataset. To our knowledge, this is the first time codebook-based descriptors are applied and studied in this domain. Experiments show that the proposed system has consistent high performance and is more robust than two recent CAD systems.

Background subtraction is a fundamental low-level processing task in numerous computer vision applications. The vast majority of algorithms process images on a pixel-by-pixel basis, where an independent decision is made for each pixel. A general limitation of such processing is that rich contextual information is not taken into account. We propose a block-based method capable of dealing with noise, illumination variations, and dynamic backgrounds, while still obtaining smooth contours of foreground objects. Specifically, image sequences are analyzed on an overlapping block-by-block basis. A low-dimensional texture descriptor obtained from each block is passed through an adaptive classifier cascade, where each stage handles a distinct problem. A probabilistic foreground mask generation approach then exploits block overlaps to integrate interim block-level decisions into final pixel-level foreground segmentation. Unlike many pixel-based methods, ad-hoc postprocessing of foreground masks is not required. Experiments on the difficult Wallflower and I2R datasets show that the proposed approach obtains on average better results (both qualitatively and quantitatively) than several prominent methods. We furthermore propose the use of tracking performance as an unbiased approach for assessing the practical usefulness of foreground segmentation methods, and show that the proposed approach leads to considerable improvements in tracking accuracy on the CAVIAR dataset.

We propose an effective approach to take advantage of the rich description provided by Spatiotemporal Oriented Energy features for the purpose of robust tracking. There are two core components in our system. The first one is a compound measure of 'Coherent Motion' and 'Identity Motion Signature' which is introduced based on motion dynamics of the targets. This measure is used for robust optimisation in occluded situations as well as an adaptive template updating scheme. The second component is a state machine which detects various states of the targets based on statistical analysis of their 'Motion Signature'. Empirical evaluations demonstrate improvement in performance of the tracking system along with the role of each component.

In contrast to comparing faces via single exemplars, matching sets of face images increases robustness and discrimination performance. Recent image set matching approaches typically measure similarities between subspaces or manifolds, while representing faces in a rigid and holistic manner. Such representations are easily affected by variations in terms of alignment, illumination, pose and expression. While local feature based representations are considerably more robust to such variations, they have received little attention within the image set matching area. We propose a novel image set matching technique, comprised of three aspects: (i) robust descriptors of face regions based on local features, partly inspired by the hierarchy in the human visual system, (ii) use of several subspace and exemplar metrics to compare corresponding face regions, (iii) jointly learning which regions are the most discriminative while finding the optimal mixing weights for combining metrics. Experiments on LFW, PIE and MOBIO face datasets show that the proposed algorithm obtains considerably better performance than several recent state of-the-art techniques, such as Local Principal Angle and the Kernel Affine Hull Method.

For covariance-based image descriptors, taking into account the curvature of the corresponding feature space has been shown to improve discrimination performance. This is often done through representing the descriptors as points on Riemannian manifolds, with the discrimination accomplished on a tangent space. However, such treatment is restrictive as distances between arbitrary points on the tangent space do not represent true geodesic distances, and hence do not represent the manifold structure accurately. In this paper we propose a general discriminative model based on the combination of several tangent spaces, in order to preserve more details of the structure. The model can be used as a weak learner in a boosting-based pedestrian detection framework. Experiments on the challenging INRIA and DaimlerChrysler datasets show that the proposed model leads to considerably higher performance than methods based on histograms of oriented gradients as well as previous Riemannian-based techniques.

With the aim of improving the clustering of data (such as image sequences) lying on Grassmann manifolds, we propose to embed the manifolds into Reproducing Kernel Hilbert Spaces. To this end, we define a measure of cluster distortion and embed the manifolds such that the distortion is minimised. We show that the optimal solution is a generalised eigenvalue problem that can be solved very efficiently. Experiments on several clustering tasks (including human action clustering) show that in comparison to the recent intrinsic Grassmann k-means algorithm, the proposed approach obtains notable improvements in clustering accuracy, while also being several orders of magnitude faster.

In the field of face recognition, Sparse Representation (SR) has received considerable attention during the past few years. Most of the related literature focuses on holistic descriptors in closed-set identification applications. The underlying assumption in identification is that the gallery always has sufficient samples per subject to linearly reconstruct a query image. Unfortunately, such assumption is easily violated in the more challenging and realistic face verification scenario. A verification algorithm is required to determine if two faces (where one or both have not been seen before) belong to the same person, while explicitly taking into account the possibility of impostor attacks. In this paper, we first discuss why most of the SR literature is not applicable to verification problems. Motivated by the success of bag-of-words methods in the field of object recognition, which describe an image as a set of local patches or interest points, we then propose to tackle the verification problem by encoding each local face patch through SR. The locally encoded sparse vectors are pooled to form regional descriptors, where each descriptor covers a relatively large portion of the face. Experiments in various challenging conditions show that the proposed method achieves high and robust verification performance.

A wireless sensor network (WSN) was designed and implemented to provide reliable long-term hazard monitoring at the Port of Brisbane, Australia. The proposed system consists of four sensor nodes, a wireless gateway and a central monitoring computer. The sensor nodes are capable of measuring a range of hazardous events along with the time and location of events in maritime environment. Each sensor node is also equipped with a Global Positioning System (GPS) module and a ZigBee module. The monitoring server is a personal computer application server with Internet connectivity. Sensor nodes utilize smart algorithm to save energy and AES-128 encryption to encode data prior to sending the data packet through the wireless ZigBee protocol to gateway. The gateway collects and decrypts the data packets and forwards them to the monitoring computer through wireless connection. A database server running on the monitoring computer stores the captured data for visualization and further analysis. The monitoring server is interfaced to Google Maps to overlay real-time data from the sensor nodes onto map in the correct corresponding locations. The WSN system was successfully deployed and tested at the Port of Brisbane, Queensland, Australia.

A convenient way of analysing Riemannian manifolds is to embed them in Euclidean spaces, with the embedding typically obtained by flattening the manifold via tangent spaces. This general approach is not free of drawbacks. For example, only distances between points to the tangent pole are equal to true geodesic distances. This is restrictive and may lead to inaccurate modelling. Instead of using tangent spaces, we propose embedding into the Reproducing Kernel Hilbert Space by introducing a Riemannian pseudo kernel. We furthermore propose to recast a locality preserving projection technique from Euclidean spaces to Riemannian manifolds, in order to demonstrate the benefits of the embedding. Experiments on several visual classification tasks (gesture recognition, person re-identification and texture classification) show that in comparison to tangent-based processing and state-of-the-art methods (such as tensor canonical correlation analysis), the proposed approach obtains considerable improvements in discrimination accuracy.

Today's work in the sensor networks community focuses on collecting and processing data from specific networks with associated base stations. One of the most important requirements in these networks is minimizing resource usage such as processing power and storage size on sensor nodes. Resource constraints in the sensor nodes can be divided into four categories: energy, communication, storage and computational power. In this paper, we present an efficient deployment of sensors with possibility of accessing most recent data through information obtained from ERP (enterprise resource planning) systems' re-configuration models. In this scheme the probability of losing any precious data or events would be minimized. In other words, our main focus in this paper is using ERP or high level distributed decision making systems' processed data or prediction models to reduce resource usage needed on sensor nodes. In the area of integration of sensors and ERPs or distributed decision making systems, very little work has been reported. However, those reported in the literature, mostly address relaying data from sensors to ERPs and tend to largely ignore issues that come with sensor resource constraints. Also they don't make use of the information processed and generated by ERPs in the cloud environment to optimize power consumption and transmission frequency of sensors, which is our aim.

A convenient way of dealing with image sets is to represent them as points on Grassmannian manifolds. While several recent studies explored the applicability of discriminant analysis on such manifolds, the conventional formalism of discriminant analysis suffers from not considering the local structure of the data. We propose a discriminant analysis approach on Grassmannian manifolds, based on a graph-embedding framework. We show that by introducing within-class and between-class similarity graphs to characterise intra-class compactness and inter-class separability, the geometrical structure of data can be exploited. Experiments on several image datasets (PIE, BANCA, MoBo, ETH-80) show that the proposed algorithm obtains considerable improvements in discrimination accuracy, in comparison to three recent methods: Grassmann Discriminant Analysis (GDA), Kernel GDA, and the kernel version of Affine Hull Image Set Distance. We further propose a Grassmannian kernel, based on canonical correlation between subspaces, which can increase discrimination accuracy when used in combination with previous Grassmannian kernels.

Recently it has been shown that the performance of image set matching methods can be improved by clustering set samples into smaller and more coherent groups. Typically, set samples are treated independently during clustering, ie., clustering criteria have not been defined to exploit set characteristics. In this paper we introduce a novel approach to image set clustering by considering the similarities between subspaces instead of similarities between samples. We exploit an ensemble learning technique to create an ensemble of subspace pairs. Each pair has the property that its members are located at the furthest distance in the sense of distances between subspaces. Object recognition experiments on the CMU-MoBO and ETH-80 datasets show that the proposed method obtains higher discrimination accuracy in comparison to several benchmark methods as well as the recently proposed Kernel Affine Hull Method.

In recent years, there has been a growing interest in IP-reuse for SoCs in order to bridge the gap between the silicon capacity and the design productivity. This research work investigates how our proposed methodology can be used to partition and schedule a JPEG encoder IP core onto an FPGA. We will also describe a novel Constraint Satisfaction Problem (CSP) formulations that are used in the proposed framework. At the same time, we will also demonstrate the effectiveness of CSP in the Bayesian Network-based framework.

A robust foreground object segmentation technique is proposed, capable of dealing with image sequences containing noise, illumination variations and dynamic backgrounds. The method employs contextual spatial information by analysing each image on an overlapping patch-by-patch basis and obtaining a low-dimensional texture descriptor for each patch. Each descriptor is passed through an adaptive multi-stage classifier, comprised of a likelihood evaluation, an illumination robust measure, and a temporal correlation check. A probabilistic foreground mask generation approach integrates the classification decisions by exploiting the overlapping of patches, ensuring smooth contours of the foreground objects as well as effectively minimising the number of errors. The parameter settings are robust against wide variety of sequences and post-processing of foreground masks is not required. Experiments on the difficult Wallflower and I2R datasets show that the proposed method obtains considerably better results (both qualitatively and quantitatively) than methods based on Gaussian mixture models, feature histograms, and normalised vector distances. Further experiments on the CAVIAR dataset (using several tracking algorithms) indicate that the proposed method leads to considerable improvements in object tracking accuracy.

This paper presents a novel generic weak classifier for object detection called "Square Patch Feature". The speed and overall performance of a detector utilising Square Patch features in comparison to other weak classifiers shows improvement. Each weak classifier is based on the difference between two or four fixed size square patches in an image. A pre-calculated representation of the image called "patch image" is required to accelerate the weak classifiers computation. The computation requires fewer arithmetic operations and fewer accesses to the main memory in comparison to the well known Viola-Jones Haar-like classifier. In addition to the faster computation, the weak classifier can be extended for in-plane rotation, where each square patch can be rotated to detect in-plane rotated objects. The results of the experiments on the MIT CBCL Face dataset show that a Square Patch Feature classifier is as accurate as the Viola-Jones Haar-like classifier, and when implemented on hardware (i.e. FPGA), it is almost 2 times faster.

This paper presents the use of local oriented energy features for real-time object tracking on embedded vision systems. Local oriented energy features are extracted using complex Gabor filters. Filtering is carried out across multiple channels with different frequencies and orientations. The effectiveness of the chosen feature set is tested using a mean-shift tracker. Our experiments show that adding local oriented energy features can significantly enhance the performance of the tracker in presence of photometric variations and geometric transformation. The realtime implementation of the system is also described in this paper. To achieve the desired performance, a hardware/software co-design approach is pursued. Multi-channel Gabor filtering, Local Oriented Energy Feature and Feature histogram Computation is implemented on hardware while mean-shift vector calculation is performed on a processor. The system was synthesized onto a Xilinx Virtex-5 XC5VSX50T using Xilinx ML506 development board and the implementation results are presented.

In this paper we consider the problem of face recognition in a scenario when the query consists of a set of images and the gallery contains a single still image per subject. This is a more challenging problem compared to image-set to image-set matching and has wider applications in advanced surveillance, smart access control and human-computer interaction. Unfortunately most of the previous matching strategies in literature fail to work or deteriorate drastically if they are provided with one sample per class as the gallery data. In this paper we demonstrate how transductive learning can be utilized to map the image-set to single image matching problem into the recently-studied framework of set matching using canonical correlations. Experimental results on different challenging datasets reveal the efficiency of the proposed method against existing approaches.

We propose a novel face image similarity measure based on Hausdorff distance (HD). In contrast to conventional HD-based measures, which are generally applied in the image space (such as edge maps or gradient images), the proposed HD-based similarity measure is applied in the feature space. By extending the concept of HD using a variable radius and reference set, we can generate a neighbourhood set for HD measures in feature space and then apply this concept for classification. Experiments on the `Labeled Faces in the Wild' and FRGC datasets show that the proposed measure improves the overall classification performance quite dramatically, especially under the highly desirable low false acceptance rate conditions.

With growing attention to ensemble learning, in recent years various ensemble methods for face recognition have been proposed that show promising results. Among diverse ensemble construction approaches, random subspace method has received considerable attention in face recognition. Although random feature selection in random subspace method improves accuracy in general, it is not free of serious difficulties and drawbacks. In this paper we present a learning scheme to overcome some of the drawbacks of random feature selection in the random subspace method. The proposed learning method derives a feature discrimination map based on a measure of accuracy and uses it in a probabilistic recall mode to construct an ensemble of subspaces. Experiments on different face databases revealed that the proposed method gives superior performance over the well-known benchmarks and state of the art ensemble methods.

We propose a region-based foreground object segmentation method capable of dealing with image sequences containing noise, illumination variations and dynamic backgrounds (as often present in outdoor environments). The method utilises contextual spatial information through analysing each frame on an overlapping block by-block basis and obtaining a low-dimensional texture descriptor for each block. Each descriptor is passed through an adaptive multi-stage classifier, comprised of a likelihood evaluation, an illumination invariant measure, and a temporal correlation check. The overlapping of blocks not only ensures smooth contours of the foreground objects but also effectively minimises the number of false positives in the generated foreground masks. The parameter settings are robust against wide variety of sequences and post-processing of foreground masks is not required. Experiments on the challenging I2R dataset show that the proposed method obtains considerably better results (both qualitatively and quantitatively) than methods based on Gaussian mixture models (GMMs), feature histograms, and normalised vector distances. On average, the proposed method achieves 36% more accurate foreground masks than the GMM based method.

Shadow detection and removal is an important step employed after foreground detection, in order to improve the segmentation of objects for tracking. Methods reported in the literature typically have a significant trade-off between the shadow detection rate (classifying true shadow areas as shadows) and the shadow discrimination rate (discrimination between shadows and foreground). We propose a method that is able to achieve good performance in both cases, leading to improved tracking in surveillance scenarios. Chromacity information is first used to create a mask of candidate shadow pixels, followed by employing gradient information to remove foreground pixels that were incorrectly included in the mask. Experiments on the CAVIAR dataset indicate that the proposed method leads to considerable improvements in multiple object tracking precision and accuracy.

While existing face recognition systems based on local features are robust to issues such as misalignment, they can exhibit accuracy degradation when comparing images of differing resolutions. This is common in surveillance environments where a gallery of high resolution mugshots is compared to low resolution CCTV probe images, or where the size of a given image is not a reliable indicator of the underlying resolution (e.g. poor optics). To alleviate this degradation, we propose a compensation framework which dynamically chooses the most appropriate face recognition system for a given pair of image resolutions. This framework applies a novel resolution detection method which does not rely on the size of the input images, but instead exploits the sensitivity of local features to resolution using a probabilistic multi-region histogram approach. Experiments on a resolution-modified version of the "Labeled Faces in the Wild" dataset show that the proposed resolution detector frontend obtains a 99% average accuracy in selecting the most appropriate face recognition system, resulting in higher overall face discrimination accuracy (across several resolutions) compared to the individual baseline face recognition systems.

An abandoned object detection system is presented and evaluated using benchmark datasets. The detection is based on a simple mathematical model and works efficiently at QVGA resolution at which most CCTV cameras operate. The pre-processing involves a dual-time background subtraction algorithm which dynamically updates two sets of background, one after a very short interval (less than half a second) and the other after a relatively longer duration. The framework of the proposed algorithm is based on the approximate median model. An algorithm for tracking of abandoned objects even under occlusion is also proposed. Results show that the system is robust to variations in lighting conditions and the number of people in the scene. In addition, the system is simple and computationally less intensive as it avoids the use of expensive filters while achieving better detection results.

Many computer vision algorithms such as object tracking and event detection assume that a background model of the scene under analysis is known. However, in many practical circumstances it is unavailable and must be estimated from cluttered image sequences. We propose a sequential technique for background estimation in such conditions, with low computational and memory requirements. The first stage is somewhat similar to that of the recently proposed agglomerative clustering background estimation method, where image sequences are analysed on a patch by patch basis. For each patch location a representative set is maintained which contains distinct patches obtained along its temporal line. The novelties lie in iteratively filling in background areas by selecting the most appropriate candidate patches according to the combined frequency responses of extended versions of the candidate patch and its neigh-bourhood. It is assumed that the most appropriate patch results in the smoothest response, indirectly enforcing the spatial continuity of structures within a scene. Experiments on real-life surveillance videos demonstrate the efficacy of the proposed method.

Recognizing faces with uncontrolled pose, illumination, and expression is a challenging task due to the fact that features insensitive to one variation may be highly sensitive to the other variations. Existing techniques dealing with just one of these variations are very often unable to cope with the other variations. The problem is even more difficult in applications where only one gallery image per person is available. In this paper, we describe a recognition method, Adapted Principal Component Analysis (APCA), that can simultaneously deal with large variations in both illumination and facial expression using only a single gallery image per person. We have now extended this method to handle head pose variations in two steps. The first step is to apply an Active Appearance Model (AAM) to the non-frontal face image to construct a synthesized frontal face image. The second is to use APCA for classification robust to lighting and pose. The proposed technique is evaluated on three public face databases — Asian Face, Yale Face, and FERET Database — with images under different lighting conditions, facial expressions, and head poses. Experimental results show that our method performs much better than other recognition methods including PCA, FLD, PRM and LTP. More specifically, we show that by using AAM for frontal face synthesis from high pose angle faces, the recognition rate of our APCA method increases by up to a factor of 4.

Although automatic identity inference based on faces has shown success when using high quality images, for CCTV based images it is hard to attain similar levels of performance. Furthermore, compared to recognition based on static images, relatively few studies have been done for video based face recognition. In this paper, we present an empirical analysis and comparison of face recognition using high quality and CCTV images in several important aspects: image quality (including resolution, noise, blurring and interlacing) as well as geometric transformations (such as translations, rotations and scale changes). The results show that holistic face recognition can be tolerant to image quality degradation but can also be highly influenced by geometric transformations. In addition, we show that camera intrinsics have much influence - when using different cameras for collecting gallery and probe images the recognition rate is considerably reduced. We also show that the classification performance can be considerably improved by straightforward averaging of consecutive face images from a CCTV video sequence.

We describe a project to trial and develop enhanced surveillance technologies for public safety. A key technology is robust recognition of faces from low-resolution CCTV footage where there may be as few as 12 pixels between the eyes. Current commercial face recognition systems require 60-90 pixels between the eyes as well as tightly controlled image capture conditions. Our group has thus concentrated on fundamental face recognition issues such as robustness to low resolution and image capture conditions as required for uncontrolled CCTV surveillance. In this paper, we propose a fast multi-class pattern classification approach to enhance PCA and FLD methods for 2D face recognition under changes in pose, illumination, and expression. The method first finds the optimal weights of features pairwise and constructs a feature chain in order to determine the weights for all features. Computational load of the proposed approach is extremely low by design, in order to facilitate usage in automated surveillance. The method is evaluated on PIE, FERET, and Asian Face databases, with the results showing that the method performs remarkably well compared to several benchmark appearance-based methods. Moreover, the method can reliably recognise faces with large pose angles from just one gallery image.

Under the constraint of using only a single gallery image per person, this paper proposes a fast multi-class pattern classification approach to 2D face recognition robust to changes in pose, illumination, and expression (PIE). This work has three main contributions: (1) we propose a representative face space method to extract robust features, (2) we apply a learning method to weight features in pairs, (3) we combine the feature pairs into a feature chain in order to find the weights for all features. The approach is evaluated for face recognition under PIE changes on three public databases. Results show that the method performs considerably better than several other appearance-based methods and can reliably recognise faces at large pose angles without the need for fragile pose estimation pre-processing. Moreover, computational load is low (comparable to standard eigenface methods), which is a critical factor in wide-area surveillance applications.

This paper describes a visual surveillance system for remote monitoring of unattended environments. For the purpose of efficiently tracking multiple people in the presence of occlusions, we propose: (i) to combine blob matching with particle filtering, and (ii) to augment these tracking algorithms with a novel colour appearance model. The proposed system efficiently counteracts the shortcomings of the two algorithms by switching from one to the other during occlusions. Results on public datasets as well as real surveillance videos from a metropolitan railway station demonstrate the efficacy of the proposed system.

We describe a pedestrian classification and tracking system that is able to track and label multiple people in an outdoor environment such as a railway station. The features selected for appearance modelling are circular colour histograms for the hue and conventional colour histograms for the saturation and value components. We combine blob matching with a particle filter for tracking and augment these algorithms with colour appearance models to track multiple people in the presence of occlusion. In the object classification stage, hierarchical chamfer matching combined with particle filtering is applied to classify commuters in the railway station into several classes. Classes of interest include normal commuters, commuters with backpacks, commuters with suitcases, and mothers with their children.

In recent years, the use of Intelligent Closed-Circuit Television (ICCTV) for crime prevention and detection has attracted significant attention. Existing face recognition systems require passport-quality photos to achieve good performance. However, use of CCTV images is much more problematic due to large variations in illumination, facial expressions and pose angle. In this paper we propose a pose variability compensation technique, which synthesizes realistic frontal face images from non-frontal views. It is based on modelling the face via Active Appearance Models and detecting the pose through a correlation model. The proposed technique is coupled with Adaptive Principal Component Analysis (APCA), which was previously shown to perform well in the presence of both lighting and expression variations. Experiments on the FERET dataset show up to 6 fold performance improvements. Finally, in addition to implementation and scalability challenges, we discuss issues related to on-going real life trials in public spaces using existing surveillance hardware.

This paper defines the 3D reconstruction problem as the process of reconstructing a 3D scene from numerous 2D visual images of that scene. It is well known that this problem is ill-posed, and numerous constraints and assumptions are used in 3D reconstruction algorithms in order to reduce the solution space. Unfortunately, most constraints only work in a certain range of situations and often constraints are built into the most fundamental methods (e.g. Area Based Matching assumes that all the pixels in the window belong to the same object). This paper presents a novel formulation of the 3D reconstruction problem, using a voxel framework and first order logic equations, which does not contain any additional constraints or assumptions. Solving this formulation for a set of input images gives all the possible solutions for that set, rather than picking a solution that is deemed most likely. Using this formulation, this paper studies the problem of uniqueness in 3D reconstruction and how the solution space changes for different configurations of input images. It is found that it is not possible to guarantee a unique solution, no matter how many images are taken of the scene, their orientation or even how much color variation is in the scene itself. Results of using the formulation to reconstruct a few small voxel spaces are also presented. They show that the number of solutions is extremely large for even very small voxel spaces (5 × 5 voxel space gives 10 to 107 solutions). This shows the need for constraints to reduce the solution space to a reasonable size. Finally, it is noted that because of the discrete nature of the formulation, the solution space size can be easily calculated, making the formulation a useful tool to numerically evaluate the usefulness of any constraints that are added.

A highly successful active contour implementation, for the automatic segmentation of cervical cell nuclei, is shown to lend itself well to a framework that further increases its success rate. The method is based upon measuring changes in the final contour as the one parameter that governs its behaviour is varied. Only one object of interest is contained in the image, but as artefacts often appear as well, the contour varies with the parameter as it finds different solutions. In contrast, simple images with no artefacts are very stable. Therefore a stability measure is calculated and each image is classified according to its degree of `difficulty'. A system can then choose at which level to operate to ensure only high quality examples are processed after segmentation.

An image segmentation scheme is shown to be exceptionally successful through the application of high-level knowledge of the required image objects (cell nuclei). By tuning the algorithm's single parameter it is shown that the performance can be maximised for the dataset, but leads to individual failures that may require alternative choices. A second stage is introduced to process each of the resulting segmentations obtained by varying the parameter over the working range. This stage gives a Bayesian interpretation of the results which indicates the probable accuracy of each of the segmentations that can then be used to make a decision upon whether to accept or reject the segmentation.

Proposes a rule-based inductive learning algorithm called multiscale classification (MSC). It can be applied to any N-dimensional real or binary classification problem to classify the training data by successively splitting the feature space in half. The algorithm has several significant differences from existing rule-based approaches: learning is incremental, the tree is non-binary, and backtracking of decisions is possible to some extent. The paper first provides background on current machine learning techniques and outlines some of their strengths and weaknesses. It then describes the MSC algorithm and compares it to other inductive learning algorithms with particular reference to ID3, C4.5, and back-propagation neural networks. Its performance on a number of standard benchmark problems is then discussed and related to standard learning issues such as generalization, representational power, and over-specialization.

This paper is an extension of work by He and Kundu (1991) in which the task of object recognition is performed on silhouettes or outlines. He and Kundu reduce a 2-dimensional image to a 1-dimensional segment sequence and use the autoregressive (AR) model for feature extraction and hidden Markov model (HMM) for classification. We show that due to the AR model's inability to estimate abrupt changes (i.e. where the signal is not bandlimited), poor image modelling is obtained. By direct application of vector quantization (VQ) to the normalized data segments, the image features are retained better than with AR modelling. Furthermore, by replacing the HMM classification with VQ distortion, better recognition results are obtained with reduced training times as compared to the HMM algorithm.>

It is shown that the conventional (linear) definitions of mean and variance can yield absurd results when applied to data defined on a circular domain. Appropriate definitions for the mean, variance, and moments of circular data are given, and these concepts are applied to reformulate a recently proposed frequency estimator to avoid bias and elevated threshold effects. The maximum likelihood frequency estimator is shown to be equivalent to least-squares regression on phase estimates. These results demonstrate the importance of appreciating the true circular nature of many quantities commonly encountered in digital signal processing.>

Nonstationary signals are partitioned into near-stationary segments using a modified Appel and Brandt algorithm. The modification requires two spectral distance measures to be used to produce an algorithm which is insensitive to changes in signal energy level which are irrelevant in this application. Performance on real and simulated data is presented. Segmentation has been used to provide an estimator of the evolutive spectrum, an application to a noisy communication signal is presented.>