Image enhancement is an essential step in developing a robust computer vision system. This study presents an image contrast enhancement method for an online fish behaviour monitoring system. The method executes in three steps. First, the critical information on the context of the image is derived from different responses of two edge detectors: one with high threshold and the other with low threshold, which differentiate the image structure into the good contrast part and the poor contrast part. Then, statistics of the gradient field in the former part are used to modify the gradient field in the latter part. Finally, in the modified region, the image contents are reconstructed through solving a Poisson equation with the Dirichlet boundary condition. Comparison with state‐of‐the‐art methods has been made both qualitatively and quantitatively using realistic images and advantages of the proposed method have been demonstrated.

This paper proposes a quadratic classification approach on the subspace of Extended Histogram of Gradients (ExHoG) for human detection. By investigating the limitations of Histogram of Gradients (HG) and Histogram of Oriented Gradients (HOG), ExHoG is proposed as a new feature for human detection. ExHoG alleviates the problem of discrimination between a dark object against a bright background and vice versa inherent in HG. It also resolves an issue of HOG whereby gradients of opposite directions in the same cell are mapped into the same histogram bin. We reduce the dimensionality of ExHoG using Asymmetric Principal Component Analysis (APCA) for improved quadratic classification. APCA also addresses the asymmetry issue in training sets of human detection where there are much fewer human samples than non-human samples. Our proposed approach is tested on three established benchmarking data sets--INRIA, Caltech, and Daimler--using a modified Minimum Mahalanobis distance classifier. Results indicate that the proposed approach outperforms current state-of-the-art human detection methods.

This paper proposes two sets of novel edge-texture features, Discriminative Robust Local Binary Pattern (DRLBP) and Ternary Pattern (DRLTP), for object recognition. By investigating the limitations of Local Binary Pattern (LBP), Local Ternary Pattern (LTP) and Robust LBP (RLBP), DRLBP and DRLTP are proposed as new features. They solve the problem of discrimination between a bright object against a dark background and vice-versa inherent in LBP and LTP. DRLBP also resolves the problem of RLBP whereby LBP codes and their complements in the same block are mapped to the same code. Furthermore, the proposed features retain contrast information necessary for proper representation of object contours that LBP, LTP, and RLBP discard. Our proposed features are tested on seven challenging data sets: INRIA Human, Caltech Pedestrian, UIUC Car, Caltech 101, Caltech 256, Brodatz, and KTH-TIPS2-a. Results demonstrate that the proposed features outperform the compared approaches on most data sets.

This paper tackles the fish motion synthesis problem with the aim to increase the temporal resolution by generating a high frame rate video from a given video captured at a lower frame rate. The proposed approach reconstructs the virtual frames via estimating motion-vector fields that represent the correspondences between consecutive frames. To be more specific, the proposed approach utilizes two regularization functions: (i) an intra regularization function that is proposed to impose a local smoothness constraint on the estimated motion vector fields; and (ii) an inter regularization function that considers both intensity consistency and kinematic consistency between the reconstructed frame and its neighboring frames. Real-world fish motion video dataset is used to demonstrate the superior performance of the proposed approach in high temporal resolution fish motion synthesis, as well as its application in fish motion analysis.

Despite superior performance of Local Binary Pattern (LBP) in texture classification and face detection, its performance in human detection has been limited for two reasons. Firstly, LBP differentiates a bright human from a dark background and vice-versa. This increases the intra-class variation of humans. Secondly, LBP is contrast and illumination invariant. It does not discriminate between weak contrast local regions and similar strong contrast ones, resulting in a similar feature representation. Non-Redundant LBP (NRLBP) has been proposed to solve the first issue of LBP. However, an inherent limitation of NRLBP is that LBP codes and their complements in the same block are mapped to the same code. Furthermore, NRLBP, like LBP, is also contrast and illumination invariant. In this paper, we propose a novel edge-texture feature, Discriminative Robust Local Binary Pattern (DRLBP), for human detection. DRLBP alleviates the problems of LBP and NRLBP by considering the weighted sum and absolute difference of a LBP code and its complement. Our experimental results show that DRLBP consistently outperforms LBP and NRLBP for human detection.

In this paper, we present a parts-based modeling framework using Extended Histogram of Gradients (ExHoG) for object detection. Visual object detection is a challenging issue in computer vision where objects need to be detected in varying illumination and contrast environments. Furthermore, objects belonging to the same class exhibit large intra-class variations. Here, we propose using ExHoG with the discriminatively trained deformable part models of Felzenszwalb et. al. [1]. This framework is based on mixtures of multiscale deformable part models. ExHoG is a novel feature proposed earlier for the purpose of human detection and has shown promising results against other state-of-the-art approaches. The proposed approach is tested on INRIA Human data set and the PASCAL VOC 2007 data set. Results demonstrate superior performance on INRIA compared to existing state-of-the-art approaches and improved performance on PASCAL VOC 2007.

This paper addresses the problem of detecting and localizing abnormal activities in crowded scenes. A spatiotemporal Laplacian eigenmap method is proposed to extract different crowd activities from videos. This is achieved by learning the spatial and temporal variations of local motions in an embedded space. We employ representatives of different activities to construct the model which characterizes the regular behavior of a crowd. This model of regular crowd behavior allows the detection of abnormal crowd activities both in local and global contexts and the localization of regions which show abnormal behavior. Experiments on the recently published data sets show that the proposed method achieves comparable results with the state-of-the-art methods without sacrificing computational simplicity.

This paper presents a novel approach for the visualization and clustering of crowd video contents by using multilinear principal component analysis (MPCA). In contrast to feature-point-based approach and frame-based dimensionality reduction approach, the proposed method maps each short video segment to a point in MPCA subspace to take temporal information into account naturally through tensorial representations. Specifically, MPCA projects each short segment of a video to a low-dimensional tensor first. A few MPCA features are then selected according to the variance captured as the final representation. Thus, a video is visualized as a trajectory in MPCA subspace. The trajectory generated enables visual interpretation of video content in a compact space as well as visual clustering of video events. The proposed method is evaluated on the PETS 2009 datasets through comparison with three existing methods for video visualization. The MPCA visualization shows superior performance in clustering segments of the same event as well as identifying the transitions between events.

We propose an appearance based eigenfeature regularization methodology for recognizing human activities. This regularization utilizes a 3-parameter based eigenmodel derived from the variances of within-class (activity) scatter matrix. Original eigenvalues are replaced by the model eigenvalues which facilitates in regularizing eigenfeatures corresponding to very small and zero eigenvalues and perform discriminant evaluation in the whole eigenspace. This is done directly from the intensity information appearing in activity images. After this regularization, low dimensional discriminative features are extracted and used for recognizing various activities. Experimental results on two benchmark databases, Weizmann and INRIA-IXMAS show the superiority of our proposed approach over other popular methods.

Unsigned Histogram of Gradients (UHoG) is a popular feature used for human detection. Despite its superior performance as reported in recent literature, an inherent limitation of UHoG is that gradients of opposite directions in a cell are mapped into the same histogram bin. This is undesirable as it will produce the same UHoG feature for two different patterns. To address this problem, we propose a new feature named the Extended Histogram of Gradients (ExHoG) in this paper. It comprises two components: UHoG and a histogram of absolute bin value differences of opposite gradient directions computed from Histogram of Gradients (HoG). Our experimental results show that the proposed ExHoG consistently outperforms the standard HoG and UHoG for human detection.

A method to extract views of different orientations of a vehicle captured using multi cameras on roadside is proposed. We expect the use of multi views would increase classification performance in tasks such as identifying vehicle types/makes. This paper does not discuss classification work in details; it accepts the concept that with more data obtained through multi camera views, the use of distinctive orientations only would improve classifier's performance. Prior to this, we have to resolve practical issues such as identifying condition of vehicle merges and shadow. We use correlated data from multi cameras to find the most optimized cut for a merge situation. We also propose a novel approach of removing vehicle's shadow using blob reconstruction technique. Views of different vehicle orientations (in our experiment, left, rear, right) are interpreted using a 3D graph fitting on images from multi cameras.

The main focus of this special issue is to bring together solutions from research and engineering for the automatic understanding of a complex scene, via a multimodal array of sensors with automation of adaptation as the theme connecting various research areas. Four papers comprise this special issue.

This paper presents a deterministic solution to an approximated classification-error based objective function. In the formulation, we propose a quadratic approximation as the function for achieving smooth error counting. The solution is subsequently found to be related to the weighted least-squares whereby a robust tuning process can be incorporated. The tuning traverses between the least-squares estimate and the approximated total-error-rate estimate to cater for various situations of unbalanced attribute distributions. By adopting a linear parametric classifier model, the proposed classification-error based learning formulation is empirically shown to be superior to that using the original least-squares-error cost function. Finally, it will be seen that the performance of the proposed formulation is comparable to other classification-error based and state-of-the-art classifiers without sacrificing the computational simplicity.

A real-time vision system operating at an outdoor swimming pool is presented in this paper. The system is designed to automatically recognize different swimming activities and to detect occurrence of early drowning incidents. We have named this system the Drowning Early Warning System (DEWS). One key challenge we faced in the problem is the relatively high level of noise in the steps of foreground detection and behavior recognition. Therefore, a set of methods in the fields of background subtraction, denoising, data fusion and blob splitting are proposed, which have been motivated by characteristics of aquatic background and crowded scenario at the pool. In the step to detect an early drowning incident, visual indicators of distress and drowning are incorporated through a set of foreground descriptors. A module comprising data fusion and hidden Markov modeling is developed to learn unique traits of different swimming behaviors, in particular, those early drowning events. The experiment of this work reports realistic on-site evaluations performed. Examples of interesting behaviors, i.e., distress, drowning, treading and numerous swimming styles, are simulated and collected. Experimental results show that we have established a prototype system which is robust and beyond the stage of proof-of-concept.

The main advantage of using the Hough Transform to detect ellipses is its robustness against missing data points. However, the storage and computational requirements of the Hough Transform preclude practical applications. Although there are many modifications to the Hough Transform, these modifications still demand significant storage requirement. In this paper, we present a novel ellipse detection algorithm which retains the original advantages of the Hough Transform while minimizing the storage and computation complexity. More specifically, we use an accumulator that is only one dimensional. As such, our algorithm is more effective in terms of storage requirement. In addition, our algorithm can be easily parallelized to achieve good execution time. Experimental results on both synthetic and real images demonstrate the robustness and effectiveness of our algorithm in which both complete and incomplete ellipses can be extracted.

This paper presents a real-time foreground detection method for monitoring swimming activities at an outdoor swimming pool. Robust performance and high accuracy of detecting objects-of-interest are two central issues of concern. Therefore, in this paper, a considerable amount of attention has been placed on the following aspects: 1) to establish a better method of modeling aquatic background, which exhibitis dynamic characteristics with random spatial movements, and 2) to establish a method of enhancing the visibility of the foreground by removing specular reflection at nighttime. First, the development of a new background modeling method is reported. In the proposed approach, the background is modeled as a composition of homogeneous blob movements. With an implementation of a spatial searching process, the proposed method shows capability in associating and distinguishing movements caused by the background. Hence, this contributes to better performance in foreground detection. On the issue of enhancing the visibility of the foreground, a decision-based filtering scheme is proposed as a preprocessing step. A defined concept term, fluctuation measure, is defined for classifying each pixel to be one of the predefined types. This has allowed suitable spatial or spatiotemporal filters to be applied accordingly for color the compensation step. All of these developments are evaluated by testing live on a busy Olympic-size outdoor public swimming pool. Both qualitative and quantitative evaluations are reported. This provides a comprehensive study of the system.

An adaptive spatio-temporal filtering scheme based on a novel concept of motion frequency is proposed as the preprocessing step to enhance human detection under a noisy aquatic environment. In this framework, each pixel is first classified into one of three categories quantified by its motion frequency, each of which is filtered using an appropriate filtering scheme. For regions affected by glistening reflections and glares, a color compensation filter is specially developed to improve partly hidden human detection and minimize errors due to moving background elements. Additionally, a blob-based verification procedure is introduced to remove the target's shadow. Experimental results demonstrate the effectiveness of the algorithm and its role in enhancing the robustness of an aquatic surveillance system for outdoor swimming pools at nighttime.

Automatically understanding events happening at a site is the ultimate goal of visual surveillance system. We investigate the challenges faced by automated surveillance systems operating in hostile conditions and demonstrate the developed algorithms via a system that detects water crises within highly dynamic aquatic environments. An efficient segmentation algorithm based on robust block-based background modelling and thresholding-with-hysteresis methodology enables swimmers to be reliably detected amid reflections, ripples, splashes and rapid lighting changes. Partial occlusions are resolved using a Markov Random Field framework that enhances the tracking capability of the system. Visual indicators of water crises are identified based on professional knowledge of water crises detection, based on which a set of swimmer descriptors has been defined. Through seamlessly fusing the extracted swimmer descriptors based on a novel functional link network, the system achieves promising results for water crises detection. The developed algorithms have been incorporated into a live system with robust performance for different hostile environments faced by an outdoor swimming pool.

Existing state-of-the-art switching-based median filters are commonly found to be nonadaptive to noise density variations and prone to misclassifying pixel characteristics at high noise density interference. This reveals the critical need of having a sophisticated switching scheme and an adaptive weighted median filter. We propose a novel switching-based median filter with incorporation of fuzzy-set concept, called the noise adaptive soft-switching median (NASM) filter, to achieve much improved filtering performance in terms of effectiveness in removing impulse noise while preserving signal details and robustness in combating noise density variations. The proposed NASM filter consists of two stages. A soft-switching noise-detection scheme is developed to classify each pixel to be uncorrupted pixel, isolated impulse noise, nonisolated impulse noise or image object's edge pixel. "No filtering" (or identity filter), standard median (SM) filter or our developed fuzzy weighted median (FWM) filter will then be employed according to the respective characteristic type identified. Experimental results show that our NASM filter impressively outperforms other techniques by achieving fairly close performance to that of ideal-switching median filter across a wide range of noise densities, ranging from 10% to 70%

We observed that certain fundamental concerns commonly exist in some state-of-the-art switching-based median filters: (i) fixed thresholding for the pre-assumed noise density, (ii) the noise decision accuracy at high density impulse noise, and (iii) the filtering scheme adopted in response to pixel characteristic type identified. In this paper, we propose a novel noise adaptive soft-switching median (NASM) filter to effectively address the above-mentioned issues and achieve much improved filtering performance in terms of efficiency in removing impulse noise and robustness against noise density variations. Experimental results also reveal that the performance of our NASM filter is fairly close to that of ideal-switching median filter.

Direct processing of JPEG images based on DCT coefficients could avoid computationally intensive full decoding and large memory storage. In this paper, we exploit the inherent information extracted from DCT coefficients to achieve unsupervised segmentation of image objects. First, a maximum entropy fuzzy clustering (MEFC) algorithm is proposed to achieve a coarse segmentation based on DCT-DC coefficients. The DCT-AC coefficients are then utilized to refine the segmentation boundary by a maximum a posteriori (MAP) approach. The major challenge of the problem is to achieve satisfactory segmentation simply based on DCT coefficients, which are quantized and coarse information in essence. Experimental results show the promising potential of the proposed algorithm in overcoming these fundamental limitations.

As video sequences are composed of dynamic video objects (VOs) in nature, VO's motion is an effective feature used to provide an overall description about the content of video sequences. In this paper, we propose a novel video indexing technique that extracts VO's motion trajectory based on macroblock motion vectors (M-Vs) of MPEG encoded bitstreams, without performing full decoding. This is an attractive approach, as it promises large saving of computation and memory storage. The proposed motion trajectory extraction algorithm comprises two phases: (i) unsupervised VO segmentation and (ii) automatic tracking of each segmented VO. Experimental results reveal promising potential of the established algorithm in extracting VOs' motion trajectories from the limited information provided by MVs.

Motion is recognized as one of the most essential video object (VO) features in indexing video contents. Previously, we directly exploited macroblock motion vectors (MVs) of MPEG bitstream without performing full decoding to extract motion trajectories of VOs. However, the performance of VO segmentation based on MVs is constrained by the limited information extracted from the MV field, since it is quantized and irregular in nature. Therefore, an effective VO tracking scheme would be essential to compensate this limitation. In this paper, we propose a bidirectional motion tracking methodology to (i) firstly validate current segmented VOs by exploring their correlations to the VOs obtained from previous and future frames and (ii) subsequently track each validated VO using discrete Kalman filter. The proposed scheme has been well tested using MPEG-7 test materials and achieves more reliable VO motion trajectory extraction.