• Propose the use of feature fusion to unify contradictory information from different domains for domain adaptation. • Propose the TG-Fusion framework to integrate the target-domain features into the training in the source domain and infer on the target domain without the feature encoder of the source domain. • TG-Fusion is the first attempt to realise training time adaptation of ground-to-UAV and cross-type crop counting. • TG-Fusion demonstrates high robustness across different cross-domain counting adaptation tasks. Domain-adaptive counting aims to accurately estimate the distribution of the objects together with the counting numbers for an unlabelled target domain. In this paper, we propose using feature fusion to unify contradictory information from different domains for domain adaptation. This leads to a scene adaptation framework, Target-Guided Fusion (TG-Fusion), which simplifies and boosts the efficiency of cross-domain counting adaptation. The framework performs especially well with the proposed ground-to-UAV and cross-type crop counting adaptation, of which the datasets are for practical usage and have a small amount of data, yet distinctive inter-domain differences. The approach trains the network to consider the target scene when predicting for the source domain. This is realised by composing a flexible entangle functionality, which dynamically integrates the target features into the training in the source domain, while disentangling the target-domain stream for inference. Moreover, the ratio for the feature fusion is trainable throughout the training time and adjusted by the supervision of the source-domain labels and the similarity among different domains. Empirical experiments demonstrate that the proposed framework trains the model for better adaptation and shows high robustness across different cross-domain counting scenarios.

Tree crown detection plays a vital role in forestry management, resource statistics and yields forecasting. RGB high-resolution aerial images have emerged as a cost-effective source of data for tree crown detection. To address the challenges in the detection using UAV optical images, we propose a one-stage object detection network, TCDNet. First, the network provides an attention enhancement feature extraction module to enable the model to distinguish between tree crowns and their complex backgrounds. Second, an efficient loss is introduced to enable it to be aware of the overlap between adjacent trees, thus effectively avoiding misdetection. The experimental results on two publicly available datasets show that the proposed network outperforms state-of-art networks in terms of precision, recall and mean average precision.

Since remote sensing images of post-fire vegetation are characterized by high resolution, multiple interferences, and high similarities between the background and the target area, it is difficult for existing methods to detect and segment the burned area in these images with sufficient speed and accuracy. In this paper, we apply Salient Object Detection (SOD) to burned area segmentation, the first time this has been done, and propose an efficient burned area segmentation network (BASNet) to improve the performance of unmanned aerial vehicle (UAV) high-resolution image segmentation. BASNet comprises positioning module and refinement module. The positioning module efficiently extracts high-level semantic features and general contextual information via global average pooling layer and convolutional block to determine the coarse location of the salient region. The refinement module adopts the convolutional block attention module to effectively discriminate the spatial location of objects. In addition, to effectively combine edge information with spatial location information in the lower layer of the network and the high-level semantic information in the deeper layer, we design the residual fusion module to perform feature fusion by level to obtain the prediction results of the network. Extensive experiments on two UAV datasets collected from Chongli in China and Andong in South Korea, demonstrate that our proposed BASNet significantly outperforms state-of-the-art SOD methods quantitatively and qualitatively. BASNet also achieves a promising prediction speed for processing high-resolution UAV images, thus providing wide-ranging applicability in post-disaster monitoring and management.

Cross-dataset facial expression recognition (FER) has remained a challenging problem due to the obvious biases caused by diverse subjects and various collection conditions. To this end, domain adaption can be adopted as an effective solution by learning invariant representations across domains (datasets). However, FER requires special consideration of its specific problems e.g., uncertainties caused by ambiguous facial images, and diverse inter- and intra-class relationship. Such uncertainties already exist in single dataset FER, and could be significantly aggravated by enlarged class-wise discrepancies under cross-dataset scenarios. To mitigate this problem, this paper proposes an unsupervised domain adaptation method via regularized conditional alignment for FER, which adversarially reduces domain- and class-wise discrepancies while explicitly dealing with uncertainties within and across domain. Specifically, the proposed method effectively suppresses uncertainties in FER transfer tasks via: 1) semantics-preserving adaptation framework which enforces both domain-invariant learning and class-level semantic consistency between source and target expression data, where discriminative cluster structures are simultaneously retained; 2) auxiliary uncertainty regularization which further constrains the ambiguity of cluster boundaries to guarantee the transferring reliability, thus discouraging the negative transfer brought by divergent facial images. Evaluation experiments on publicly available datasets demonstrate that the proposed method significantly outperforms the current state-of-the-art methods.

Emotion recognition using brain wave signals involves using high dimensional electroencephalogram (EEG) data. In this paper, a window selection method based on mutual information is introduced to select an appropriate signal window to reduce the length of the signals. The motivation of the windowing method comes from EEG emotion recognition being computationally costly and the data having low signal-to-noise ratio. The aim of the windowing method is to find a reduced signal where the emotions are strongest. In this paper, it is suggested, that using only the signal section which best describes emotions improves the classification of emotions. This is achieved by iteratively comparing different-length EEG signals at different time locations using the mutual information between the reduced signal and emotion labels as criterion. The reduced signal with the highest mutual information is used for extracting the features for emotion classification. In addition, a viable framework for emotion recognition is introduced. Experimental results on publicly available datasets, DEAP and MAHNOB-HCI, show significant improvement in emotion recognition accuracy.

A dynamic descriptor facilitates robust recognition of facial expressions in video sequences. The current two main approaches to the recognition are basic emotion recognition and recognition based on facial action coding system (FACS) action units. In this paper we focus on basic emotion recognition and propose a spatio-temporal feature based on local Zernike moment in the spatial domain using motion change frequency. We also design a dynamic feature comprising motion history image and entropy. To recognise a facial expression, a weighting strategy based on the latter feature and sub-division of the image frame is applied to the former to enhance the dynamic information of facial expression, and followed by the application of the classical support vector machine . Experiments on the CK+ and MMI datasets using leave-one-out cross validation scheme demonstrate that the integrated framework achieves a better performance than using individual descriptor separately. Compared with six state-of-arts methods, the proposed framework demonstrates a superior performance.

Existing view-invariant gait recognition methods encounter difficulties due to limited number of available gait views and varying conditions during training. This paper proposes gait partial similarity matching that assumes a 3D object shares common view surfaces in significantly different views. Detecting such surfaces aids the extraction of gait features from multiple views; 3D parametric body models are morphed by pose and shape deformation from a template model using 2D gait silhouette as observation. The gait pose is estimated by a level set energy cost function from silhouettes including incomplete ones. Body shape deformation is achieved via Laplacian deformation energy function associated with inpainting gait silhouettes. Partial gait silhouettes in different views are extracted by gait partial region of interest elements selection and re-projected onto 2D space to construct partial gait energy images. A synthetic database with destination views and multi-linear subspace classifier fused with majority voting is used to achieve arbitrary view gait recognition that is robust to varying conditions. Experimental results on CMU, CASIA B, TUM-IITKGP, AVAMVG, and KY4D data sets show the efficacy of the propose method.

This paper proposes an arbitrary view gait recognition method where the gait recognition is performed in 3-dimensional (3D) to be robust to variation in speed, inclined plane and clothing, and in the presence of a carried item. 3D parametric gait models in a gait period are reconstructed by an optimized 3D human pose, shape and simulated clothes estimation method using multi-view gait silhouettes. The gait estimation involves morphing a new subject with constant semantic constraints using silhouette cost function as observations. Using a clothes-independent 3D parametric gait model reconstruction method, gait models of different subjects with various postures in a cycle are obtained and used as galleries to construct 3D gait dictionary. Using a carrying-items posture synthesized model, virtual gait models with different carrying-items postures are synthesized to further construct an over-complete 3D gait dictionary. A self-occlusion optimized simultaneous sparse representation model is also introduced to achieve high robustness in limited gait frames. Experimental analyses on CASIA B dataset and CMU MoBo dataset show a significant performance gain in terms of accuracy and robustness.

This paper proposes a gait recognition method which is invariant to maximum number of challenging factors of gait recognition mainly unpredictable variation in clothing and carrying conditions. The method introduces an averaged gait key-phase image (AGKI) which is computed by averaging each of the five key-phases of the gait periods of a gait sequence. It analyses the AGKIs using high-pass and low-pass Gaussian filters , each at three cut-off frequencies to achieve robustness against unpredictable variation in clothing and carrying conditions in addition to other covariate factors, e.g., walking speed, segmentation noise, shadows under feet and change in hair style and ground surface. The optimal cut-off frequencies of the Gaussian filters are determined based on an analysis of the focus values of filtered human subject’s silhouettes. The method applies rotation forest ensemble learning recognition to enhance both individual accuracy and diversity within the ensemble for improved identification rate. Extensive experiments on public datasets demonstrate the efficacy of the proposed method.

Facial expression causes different parts of the facial region to change over time and thus dynamic descriptors are inherently more suitable than static descriptors for recognising facial expressions. In this paper, we extend the spatial pyramid histogram of gradients to spatio-temporal domain to give 3-dimensional facial features and integrate them with dense optical flow to give a spatio-temporal descriptor which extracts both the spatial and dynamic motion information of facial expressions. A multi-class support vector machine based classifier with one-to-one strategy is used to recognise facial expressions. Experiments on the CK+ and MMI datasets using leave-one-out cross validation scheme demonstrate that the integrated framework achieves a better performance than using individual descriptor separately. Compared with six state of the art methods , the proposed framework demonstrates a superior performance.

In this paper, a method is presented to label and track anatomical landmarks (e.g., head, hand/arm, feet), which are referred to as significant body points (SBPs), using implicit body models. By considering the human body as an inverted pendulum model, ellipse fitting and contour moments are applied to classify it as being in Stand, Sit, or Lie posture. A convex hull of the silhouette contour is used to determine the locations of SBPs. The particle filter or a motion flow-based method is used to predict SBPs in occlusion. Stick figures of various activities are generated by connecting the SBPs. The qualitative and quantitative evaluation show that the proposed method robustly labels and tracks SBPs in various activities of two different (low and high) resolution data sets.

In this paper, we propose an adaptive image equalization algorithm that automatically enhances the contrast in an input image. The algorithm uses the Gaussian mixture model to model the image gray-level distribution, and the intersection points of the Gaussian components in the model are used to partition the dynamic range of the image into input gray-level intervals. The contrast equalized image is generated by transforming the pixels' gray levels in each input interval to the appropriate output gray-level interval according to the dominant Gaussian component and the cumulative distribution function of the input interval. To take account of the hypothesis that homogeneous regions in the image represent homogeneous silences (or set of Gaussian components) in the image histogram, the Gaussian components with small variances are weighted with smaller values than the Gaussian components with larger variances, and the gray-level distribution is also used to weight the components in the mapping of the input interval to the output interval. Experimental results show that the proposed algorithm produces better or comparable enhanced images than several state-of-the-art algorithms. Unlike the other algorithms, the proposed algorithm is free of parameter setting for a given dynamic range of the enhanced image and can be applied to a wide range of image types.

This paper proposes an algorithm that enhances the contrast of an input image using interpixel contextual information. The algorithm uses a 2-D histogram of the input image constructed using a mutual relationship between each pixel and its neighboring pixels. A smooth 2-D target histogram is obtained by minimizing the sum of Frobenius norms of the differences from the input histogram and the uniformly distributed histogram. The enhancement is achieved by mapping the diagonal elements of the input histogram to the diagonal elements of the target histogram. Experimental results show that the algorithm produces better or comparable enhanced images than four state-of-the-art algorithms.

In this letter, a complex wavelet-domain image resolution enhancement algorithm based on the estimation of wavelet coefficients is proposed. The method uses a forward and inverse dual-tree complex wavelet transform (DT-CWT) to construct a high-resolution (HR) image from the given low-resolution (LR) image. The HR image is reconstructed from the LR image, together with a set of wavelet coefficients, using the inverse DT-CWT. The set of wavelet coefficients is estimated from the DT-CWT decomposition of the rough estimation of the HR image. Results are presented and discussed on very HR QuickBird data, through comparisons between state-of-the-art resolution enhancement methods.

The marching cubes (MC) is a general method which can construct a surface of an object from its volumetric data generated using a shape from silhouette method. Although MC is efficient and straightforward to implement, a MC surface may have discontinuity even though the volumetric data is continuous. This is because surface construction is more sensitive to image noise than the construction of volumetric data. To address this problem, we propose a surface construction algorithm which aggregates local surfaces constructed by the 3-D convex hull algorithm. Thus, the proposed method initially classifies local convexities from imperfect MC vertices based on sliced volumetric data. Experimental results show that continuous surfaces are obtained from imperfect silhouette images of both convex and nonconvex objects.

This paper addresses two challenging issues in unsupervised multiscale texture segmentation: determining adequate spatial and feature resolutions for different regions of the image, and utilizing information across different scales/resolutions. The center of a homogeneous texture is analyzed using coarse spatial resolution, and its border is detected using fine spatial resolution so as to locate the boundary accurately. The extraction of texture features is achieved via a multiresolution pyramid. The feature values are integrated across scales/resolutions adaptively. The number of textures is determined automatically using the variance ratio criterion. Experimental results on synthetic and real images demonstrate the improvement in performance of the proposed multiscale scheme over single scale approaches.

In the context of real-time collision/obstacle avoidance this paper investigates an improvement of a direct method of estimating dense scene structure and image motion. Our proposed method combines stereo-vision (involving geometric constraints), optical flow (involving photometric constraints) and additional model constraints. Modifications to the direct method are introduced through a planarity constraint, an illumination model and a new computational framework in order to deal with significant illumination drifts and obstacle detection on static objects or moving vehicle. Experimental results on real image sequences demonstrate the effectiveness of the method.

This paper proposes a technique that uses a planar calibration object and projective constraints to calibrate parametric and nonparametric distortions. An iterative surface fitting is first used to constrain a B-spline distortion model by fusing the corrective distortion maps obtained from multiple views. The model is then refined within the whole camera-calibration process.

This paper presents a scheme that addresses the practical issues associated with producing a geometric model of a scene using a passive sensing technique. The proposed image-based scheme comprises a recursive structure recovery method and a recursive surface reconstruction technique. The former method employs a robust corner-tracking algorithm that copes with the appearance and disappearance of features and a corner-based structure and motion estimation algorithm that handles the inclusion and expiration of features. The novel formulation and dual extended Kalman filter computational framework of the estimation algorithm provide an efficient approach to metric structure recovery that does not require any prior knowledge about the camera or scene. The newly developed surface reconstruction technique employs a visibility constraint to iteratively refine and ultimately yield a triangulated surface that envelops the recovered scene structure and can produce views consistent with those of the original image sequence. Results on simulated data and synthetic and real imagery illustrate that the proposed scheme is robust, accurate, and has good numerical stability, even when features are repeatedly absent or their image locations are affected by extreme levels of noise.

In the context of real-time collision-obstacle avoidance this paper investigates a direct method of estimating dense scene structure and image motion. Our proposed method combines both stereo vision (involving geometric constraints) and optical flow (involving photometric constraints). It is applied to multiple motion images (i.e., images with vehicle ego-motion plus obstacle motion). Modifications to the direct method were introduced to improve the reliability of the motion estimation. The multiple-motion-estimation scheme comprises local motion estimations followed by a classification of these estimates in a simplified motion parameter space through cluster analysis. The classification enables segmentation of the different motions that are used to estimate more accurately the dense structure and motion of objects in a scene. Experimental results on both synthetic and real image sequences demonstrate the potential of the method.

This paper presents a method for optical flow estimation and segmentation through polynomial surface fitting, robust least-median-squares regression, and robust statistic clustering mean shift. This approach consists of three stages. First, a standard polynomial surface fitting is used to smooth an image, and least-median-of-squares (LMedS) robust regression is used to calculate the optical flow, which can tolerate up to 50% outlier contamination. Second, the estimated optical flow map is segmented through a mean shift technique. Third, an affine flow model is employed to fit the coarse flow estimates within the segmented regions, and the affine fitted motion of the regions is refined with a robust least median squares process based on optical flow constraints. The experimental results have demonstrated that our approach achieved good performance in most synthetic and real video sequences.

Presents a novel method which uses a special type of multi-scale isotropic band-pass filters to detect image landmark and corner features. This paper mainly contributes to the solutions of the following problems: (1) defining a general family of feature detectors under a unified framework that are able to enhance and detect the desired features; (2) explaining theoretically and experimentally why these feature points can be detected by the proposed detectors; and (3) extending the detectors to multi-scale versions for jointly achieving good detectability and localization where an automatic scale selection method is applied. The paper then presents several applications for detecting landmarks and corners using the proposed methods, in order to illustrate their usage. These include detecting landmarks from gesture images (of the face and hand), from airborne and vehicle-borne IR landmine images, and from images containing object corners. Experiments have been performed using the proposed detectors to these applications. Some comparisons and evaluations have also been performed. The results have demonstrated the effectiveness of the proposed detectors in terms of feature detectability and localization.

Most existing work on the recovery of scene structure from an image sequence assume prior knowledge of the camera or do not attempt to recover metric geometry, and that feature points have been extracted with the required correspondences established, often by human operators. This paper presents a scheme that removes the above assumptions. Using an extended Kalman filter (EKF) framework the scheme performs simultaneous comer feature tracking with correspondence analysis, and the reconstruction of three-dimensional (3D) rigid objects. The paper introduces the concept of individual global rotation quaternions and translational offset vectors so that it is not necessary for all features associated with an object to be present in the first frame of the image sequence Experiments on two image sequences containing a soccer ball and a model house show the potential and effectiveness of the scheme.

This paper addresses the problems of high spatial resolution in the estimated optical flow velocity vector map and robustness to large object displacement. We present a novel multi-resolution adaptive shifting method (MRAS), which uses variable image region shifts to refine velocity vectors calculated using the optical flow constraint equation at each level in a multi-resolution Gaussian pyramid. Furthermore an overlapping window technique is presented to overcome the spatial resolution limitations of the multi-point approach. The performance of MRAS is compared to a published multi-resolution clustering (MRC) technique, for two image sequences. Significant improvements in robustness with respect to large displacements is shown for MRAS compared to MRC, with little degradation in velocity vector accuracy as the magnitude of the displacement increases.

This paper addresses the problems of object corner detection and the applications of corners to affine-similar object matching for object retrieval from a database. Two simple and efficient corner detection methods are presented. One is based on detecting sharp angles on a smoothed object boundary curve, and the other is based on a 2D rotationally symmetric bandpass filter applied onto an image. Two object matching approaches which exploit these corners are then investigated. In the first approach, affine objects are retrieved by matching their features derived from the corners. Dissimilarity measures for objects are formulated based on these features. In the second approach, affine objects are retrieved by matching object boundary curves modeled by a constrained active B-spline curve model. The model differs from the conventional B-spline by retaining significant object corners as a subset of B-spline knot points. This enables better correspondences of corners and easy selection of starting points for modeled curves, and hence a more accurate affine object matching. Various images and object boundary curves were used to verify the proposed methods, and the experimental results are convincing.

In this paper, we consider the problem of matching 2D planar object curves from a database, and tracking moving object curves through an image sequence. The first part of the paper describes a curve data compression method using B-spline curve approximation. We present a new constrained active B-spline curve model based on the minimum mean square error (MMSE) criterion, and an iterative algorithm for selecting the "best" segment border points for each B-spline curve. The second part of the paper describes a method for simultaneous object tracking and affine parameter estimation using the approximate curves and profiles. We propose a novel B-spline point assignment algorithm which incorporates the significant corners for interpolating corresponding points on the two curves to be compared. A gradient-based algorithm is presented for simultaneously tracking object curves, and estimating the associated translation, rotation and scaling parameters. The performance of each proposed method is evaluated using still images and image sequences containing simple objects.

This paper shows that in edge detection the regularization factor alpha is a better scale parameter than the standard deviation (sigma) of the Gaussian pre-filter. The alpha scale space, which exhibits the evolutionary behaviour of an edge in various scales, is the basis for the design of a multiscale edge detector (MRCBS). In MRCBS, the scale is determined adaptively according to the local noise level; the thresholds which control the amount of edge details are adjusted according to the scale; and the anisotropic diffusion is applied in the finest scale to further suppress noise.

One problem in tracking an object through video frames and estimating the associated motion parameters is that the object of interest may undergo various affine transformations and a small deformation. This paper presents a multi-stage algorithm for tracking moving objects using joint segmentation and B-spline contour modelling and motion parameters estimation. The segmentation algorithm uses merging thresholds which are based on the local image property. A fast gradient-based B-spline contour matching algorithm and a robust knot re-assignment method are proposed for tracking moving objects through image frames and estimating their motion parameters. The application of the scheme on images of hand and tool movements, and an image sequence containing a screwdriver with affine transformations shows that the estimated parameters are in very close agreement with the actual parameters.

Interreflection changes image intensities in a consistent, non-random way, and if unaccounted for can easily confuse computer vision algorithms. This paper proposes an image model which enables regions of interreflection on objects of inhomogeneous dielectric materials to be identified. The model is based on the dichromatic reflection model and utilises the concept of the one-bounce model of mutual reflection between two object surfaces. One surface is viewed as a source of low-intensity illumination and the second as a function of the light emitted from the first. The proposed model indicates the presence of an additional matte cluster which corresponds to the region of interreflection.

X-ray rocking curve analysis is widely used in research and industry to investigate the perfection of a variety of natural and synthetic crystals. In this article a method is demonstrated for the effective self-evaluation of an expert system for x-ray rocking curve analysis. the method uses a combination of fuzzy logic and machine learning, the latter defined as a self-adaptive system that improves system performance over time. the heuristics of several experts are combined using rules, frames, and connection matrices. Each expert is weighted on the basis of experience and these credibility weights are used to influence the decision processes of the expert system. All weights are evaluated over time and the basis for evaluation is successful or unsuccessful expert system decisions. Individual rules are also evaluated and whenever a rule is shown to be ineffective it is hidden from the reasoning processes of the expert system. When new situations occur that have not been allowed for in the rules of the expert system, then existing rules are fine-tuned and changed to deal with these new facts. New rules are inferred and evaluated in the same way as the heuristics of the human experts. © 1994 John Wiley & Sons, Inc.

This paper discusses user modelling techniques and presents the design and implementation of the 3M user modelling interface of INTEREX. INTEREX is an expert system for X-ray topographic image interpretation which assists its users in identifying and analysing a number of defects that can occur in high-quality crystals. 3M consists of a monitor, a model and a modifier. It is used to adapt the consultation route and the explanations provided by INTEREX to three categories of users. It demonstrates the use-of an implicit, individual, dynamic and long-term user model to enable an expert system to accommodate users with different levels of expertise.