Few-shot segmentation aims to recognize unseen objects from a few examples per class. It divides the training process into episodes with a support set and a query set. The support set provides annotated examples, and the query set contains images for annotation. Most models follow a prototypical approach, which extracts representative prototypes from support images to compare against query image features. PFENet proposed using prior maps to focus models on likely foreground areas, which effectively improves the model’s performance and is widely used in subsequent methods. However, existing prior-based techniques solely derive priors from high-level features. High-level features tend to lack detailed structure information due to the cumulative effects of convolutions and pooling operations, which leads to their poor performance in recognizing the boundaries of foreground areas. The uncertainty of high-level priors near boundaries may erroneously introduce category-irrelevant noise, as shown in Fig. 1. Moreover, high-level priors often overly focus on the most relevant parts, potentially overlooking other representative structures and losing semantic cues.

Human detection has received great attention during the past few decades, which is yet still a challenging problem. In this paper, we focus on the problem of 3-D human detection, i.e., finding the human bodies and determining their 3-D coordinates in complex 3-D space using depth data only. Since the traditional sliding-window-based approaches for target localization are time-consuming and the recent deep-learning-based object detectors generate too many region proposals, we propose to utilize the candidate head-top locating stage to efficiently and quickly find the plausible head-top locations. In the second stage, we propose a Depth map, Multiorder depth template, and Height difference map representation encoding three channels of information for each candidate region to utilize the neural network pretrained on large-scale well-annotated datasets to classify the candidate regions. We evaluate our method on four publicly available challenging datasets. Extensive experimental results demonstrate that the proposed method is superior to the state-of-the-art methods while achieving real-time performance.

This paper proposes a reliable 3D fish tracking method using a novel master-slave camera setup. Instead of conventional dynamic models that rely on prior knowledge about target kinematics, the proposed method learns the kinematic model with a Long Short-Term Memory (LSTM) network. On this basis, the 3D state of fish at each moment is predicted by LSTM network. We propose to use an innovative master-view-tracking-first strategy. The fish are first tracked in the master view. Cross-view association is then established utilizing motion continuity and epipolar constraint cues. Experiments on data sets of different fish densities show that the proposed method is effective and outperforms the state-of-the-art methods.

We present a Temporal Context Network (TCN) for precise temporal localization of human activities. Similar to the Faster-RCNN architecture, proposals are placed at equal intervals in a video which span multiple temporal scales. We propose a novel representation for ranking these proposals. Since pooling features only inside a segment is not sufficient to predict activity boundaries, we construct a representation which explicitly captures context around a proposal for ranking it. For each temporal segment inside a proposal, features are uniformly sampled at a pair of scales and are input to a temporal convolutional neural network for classification. After ranking proposals, non-maximum suppression is applied and classification is performed to obtain final detections. TCN outperforms state-of-the-art methods on the ActivityNet dataset and the THU-MOS14 dataset.

This paper presents a computer-aided system for automatic diagnosis of cirrhosis based on ultrasound images. We first propose a dynamic programming algorithm to automatically extract the liver capsule, and then the continuity and smoothness of capsule serve as an important guideline for image classification. Via the decomposition of the ultrasound image in spatial and gray scales, the density and entropy of suspected nodular areas are used to describe texture features of liver parenchyma. Finally, a trained SVM classifier is applied to classify the samples into normal, mild, moderate and severe clinical stages of the disease. Experiment results show that the proposed method achieves better performance than existing approaches. Moreover, it can be used as an efficient method for early cirrhosis diagnosis in consideration of its high accuracy in distinguishing between normal and abnormal cases.

Efficient and robust detection of humans has received great attention during the past few decades. This paper presents a two-staged approach for human detection in RGB-D images. As the traditional sliding window-based methods for target localization are often time-consuming, we propose to use the super-pixel method in depth data to efficiently locate the plausible head-top locations in the first stage. In the second stage, we propose to use Random Ferns to seek the features by combining information from different image spaces, which can select the most discriminative features and compute simple and fast Local Binary Features (LBFs) allowing for real-time applications. We evaluate our method on three publicly available challenging datasets taken by a Kinect camera. Experimental results demonstrate that the proposed approach can robustly detect humans in complicated environments.

Accurately and reliably tracking the 3D position and orientation of individuals in large flying swarms is valuable not only for scientific researches but also practical applications. However, large quantity, frequent occlusions, similar appearance, tiny body size and abrupt motion make it remain an open problem. The 3D flying swarm tracking method proposed in this paper tracks both position and orientation of each individual in the swarm using the particle filter framework. Particles are scattered more pertinently by the dynamic model based on the learned kinematic pattern of a single target with a Long Short-Term Memory (LSTM) network. In addition, the observation model combines the Weighted Occupancy Ratio (WOR) and Temporal Appearance Coherency (TAC) cues in each view to improve the accuracy and robustness of the reconstructed body orientation. Experiments on both simulation and real-world data sets demonstrate the effectiveness and superiority of the proposed method.

Real-time human detection is important for a wide range of applications. In this paper, a two-staged method has been developed for real-time human detection in cluttered and dynamic environments with depth data. We start with generating a set of possible human head-tops to ensure all human locations are included. To this end, a novel physical radius-depth (PRD) detector is proposed to quickly detect human candidates. The second stage applies a convolutional neural network (CNN), aiming at extracting feature of human upper body automatically instead of hand-crafting, and then on the basis of CNN feature genuine human candidates are preserved while false ones are filtered out. Experiment results on four publicly available datasets, including a dataset under weak illumination or even total darkness, show that the proposed method can reliably detect human in RGB-D videos in real time without GPU acceleration, and yields higher accuracy than the compared state-of-the-art approaches.

We propose a multistream multitask deep network for joint human detection and head pose estimation in RGB-D videos. To achieve high accuracy, we jointly utilize appearance, shape, and motion information as inputs. Based on the depth information, we generate scale invariant proposals, which are then fed into a novel contextual region of interest pooling (CRP) layer in our deep network. This CRP has two branches to deal with contextual information for each subject. The proposed method outperforms state-of-the-art approaches on three public datasets.

3D motion data of fish school is more valuable than 2D data for behavior and other researches. This paper proposes to use a master view tracking first strategy based on a novel master-slave camera setup. On this basis, fish are firstly tracked in master view in 2D after being extracted via an eye-focused Gaussian Mixture Model (E-GMM) detector. Then 3D trajectories are reconstructed by associating 2D tracking results in master view and detection results in slave views after fish in slave views are localized using an eye-focused Gabor (E-Gabor) detector. Experiments on data sets with different fish densities demonstrate that the proposed method outperforms two state-of-the-art methods in terms of 5 evaluation metrics.

This paper proposes a novel cirrhosis diagnosis method using high-frequency ultrasound imaging that is able to not only diagnose cirrhosis, but also determine its stage. We propose combined features extracted from both liver capsule and parenchyma texture to avoid the bias caused by considering only one aspect. The liver capsule is localized using a multi-scale, multi-objective optimization method and indices are proposed to measure the smoothness and continuity of the capsule. The parenchyma texture is modeled with Gaussian mixture model (GMM), and the lesions in the parenchyma are detected by a scale-space defect detection algorithm. The degree of pathological changes of the liver is quantitatively evaluated by 7 features describing morphology of the capsule and lesions in the parenchyma. Then SVM classifiers are trained to classify the samples into different cirrhosis stages. Experiment results demonstrate the effectiveness of the proposed method, which outperforms other 4 state-of-the-art methods and the proposed method that solely uses capsule or parenchyma texture features.

Perception of the presence and position of human is crucial for many kinds of Artificial Intelligence (AI) applications. In this paper, we have developed a novel two-staged method for realtime human detection in depth image. The first stage is to quickly scan through the image to detect possible head-top locations in order to ensure all the candidate locations are included. The second stage is to use a novel head-shoulder descriptor (HSD) which jointly encodes the One-hot Depth Difference information and local geometric characteristics of human upper body to filter the detections so as to keep the genuine human locations and discard false positives. The results show that our approach using only depth data is superior to other methods using color and depth images on four datasets. In addition, our method performs well under weak illumination conditions or even total darkness. Moreover, our system is also able to run in real-time on conventional PC without GPU acceleration.

Multi-camera three-dimensional (3D) tracking technology is the most effective gateway to acquire quantitative motion data for studying behaviors of flying swarms. While the state-of-the-art tracking methods usually estimate positions of flying targets, here we present the 3D tracking method which estimates both a target's position and its orientation for tracking individuals of flying swarms, in which videos captured by at least three synchronized and calibrated cameras are necessary. Experiments show the proposed method outperforms the state-of-the-art methods.

This paper proposes a two-staged approach to real-time human detection in cluttered environments using RGB-D camera. The first stage is a novel physical blob (P-Blob) detection that can quickly find plausible human heads. The second stage uses a combination of human upper-body features to filter out false positives. Experiment results on three publicly available datasets show that the proposed method can reliably detect people in RGB-D video in real time.

Object tracking is a key step of video analysis, while a motion model is crucial for object tracking. Concerning videos captured with fixed cameras, a sequence of a target's motion data may suggest the target's kinematic model with respect to the imaging system. In this paper we model the target's kinematic model by learning a long short-term memory network. This kinematic model can serve as a discriminative model and determine the probability of a sequence of velocities. In order to improve the expressive ability of the kinematic model, we partition units of the network into groups and activate groups at different temporal resolutions. With this improvement the kinematic model can also describe the abrupt motion of targets. We have conducted experiments to evaluate the performance of the proposed method, using both a fish tracking method and state-of-the-art tracking methods.

Automatically and reliably tracking numerous flying objects in 3D space is of great significance for not only scientific researches such as collective behavior analysis, but also practical applications such as designing multi-agent robots. However, it remains a challenging task due to the large population, similar appearance, and severe occlusion happening in 2D images. This paper proposes a 3D tracking method that is capable of tracking individuals of a swarm of flying objects using the particle filtering technique. Each particle is not only weighted by the observation model but also weighted by the kinematic model. The kinematic model is modeled by learning a long short-term memory network on sequences of velocities. Experimental results show that the kinematic model significantly improves the efficiency of estimating target's motion state, and show that the proposed method outperforms the state-of-the-art methods.

This paper proposes a novel method to extract and describe liver capsule contour in high-frequency ultrasound image for early diagnosis of HBV cirrhosis. The proposed method combines vertical gradient optimization and minimum of deflection error to approximate the outline of liver capsule using a number of reference points determined through interactive selection. We also propose a Continuity of Capsule (CoC) index and a Smoothness of Capsule (SoC) index to describe the geometric characteristics of the liver capsule. Experiments show that the proposed method can accurately locate the outline of the liver capsule, and the CoC index is effective in detecting the existence of HBV cirrhosis especially in the early stages, and the SoC index has statistical significance in classifying among normal, mild, moderate and severe clinical stages of the disease.

Real-time human detection in crowded and dynamic environments poses a significant challenge, due to complex background, occlusion and different human poses. In this paper, we propose a two-staged approach using color and depth data taken by an RGB-D camera. The first stage is to find plausible head-top locations quickly in depth image. The second stage is to extract effective discrimination features from RGB-D data to discard the false positives with support vector machine. The experiments on the publicly available office dataset, mobile platform dataset and clothing store dataset show that the system achieves accurate and real-time detection results on a single CPU core and its performance is significantly superior to state-of-the-art methods.

Accurately and reliably tracking the undulatory motion of deformable fish body is of great significance for not only scientific researches but also practical applications such as robot design and computer graphics. However, it remains a challenging task due to severe body deformation, erratic motion and frequent occlusions. This paper proposes a tracking method which is capable of tracking the midlines of multiple fish based on midline evolution and head motion pattern modeling with Long Short-Term Memory (LSTM) networks. The midline and head motion state are predicted using two LSTM networks respectively and the predicted state is associated with detections to estimate the state of each target at each moment. Experiment results show that the system can accurately track midline dynamics of multiple zebrafish even when mutual occlusions occur frequently.

BACKGROUND: Fish tracking is an important step for video based analysis of fish behavior. Due to severe body deformation and mutual occlusion of multiple swimming fish, accurate and robust fish tracking from video image sequence is a highly challenging problem. The current tracking methods based on motion information are not accurate and robust enough to track the waving body and handle occlusion. In order to better overcome these problems, we propose a multiple fish tracking method based on fish head detection. RESULTS: The shape and gray scale characteristics of the fish image are employed to locate the fish head position. For each detected fish head, we utilize the gray distribution of the head region to estimate the fish head direction. Both the position and direction information from fish detection are then combined to build a cost function of fish swimming. Based on the cost function, global optimization method can be applied to associate the target between consecutive frames. Results show that our method can accurately detect the position and direction information of fish head, and has a good tracking performance for dozens of fish. CONCLUSION: The proposed method can successfully obtain the motion trajectories for dozens of fish so as to provide more precise data to accommodate systematic analysis of fish behavior.

Mitochondria play an important role in cellular physiology and synaptic function. Recent electron microscopy (EM) advances make it possible to observe mitochondrial structure on nanoscale, but the attendant massive EM data unfortunately requires months of tedious manual labor. In this paper, we present an automatic approach for the 3D reconstruction of mitochondria from anisotropic EM stack. We first extract a novel local intensity distribution signature (LIDS) feature and learn a random forest classifier (RF) in x-y directions to obtain coarse superpixels. A random disjoint-set forest algorithm can then cluster these superpixels into supervoxels. Next, we use a 3D erosion and dilation method to discard unsatisfying structures, e.g., neural membrane or synapse. At last, the second random forest classifier is learned combining with mitochondrial shape and texture features, which can select the real mitochondria effectively. The confidence values are given to help human experts decide which mitochondrion to be reviewed first. We evaluate the proposed approach on two different anisotropic EM stacks of drosophila brain and compare against the current state-of-the-art methods.

This paper presents a new approach to real-time human detection and tracking in cluttered and dynamic environments by integration of RGB and depth data. We introduce the notion of Point Ensemble Image, which fully encodes both RGB and depth information from a virtual plan-view perspective, and we reveal that human detection and tracking in 3D space can be performed very effectively based on this new representation. Our human detector is able to take advantage of depth data by effectively locate physically plausible candidates as a first step, and then both depth and color information is made full use of in a supervised learning manner at the second stage. 3D trajectories of humans are finally generated by data association in which joint statistics of color and height are computed and compared. Experimental results show that the system is able to work satisfactorily in complex real-world situations.

This paper addresses the problem of reconstructing the 3D motion trajectories of particle swarms using two temporally synchronized and geometrically calibrated cameras. The 3D trajectory reconstruction problem involves two challenging tasks - stereo matching and temporal tracking. Existing methods separate the two and process them one at a time sequentially, and suffer from frequent irresolvable ambiguities in stereo matching and in tracking. We unify the two tasks, and propose a Global Correspondence Selection scheme to solve stereo matching and temporal tracking simultaneously. It treats 3D trajectory acquisition problem as selecting appropriate stereo correspondences among all possible ones for each target by minimizing a cost function. Experiment results show that the proposed method has significant performance advantage over existing approaches.

When classes are nonseparable or overlapping, training samples in a local neighborhood may come from different classes. In this situation, the samples with different class labels may be comparable in the neighborhood of query. As a consequence, the conventional nearest neighbor classifier, such as kappa-nearest neighbor scheme, may produce a wrong prediction. To address this issue, in this paper, we propose a new classification method, which performs a classification task based on the local probabilistic centers of each class. This method works by reducing the number of negative contributing points, which are the known samples falling on the wrong side of the ideal decision boundary, in a training set and by restricting their influence regions. In classification, this method classifies the query sample by using two measures of which one is the distance between the query and the local categorical probability centers, and the other is the computed posterior probability of the query. Although both measures are effective, the experiments show that the second one achieves the smaller classification error. Meanwhile, the theoretical analyses of the suggested methods are investigated, and some experiments are conducted on the basis of both constructed and real datasets. The investigation results show that this method substantially improves the classification performance of the nearest neighbor algorithm.

Most 3D reconstruction solutions focus on surfaces, and there has not been much research attention paid to the problem of reconstructing 3D scenes made up of large numbers of particles, while the ability to reconstruct such dynamic scenes is potentially very useful in many areas such as colony behavior research and visual modeling. This paper proposes an approach - relative epipolar motion (REM) - towards solving the correspondence problem in stereopsis by utilizing the motion clue. It matches feature trajectories instead of the features themselves as used by existing methods. The proposed method has the following new capabilities: (1) it supports reconstructing dynamic 3D scenes of large number of undistinguishable drifting particles; (2) It is applicable to correspondence establishment for dynamic surfaces made up of repetitive textures; (3) It offers an alternative way to project structured light in active mode for deforming surface reconstruction. Experiment results on both simulated and real-world scenes demonstrate its effectiveness.

Deriving from the artificial life theory, this paper proposes an artificial co-evolving tribes model and applies it to solve the image segmentation problem. During the evolution process, the individuals in this model making up the tribes effect communication cooperatively from one agent to the other in order to increase the homogeneity of the ensemble of the image regions they represent. Two remarkable properties, that is, the monotone contraction and the conservation of the system are proved. Stability and scale control of the proposed method are carefully analyzed. Experimental results are presented and compared with two latest segmentation methods, both quantitatively and visually. We also discuss the results matching with human visual perception.

This paper proposes a novel shape representation scheme — Interior Angle Chain (IAC) — which is invariant to translation, rotation and scaling. The proposed method first approximates the contour of a planar shape with an equilateral polygon and then makes a representation using the polygon's interior angle chain. The difference between two shapes is measured by the distance between their IAC's. An algorithm to obtain equilateral polygon approximation and its associated IAC is proposed in this paper. The proposed shape representation scheme has been tested on two benchmarks and applied to lake recognition in SAR (Synthetic Aperture Radar) images. The results show that IAC is an effective shape representation scheme.

This paper presents a novel approach to image texture classification, which involves a model of artificial organisms i.e. Artificial Crawlers (ACrawlers) and a series of evolution curves representing the features of the texture. The distributed ACrawlers locally interact with their living environment, i.e. textured regions, and each ACrawler acts according to a set of homogenous rules for isotropic motion, energy absorption and colony formation, etc. The ACrawlers evolve through natural selection, which produces the specific curves of agent evolution, habitant settlement and colony formation as well as the scale distribution of all colonies. The feasibility and effectiveness of the proposed method have been demonstrated by experiments.

This paper deals with the problem of matching curves and structures extracted from 2D images that are subject to translation, rotation, scaling and other geometric transformations. We present in this paper a novel approach, Connected Equi-Length Line Segments (CELLS), for curve representation and matching. In our framework, a curve is represented by a number of connected equi-length line segments and a new matrix called Orientation Difference Matrix (ODM) is constructed for the curve, which reflects the distribution of the rest of the line segments with respect to the current one using orientation differences between them. The representation is invariant to rotation, scaling and translation. The problem of structure matching is also considered in this paper and is solved based on CELLS. The matching of structures is performed by (1) detecting tri-junctions and quad-junctions on the structures, (2) representing each arch using CELLS. A practical use of the proposed approach is demonstrated by registering a SAR image of a certain area to a map.

The harmonic transform is designed for harmonic signals, which are composed of a base tone and some harmonics (e.g., voiced speech). As a generalization of the Fourier transform, the harmonic transform represents signals by the sum of a base tone and some harmonics, which may give more concise results for harmonic signals than the Fourier transform. Some experiments of speech signals are used to demonstrate the advantages of the harmonic transform on harmonic signal processing.

Artificial life has been successfully used for understanding biological systems and in many applications in robotics, computer graphics, etc. In this paper, we pioneer the use of artificial life for image segmentation, a challenging area in image processing. Our method associates each pixel in an image with a life and it evolves according to a system of rules. The segmented partitions emerge when the state of the lives reaches an equilibrium. The artificial life approach is promising in image processing because it is inherently parallel and coincides with the self-governing biological process. In addition, it has the advantage of the integration of both detail preservation and noise removal. The experiments demonstrate the feasibility of the artificial life approach on both intensity images and color images. We also compared the approach with other four commonly used methods for three different kinds of noise corrupted images.

A novel adaptive harmonic fractional Fourier transform is proposed for analysis of voiced speech signals. It provides a higher concentration than STFT and avoids the cross interference components produced by the Wigner-Ville distribution and other bilinear representation. The proposed method rotates the base tone and harmonics in time-frequency domain. After the rotation, base tone and harmonics become parallel to the time axis in time-frequency domain so that a high concentration can be achieved.

An alternative novel artificial neural network (ANN) for the purpose of dynamic nonlinear system identification is proposed. The main drawback of feedforward neural networks such as a multi-layer perceptron (MLP) trained with backpropagation (BP) algorithm is that it requires a large amount of computation and the rate of error convergence is slow. The proposed Chebyschev functional link ANN (C-FLANN) is found to have much less computational requirement and its performance is found to be superior to that of a MLP for the complex task of nonlinear dynamic system identification, even in the case of additive input noise to the system.

Multilayer perceptrons with hard-limiting (signum) activation functions can form complex decision regions. It is well known that a three-layer perceptron (two hidden layers) can form arbitrary disjoint decision regions and a two-layer perceptron (one hidden layer) can form single convex decision regions. This paper further proves that single hidden layer feedforward neural networks (SLFN's) with any continuous bounded nonconstant activation function or any arbitrary bounded (continuous or not continuous) activation function which has unequal limits at infinities (not just perceptrons) can form disjoint decision regions with arbitrary shapes in multidimensional cases. SLFN's with some unbounded activation function can also form disjoint decision regions with arbitrary shapes.

A new approach, designed for clustering data whose underlying distribution shapes are arbitrary, is presented. This study is concerned with the use of the skeleton of a cluster as its prototype, which can represent the cluster more closely than that of using a single data point. The given data set is then partitioned into those skeleton-represented clusters without any prior knowledge nor assumptions of hidden structures. A novel function called cluster characteristic function (CCF) has been constructed and the associated theorems have been proposed and proved that the proper number of clusters can be determined with the approach.

Clustering a data set that exhibits arbitrary shapes is an important research area in unsupervised data labeling. The paper is concerned with using a shape-matched curve as the prototype of that cluster. Data points are then labeled by those curve-represented clusters without any preknowledge and priori assumptions of hidden structures in the data set. Since the shapes of these curves are often dendritic, we call this learning process dendritic curve clustering (DCC). DCC makes use of fuzzy c-means, and each shape-matched cluster curve is achieved through cluster growing. Experimental results demonstrate the DCC approach is feasible.

The fractal dimension has been studied as a feature for texture analysis. It has been found that the fractal dimension is not an effective image texture measure but little is known about the reasons for the fractal dimension failing to be effective for texture analysis. This paper investigates into the underlying causes why the fractal dimension is not an effective image texture feature. Four mathematical properties have been identified which are responsible for the fractal dimension's ineffectiveness. The experimental results show that while the fractal dimension itself is hardly an effective feature for texture classification, it can considerably enhance other feature sets.

A novel adaptive harmonic fractional Fourier transform is proposed for analysis of voiced speech signals. It provides a higher concentration than the short time Fourier transform (STFT) and avoids the cross interference components produced by the Wigner-Ville distribution and other bilinear representations. The proposed method rotates the base tone and harmonics in time-frequency domain. After the rotation, base tone and harmonics become in parallel to the time axis in time-frequency domain so that a high concentration can be achieved.

This paper presents a new texture feature set based on the statistics of geometrical attributes of connected regions in a stack of binary images obtained from a texture image. Systematic evaluation using all the Brodatz textures shows that the new set outperforms the well-known statistical gray level dependence matrix, the recently proposed statistical feature matrix, and Liu's features.>