Active object tracking adjusts camera poses to keep single target within field-of-view, and therefore is more complex than video object tracking. In this paper, we propose KDAOT, a knowledge driven active object tracking framework in complex scenarios like occlusion. Firstly, we utilize multi-view object appearance with occlusion knowledge. Secondly, We model view angle switches using Markov chain and select effective view angle. Thirdly, we store and predict target movements and adversarial tendency to hide from camera surveillance in trajectory knowledge. Camera pose actions are determined by artificial potential fields of these hybrid knowledge. We construct three meta-verse environments for experiments, namely, Room, Forest and Yard. Results validate the overall effectiveness of our method.

Pedestrian detection in crowded scenes is a challenging problem. To avoid missing targets in a crowd, a high non-maximum suppression (NMS) threshold is necessary. However, a high NMS threshold may lead to a large number of false positives (FP), which results in serious degradation of pedestrian detection performance. In this paper, we propose a novel label assignment scheme for one-stage detectors, called CrowdAssign, which focuses on FP suppression. Firstly, we analyze the sources of FPs and classify them into different categories. Secondly, we examine the cause of different types of FP and develop a dynamic classification label adjustment strategy, which aims to reduce the confidence of the positive samples with high risk of being FPs. With our robust label assignment scheme, a single FCOS-ResNet-50 detector can reach $45.0 \ M R^{-2}$ on CrowdHuman under 1x schedule and outperforms state-of-the-art label assignment methods. Experiments on Citypersons demonstrate the desired generalization ability of our algorithm.

Occlusion is a great challenge for target model update in visual tracking. The target template may be corrupted by non-object information in the process of online learning due to occlusion. In this paper, we propose a context-based active occlusion recognition framework that can be integrated with various tracking approaches. The basic idea is to recognize the occlusion patches by actively tracking context patches and distinguish them from target based on a target model that integrates the information of both target and context. The framework consists of a context patch tracker, an occlusion patch recognizer and a local target template updater. The context patch tracker locates the context patches. The occlusion patch recognizer identifies the context patches occluding target. The local target template updater updates only non-occluded regions of the target template. In this way, the target template not only learns current target features but also avoids context corruption caused by occlusion. The experimental results show that our framework can improve the tracking performance in cases of heavy occlusion.

Bone scintigraphy is widely used to diagnose bone tumor and metastasis. Accurate hotspot segmentation from bone scintigraphy is of great importance for tumor metastasis diagnosis. In this paper, we propose a new framework to detect and extract hotspots in thoracic region by integrating the techniques of both conditional generative adversarial networks (cGAN) and multiple instance learning (MIL). We first use cGAN to train a generator, which can be applied to separate input bone scan image into four anatomical regions and provide location information. A multi-dimensional feature is constructed to integrate contrast, texture and location information. We then use MIL to train a patch-level classifier with this constructed feature. In hotspot segmentation, a hotspots probability map can be estimated with the patch-level classifier. The hotspot segmentation is performed with level set method, in which the hotspot boundary is initialized based on the hotspot probability map. We evaluate the proposed framework quantitatively on the hotspot dataset, and compare it with other methods.

Occlusion is a special challenge in visual tracking, which may cause target template corrupted by background information. In this paper, we propose an adaptive occlusion detection framework for robust tracking against occlusion. The framework consists of a patch tracker, an occlusion detector, a template updater and a search window predictor. The patch tracker applies KCF-based method to track background patch individually, which may occlude target. The occlusion detector searches for background patches occluding target with an adaptive threshold. The template updater evaluates the occlusion state and applies appropriate target template update strategy. The search window predictor adaptively rescales the size of search window based on occlusion state. Experiments in OTB50 demonstrate that our tracker achieves comparable performance compared with other state-of-art trackers and outperforms them in cases of occlusion.

Correlation filter based tracking algorithms have been commonly used in object tracking community. Recently, hand-craft features are replaced by deep convolutional features pre-trained on lager scale image datasets. The low level features with high resolution can locate the position of targets more accurate while the high level features contain more semantic information. In this paper, we construct several single conv-feature correlation filters as weak classifiers. Then, we apply boosting learning method to train a multi conv-features tracker for combining both high resolution features and semantic features. The boosting learner assigns adaptive weights for weak classifiers and the position of target is estimated by the adaptive weighted response map. Experimental results on comprehensive dataset OTB2013 demonstrate that our tracking algorithm can achieve accurate and robust performance compared with baselines and other state-of-art trackers.

Recently, fully convolutional network (FCN) and dilated convolution have shown significantly improvment in semantic segmentation task. Deep residual network (ResNet) has shown strong ability in object recognition. However, FCN-based methods utilize intermediate layers and spatial context information ineffectively. Repeated 2-step striding in ResNet is harmful for segmentation tasks. In this paper, we propose a new semantic segmentation method based on FCN and ResNet. Here, we combine the dilated convolution designed for semantic segmentation with residual unit to enlarge receptive field of ResNet. Meanwhile, “pre-activation” method is used in dilated residual unit. We propose a new segmentation architecture which intergrate multiple intermediate layers and global context information. We extract low-level features of intermediate layers with multi-path refinement which consists of relu-conv unit and chained residual pooling. Global context is gathered by a pyramid pooling method which is connected to the final output of ResNet. Outputs of these 2 modules are then fused to reach high-resolution prediction. In this way, global and local information from pyramid pooling can be enhanced by multi-path refinement. Fully-connected conditional random field is added as a “post-processing” after fusion to receive accurate boundary performance. Our proposed approach achieves 74.7% IoU on Cityscapes benchmark.

Bone scintigraphy is widely used to diagnose bone diseases. Accurate hotspot segmentation is a critical task for tumor metastasis diagnosis. In this paper, we propose an interactive approach to detect and extract hotspots in thoracic region based on a new multiple instance learning (MIL) method called EM-MILBoost. We convert the segmentation problem to a multiple instance learning task by constructing positive and negative bags according to the input bounding box. In order to be robust against noisy input, we train a region-level hotspot classifier with EM-MILBoost and develop several segmentation strategies based on it. The experimental results demonstrate that our method outperforms other methods and is robust against various noisy input.

In recent years, correlation filter based trackers outperform better than other trackers. Nevertheless, they only employ one feature and a single kernel, so they are usually not robust in complex scenes. In this paper, we derive a multi-feature and multi-kernel correlation filter based tracker which fully takes advantage of the invariance-discriminative power spectrums of various features and kernels to further improve the performance. A novel bootstrap learning method is utilized to obtain a strong classifier by fusing these weak kernel correlation filters (KCFs). Moreover, a new target scale estimation strategy is incorporated into our framework. The efficient and effective scale estimation method is based on target dictionary representation. The proposed method is tested on several videos and compared with seven state-of-the-art methods. Experimental results have provided further support to the effectiveness and robustness of the proposed method.

Automatic diagnosis for fetal echocardiography plays an important part in diagnostic aid in the discrimination of congenital heart disease (CHD). Instead of traditional methods analyzing 2D cardiac echo video that need to find the standard view for discrimination, in this paper, we proposed a new system for automatic discrimination of CHD applying 4D original echocardiogram, which avoids the challenging work of searching standard views. We extracted the features of 3D static structure and 3D motion via 3D SIFT and 3D Histogram of Optical Flow (HOF) from the original 4D (3D+T) echocardiogram data, respectively. Bag of Words (BoW) method was employed to construct the quantized feature. Both static and motion features were fused to form the final image representation. One-Class SVM classifier was utilized to discriminate CHD due to the lack of CHD data and the significant difference in all the CHD cases. Experiments on the real data demonstrate the improved discrimination accuracy due to the fused feature.

Hemorrhages in color fundus images usually vary in size and shape, and some are even connected with the retinal vasculature such that they are often omitted by previous detection methods. In this paper, we propose a new method to deal with these problems. During the hemorrhage candidate extraction stage, dark regions and retinal vasculature are segmented out respectively. Their individual advantages are fused together into a binary image with good vascular continuity. Then the gradual removal of vascular branches is applied to generate hemorrhage candidates. In the classification stage, a SVM classifier using 49 features is trained to classify the candidates into hemorrhages and non-hemorrhages. The experiments on DIARETDB1 and DIARETDB0 show that our method achieves good performance on sensitivity and specificity, especially at lesion level.

Superpixels become more and more popular as image preprocessing step in computer vision applications. In this paper, we propose an improved simple linear iterative clustering (SLIC) superpixel approach based on nonstationarity measure (NS-M), which is called nSLIC. An adjustive distance measure is developed in the five-dimensional [labxy] space. The nSLIC superpixel replaces the predefined fixed value of compactness parameter by the nonstationarity measure map of each image, which exploits the image information and is therefore adaptive to the color feature of the image. It also avoids the difficulty of pre-setting compactness parameter and reduces the parameters needed setting to only one indeed. The nSLIC superpixel improves not only segmentation quality bust also computational efficiency by the way of achieving faster convergence. Experiments done on BSD500 dataset show that nSLIC adheres better to image edges meanwhile producing regular and compact superpixels as much as possible, compared to various popular versions of SLIC.

This paper proposes a novel interactive segmentation method based on conditional random field (CRF) model to utilize the location and color information contained in user input. The CRF is configured with the optimal weights between two features, which are the color Gaussian Mixture Model (GMM) and probability model of location information. To construct the CRF model, we propose a method to collect samples for the cuttraining tasks of learning the optimal weights on a single image׳s basis and updating the parameters of features. To refine the segmentation results iteratively, our method applies the active learning strategy to guide the process of CRF model updating or guide users to input minimal training data for training the optimal weights and updating the parameters of features. Experimental results show that the proposed method demonstrates qualitative and quantitative improvement compared with the state-of-the-art interactive segmentation methods. The proposed method is also a convenient tool for interactive object segmentation.

In this paper, we propose a robust visual tracking algorithm based on online learning of a joint sparse dictionary. The joint sparse dictionary consists of positive and negative sub-dictionaries, which model foreground and background objects respectively. An online dictionary learning method is developed to update the joint sparse dictionary by selecting both positive and negative bases from bags of positive and negative image patches/templates during tracking. A linear classifier is trained with sparse coefficients of image patches in the current frame, which are calculated using the joint sparse dictionary. This classifier is then used to locate the target in the next frame. Experimental results show that our tracking method is robust against object variation, occlusion and illumination change.

This paper presents a new fully automatic approach for multi-view urban scene reconstruction. Our algorithm is based on the Manhattan-World assumption, which can provide compact models while preserving fidelity of synthetic architectures. Starting from a dense point cloud, we extract its main axes by global optimization, and construct a nonuniform volume based on them. A graph model is created from volume facets rather than voxels. Appropriate edge weights are defined to ensure the validity and quality of the surface reconstruction. Compared with the common pointcloud- to-model methods, the proposed methodology exploits image information to unveil the real structures of holes in the point cloud. Experiments demonstrate the encouraging performance of the algorithm.

This paper presents a robust method to search for the correct SIFT keypoint matches with adaptive distance ratio threshold. Firstly, the reference image is analyzed by extracting some characteristics of its SIFT keypoints, such as their distance to the object boundary and the number of their neighborhood keypoints. The matching credit of each keypoint is evaluated based on its characteristics. Secondly, an adaptive distance ratio threshold for the keypoint is determined based on its matching credit to identify the correctness of its best match in the source image. The adaptive threshold loosens the matching conditions for keypoints of high matching credits and tightens the conditions for those of low matching credits. Our approach improves the scheme of SIFT keypoint matching by applying adaptive distance ratio threshold rather than global threshold that ignores different matching credits of various keypoints. The experiment results show that our algorithm outperforms the standard SIFT matching method in some complicated cases of object recognition, in which it discards more false matches as well as preserves more correct matches.