We propose an innovative method for combining sparse optical flow tracking and descriptor matching algorithms. The proposed approach solves the following problems that are inherent to keypoint-based and optical flow based tracking algorithms: spatial jitter, extreme scale transformation, extreme perspective transformation, degradation in the number of tracking points, and drifting of tracking points. Our algorithm provides smooth object-position tracking under six degrees of freedom transformations with a small computational cost for providing a high-quality real-time AR experience on mobile platforms. We experimentally demonstrate that our approach outperforms the state-of-the-art tracking algorithms while offering faster computational time. A mobile augmented reality (AR) application, which is developed using our approach, delivers planar object tracking with 30 FPS on modern mobile phones for a camera resolution of 1280×720. Finally, we compare the performance of our AR application with that of the Vuforia-based AR application on the same planar objects database. The test results show that our AR application delivers better AR experience than Vuforia in terms of smooth transition of object-pose between video frames.

Deep learning has undergone tremendous advancements in computer vision studies. The training of deep learning neural networks depends on a considerable amount of ground truth datasets. However, labeling ground truth data is a labor-intensive task, particularly for large-volume video analytics applications such as video surveillance and vehicles detection for autonomous driving. This paper presents a rapid and accurate method for associative searching in big image data obtained from security monitoring systems. We developed a semi-automatic moving object annotation method for improving deep learning models. The proposed method comprises three stages, namely automatic foreground object extraction, object annotation in subsequent video frames, and dataset construction using human-in-the-loop quick selection. Furthermore, the proposed method expedites dataset collection and ground truth annotation processes. In contrast to data augmentation and data generative models, the proposed method produces a large amount of real data, which may facilitate training results and avoid adverse effects engendered by artifactual data. We applied the constructed annotation dataset to train a deep learning you-only-look-once (YOLO) model to perform vehicle detection on street intersection surveillance videos. Experimental results demonstrated that the accurate detection performance was improved from a mean average precision (mAP) of 83.99 to 88.03.

Crowd management has been a topic of concern for many years because accidents frequently occur in situations with a high crowd density. With only a finite amount of space available during shows, protests, or other special occasions, a high crowd density can present a clear danger for those in the area. Considering these challenges, we employed and modified a three-tier multicolumn convolutional neural network (MCNN) system architecture to precisely estimate crowd density. We distinguished three regions from the near to far field to produce a crowd density map. Based on the MCNN system architecture, we detected changes in the size of a crowd according to a distance measure and examined additional features that can be incorporated to demonstrate their effects on crowd density maps. Examining these features using the Shanghaitech dataset demonstrated that compared with the native MCNN, the accuracy of estimating crowd counting by using our proposed method increased by 22.97% and 18.64% in terms of mean absolute error (MAE) and mean square error (MSE), respectively. A performance comparison with other state-of-the-art methods was also made. From this, we can infer that the proposed system is compatible with the other listed methods and is worthy of further investigation.

Developing reliable and robust face verification systems has been a tough challenge in computer vision, for several decades. The variation in illumination and head pose may seriously inhibit the accuracy of two-dimensional face recognition. With the invention of a depth map sensor, more three-dimensional volume data can be processed to mitigate the problem associated with face verification. This paper presents a three-dimensional face verification approach that includes three phases. First, point cloud library is applied to estimate features such as normal vectors and principal curvatures of every point on a human face point cloud acquired from three-dimensional depth sensor. Next, we adopt deep belief networks to train the identification model using extracted features. Finally, face verification is accomplished by using the pre-trained deep belief networks to justify if new incoming face point cloud feature is the one we specified. The experimental results demonstrate that the proposed system performs exceptionally well with about 96.43% verification accuracy.

Recently, health-care has become a popular issue. Having a good physique is also commonly regarded as important for being healthy. For evaluating our body status, the Body Mass Index (BMI) is a widely used indicator. However, calculating BMI is inconvenient and requires the physical measuring of people's weights and heights. In this paper, we are interested in building a mobile-based BMI prediction system using Kinect and data mining techniques so that everybody can easily monitor their BMI everywhere by taking a snapshot of their face. The rationale behind this is the intuition that there is a correlation between the shape of one's face and one's BMI values, which people often act on when noticing a friend has either gained or lost weight. Through the evaluations of 50 volunteers, we show that the rules for training BMI prediction match with people's common intuitions.

This paper presents an effective approach for detecting abandoned luggage in surveillance videos. We combine short- and long-term background models to extract foreground objects, where each pixel in an input image is classified as a 2-bit code. Subsequently, we introduce a framework to identify static foreground regions based on the temporal transition of code patterns, and to determine whether the candidate regions contain abandoned objects by analyzing the back-traced trajectories of luggage owners. The experimental results obtained based on video images from 2006 Performance Evaluation of Tracking and Surveillance and 2007 Advanced Video and Signal-based Surveillance databases show that the proposed approach is effective for detecting abandoned luggage, and that it outperforms previous methods.

We present an abandoned object detection system in this paper. A finite-state-machine model is introduced to extract stationary foregrounds in a scene for visual surveillance, where the state value of each pixel is inferred via the cooperation of short-term and long-term background models constructed in the proposed approach. To identify the left-luggage event, we then verify whether the static foregrounds are abandoned objects through the analysis of owner's moving trajectory back-tracked to the static foreground locations. Experimental results reveal that the proposed approach tackles the problem well on publicly available datasets.

As the density of road traffic increases, it becomes ever more important to analyze the traffic information for improving the transportation safety and reducing vehicle congestion. Vehicle trajectory may provide useful information. This paper presents a novel vehicle trajectory analysis system based on computer vision technology. By utilizing super pixel segmentation and object modeling, the proposed approach is able to increase the robustness of object matching and tracking overtime. We have developed an algorithm which is capable of detecting illegal-left-turn vehicle from a forward-only-lane using the proposed trajectory map constructed based on the vehicles trajectory. Experimental results show that correct object tracking rate is as high as 98.8%. The proposed system can handle illegal-left-turn vehicle detection on the crossroad with superior performance.

This work presents a framework of a collaborative multiple-camera tracking system for seamlessly object tracking across fixed cameras in overlapping and nonoverlapping fields of view (FOVs). The proposed system is constructed by two major elements: a client part responsible for single-camera object detection and tracking, and a server part responsible for multiple-camera collaborative tracking on the other hand. To improve object segmentation, this work proposes the concept of foreground edge extraction and compensation with an active edge image. For the single-camera tracking, the Kalman filter was applied. When the object was tracked, the system identified useful features for object matching. Color clustering was conducted using a K-mean clustering method, and the gait period was extracted. Furthermore, the homography technique was used to find the corresponding FOV lines in each camera view. Finally, this work used FOV lines for multiple-camera switching and then integrated various features for object matching. In addition, in order to provide a clear view for monitoring, we have implemented a visualization interface that will automatically show the continuous tracking result through a homography mapping. Simulation results indicate that the proposed system effectively addresses the seamless tracking problem in overlapping and nonoverlapping areas. This paper inspires a paradigm of human visual perception for multicamera object tracking, collaboration, and fusion through distributed cameras and computers.

Multi-camera tracking is a current trend in video surveillance. This paper proposes a framework for a collaborative multiple-camera tracking system for seamlessly tracking pedestrians across adjacent cameras. This study develops a system consisting of several single camera tracking clients and an information fusing server, and then use a TCP/IP network to exchange information between tracking clients. This work inspires a paradigm of human visual perception, collaboration and fusion through distributed cameras and computers. The proposed system is described in two sections corresponding to the two major elements of the system: A client part responsible for single camera object detection and tracking, and a server part responsible for the multiple cameras collaborative tracking on the other hand. To improve the performance of moving pedestrian matching, gait analysis is adopted based on the feet distance change of the moving objects. Furthermore, this study proposes a cameras switching algorithm to determine whether or not the pedestrian has left the field of view. Simulation results show that the developed system performs object matching and seamless tracking in various environments robustly. The tracking accuracy is as high as 96.9% and 99.7% for two test video sequences, respectively. The resulting system is promising and can be applied to wide-area monitoring and collaborative intelligent surveillance.

Although digitalization and the Internet are very convenient and useful for personal users and industry, they suffer from shortcomings of security. For instance, private content is difficult to protect and fake versions of multimedia data can be produced. Digital watermarking offers some effective solutions. This work presents a video watermarking scheme that uses motion vectors to define the place where the watermarks are embedded, and the number of watermark bits vary dynamically among frames. Fuzzy C-means (FCM) clustering method is utilized to select the motion vectors and the positions of the watermarks. The advantages of this work are twofold. A novel watermarking strategy is presented that does not require manual selection of watermark bits location. The number of embedded motion vector clusters differs depending on the motion characteristics in each frame. This significant property possesses higher security. As can be concluded from the experimental results, the watermarked video retains satisfactory quality with very small degradation.

In this paper, an effective model-based approach for computer-aided kidney segmentation of abdominal CT images with anatomic structure consideration is presented. This automatic segmentation system is expected to assist physicians in both clinical diagnosis and educational training. The proposed method is a coarse to fine segmentation approach divided into two stages. First, the candidate kidney region is extracted according to the statistical geometric location of kidney within the abdomen. This approach is applicable to images of different sizes by using the relative distance of the kidney region to the spine. The second stage identifies the kidney by a series of image processing operations. The main elements of the proposed system are: 1) the location of the spine is used as the landmark for coordinate references; 2) elliptic candidate kidney region extraction with progressive positioning on the consecutive CT images; 3) novel directional model for a more reliable kidney region seed point identification; and 4) adaptive region growing controlled by the properties of image homogeneity. In addition, in order to provide different views for the physicians, we have implemented a visualization tool that will automatically show the renal contour through the method of second-order neighborhood edge detection. We considered segmentation of kidney regions from CT scans that contain pathologies in clinical practice. The results of a series of tests on 358 images from 30 patients indicate an average correlation coefficient of up to 88% between automatic and manual segmentation.

Real-time eye tracking is a challenging problem. A face-circle fitting (FCF) method is proposed to confirm the extracted face region. Once the face is detected, a dark-pixel filter (DPF) is used to perform eye detection and tracking. Furthermore, The proposed system has been implemented on a personal computer with a PC-camera and can perform eye detection and tracking in real-time. The correct eye identification rate is as high as 92% during the actual operation

We present in this paper an approach of extracting and recognizing the features of hand gesture with low computational requirement. For the sake of real-time implementation, we developed two main algorithms: Curve Detection Algorithm (CDA) and Peak Detection Algorithm (PDA), where CDA extracts the feature from the silhouette pattern of image and PDA extracts specific patterns in silhouette image which implied peak information. Support Vector Machines is then applied for recognition. The overall performance of recognition achieved 96% in average.