Gaining environmental awareness through lateral head scanning (yaw rotations) is important for driving safety, especially when approaching intersections. Therefore, head scanning movements could be an important behavioral metric for driving safety research and driving risk mitigation systems. Tracking head scanning movements with a single in-car camera is preferred hardware-wise, but it is very challenging to track the head over almost a 180° range. In this paper we investigate two state-of-the-art methods, a multi-loss deep residual learning method with 50 layers (multi-loss ResNet-50) and an ORB feature-based simultaneous localization and mapping method (ORB-SLAM). While deep learning methods have been extensively studied for head pose detection, this is the first study in which SLAM has been employed to innovatively track head scanning over a very wide range. Our laboratory experimental results showed that ORB-SLAM was more accurate than multi-loss ResNet-50, which often failed when many facial features were not in the view. On the contrary, ORB-SLAM was able to continue tracking as it doesn't rely on particular facial features. Testing with real driving videos demonstrated the feasibility of using ORB-SLAM for tracking large lateral head scans in naturalistic video data.

Local features have widely been used in visual tracking to improve robustness in the presence of partial occlusion, deformation, and rotation. In this paper, a local fragment-based object tracking algorithm is proposed. Unlike many existing fragment-based algorithms using all the fragments and allocating the weight to each fragment according to similarity, the proposed algorithm only selects discriminative, unique, and valid fragments for tracking. First, discrimination and uniqueness metric are defined for each local fragment, and an automatic pre-selection mechanism is proposed for all these fragments. Second, a Harris-SIFT filter is used to select the current valid fragments and exclude the occluded or highly deformed fragments. By selecting the discriminative, unique, and valid fragments, these fragments are used to construct a structured description for the object. Finally, the object tracking is performed using the selected fragments combining the displacement and similarity, as well as spatial constraint of the selected fragments. The object template can be updated by fusing feature similarity and structural consistency. The experimental results on a recent OTB 2013 tracking benchmark data set demonstrate that the proposed algorithm can achieve reliable tracking results even in the presence of significant appearance changes, partial occlusion, and similar disturbances.

Lane changes are important behaviors to study in driving research. Automated detection of lane-change events is required to address the need for data reduction of a vast amount of naturalistic driving videos. This paper presents a method to deal with weak lane-marker patterns as small as a couple of pixels wide. The proposed method is novel in its approach to detecting lane-change events by accumulating lane-marker candidates over time. Since the proposed method tracks lane markers in temporal domain, it is robust to low resolution and many different kinds of interferences. The proposed technique was tested using 490 h of naturalistic driving videos collected from 63 drivers. The lane-change events in a 10-h video set were first manually coded and compared with the outcome of the automated method. The method's sensitivity was 94.8% and the data reduction rate was 93.6%. The automated procedure was further evaluated using the remaining 480-h driving videos. The data reduction rate was 97.4%. All 4971 detected events were manually reviewed and classified as either true or false lane-change events. Bootstrapping showed that the false discovery rate from the larger data set was not significantly different from that of the 10-h manually coded data set. This study demonstrated that the temporal processing of lane markers is an effcient strategy for detecting lane-change events involving weak lane-marker patterns in naturalistic driving.

This paper proposes a bio-inspired visual motion estimation algorithm based on motion energy, along with its compact very-large-scale integration (VLSI) architecture using low-cost embedded systems. The algorithm mimics motion perception functions of retina, V1, and MT neurons in a primate visual system. It involves operations of ternary edge extraction, spatiotemporal filtering, motion energy extraction, and velocity integration. Moreover, we propose the concept of confidence map to indicate the reliability of estimation results on each probing location. Our algorithm involves only additions and multiplications during runtime, which is suitable for low-cost hardware implementation. The proposed VLSI architecture employs multiple (frame, pixel, and operation) levels of pipeline and massively parallel processing arrays to boost the system performance. The array unit circuits are optimized to minimize hardware resource consumption. We have prototyped the proposed architecture on a low-cost field-programmable gate array platform (Zynq 7020) running at 53-MHz clock frequency. It achieved 30-frame/s real-time performance for velocity estimation on 160 × 120 probing locations. A comprehensive evaluation experiment showed that the estimated velocity by our prototype has relatively small errors (average endpoint error < 0.5 pixel and angular error < 10°) for most motion cases.

The enhancement of non-uniformly illuminated images often suffers from over-enhancement and produces unnatural results. This paper presents a naturalness preserved enhancement method for non-uniformly illuminated images, using a priori multi-layer lightness statistics acquired from high-quality images. This paper makes three important contributions: designing a novel multi-layer image enhancement model; deriving the multi-layer lightness statistics of high-quality outdoor images, which are incorporated into the multi-layer enhancement model; and showing that the overall quality rating of enhanced images is consistent with a combination of contrast enhancement and naturalness preservation. Two separate human observer evaluation studies were conducted on naturalness preservation and overall image quality. The results showed the proposed method outperformed four compared state-of-the-art enhancement methods.

Gaze tracking is crucial for studying driver's attention, detecting fatigue, and improving driver assistance systems, but it is difficult in natural driving environments due to nonuniform and highly variable illumination and large head movements. Traditional calibrations that require subjects to follow calibrators are very cumbersome to be implemented in daily driving situations. A new automatic calibration method, based on a single camera for determining the head orientation and which utilizes the side mirrors, the rear-view mirror, the instrument board, and different zones in the windshield as calibration points, is presented in this paper. Supported by a self-learning algorithm, the system tracks the head and categorizes the head pose in 12 gaze zones based on facial features. The particle filter is used to estimate the head pose to obtain an accurate gaze zone by updating the calibration parameters. Experimental results show that, after several hours of driving, the automatic calibration method without driver's corporation can achieve the same accuracy as a manual calibration method. The mean error of estimated eye gazes was less than 5°in day and night driving.