Performing driving behaviors based on causal reasoning is essential to ensure driving safety. In this work, we investigated how state-of-the-art 3D Convolutional Neural Networks (CNNs) perform on classifying driving behaviors based on causal reasoning. We proposed a perturbation-based visual explanation method to inspect the models' performance visually. By examining the video attention saliency, we found that existing models could not precisely capture the causes (e.g., traffic light) of the specific action (e.g., stopping). Therefore, the Temporal Reasoning Block (TRB) was proposed and introduced to the models. With the TRB models, we achieved the accuracy of $\mathbf{86.3\%}$, which outperform the state-of-the-art 3D CNNs from previous works. The attention saliency also demonstrated that TRB helped models focus on the causes more precisely. With both numerical and visual evaluations, we concluded that our proposed TRB models were able to provide accurate driving behavior prediction by learning the causal reasoning of the behaviors.

Single image dehazing is the ill-posed two-dimensional signal reconstruction problem. Recently, deep convolutional neural networks (CNN) have been successfully used in many computer vision problems. In this paper, we propose a Y-net that is named for its structure. This network reconstructs clear images by aggregating multi-scale features maps. Additionally, we propose a Wavelet Structure SIMilarity (W-SSIM) loss function in the training step. In the proposed loss function, discrete wavelet transforms are applied repeatedly to divide the image into differently sized patches with different frequencies and scales. The proposed loss function is the accumulation of SSIM loss of various patches with respective ratios. Extensive experimental results demonstrate that the proposed Y-net with the W-SSIM loss function restores high-quality clear images and outperforms state-of-the-art algorithms. Code and models are available at https://github.com/dectrfov/Y-net.

Discovering and exploiting the causality in deep neural networks (DNNs) are crucial challenges for understanding and reasoning causal effects (CE) on an explainable visual model. "Intervention" has been widely used for recognizing a causal relation ontologically. In this paper, we propose a causal inference framework for visual reasoning via docalculus. To study the intervention effects on pixel-level features for causal reasoning, we introduce pixel-wise masking and adversarial perturbation. In our framework, CE is calculated using features in a latent space and perturbed prediction from a DNN-based model. We further provide a first look into the characteristics of discovered CE of adversarially perturbed images generated by gradient-based methods1. Experimental results show that CE is a competitive and robust index for understanding DNNs when compared with conventional methods such as class-activation mappings (CAMs) on the Chest X-Ray-14 dataset for human-interpretable feature(s) (e.g., symptom) reasoning. Moreover, CE holds promises for detecting adversarial examples as it possesses distinct characteristics in the presence of adversarial perturbations.

Traffic sign detection and recognition has been studied in multiple areas including civil and transportation engineering, automated driving, and computer vision. However, previous work has devoted relatively less attention to U.S. signs. Among all types of U.S. signs, warning signs are the most crucial to road safety. In this work, we propose a customized detection and recognition method for U.S. warning signs that does not require training. For detection, preprocessing with a two-layer HSV-B filter and style transfer reduces color bias and a substantial number of false positives. We formulate the recognition process as a template-matching problem in which pre-trained deep networks serve as feature extractors and we use the cosine distance as the distance metric. Best results on selected images from Georgia State Route 2 achieve above 90% precision and recall in detection and 92.6% accuracy in recognition. Further tests on large datasets demonstrate that the proposed method is promising, and it can support transportation agencies in the management of their warning sign assets.

Statistical similarity measurements, such as mutual information (MI) and normalized mutual information (NMI), show potential in the registration of 2D-image to 3D-range scans collected in urban environments. However 2D-3D registration with these measurements are of limited usage in urban sensing applications because: 1) it relies on the diversity and dependency between pre-defined pair of 2D-3D attributes, such as the intensity from images and reflectivity from range scans, in the urban sensing environment, and 2) it requires high-end range sensors with strong abilities to capture reflectivity in urban scenarios. In this paper, we propose a robust way of estimating statistical similarity measurements for 2D-3D data that are collected in various urban scenes with both low-cost and high-end range sensors. Rather than estimate the similarity of 2D-3D data on specific pair of 2D-3D attributes, we compute similarity measurements between a set of 2D-3D attribute-pairs that could be dominant in the category of sensed urban scene and combine them into a reliable similarity measurement. By applying the combined similarity measurement to the common framework of statistical 2D-3D registration, we get superior results when compared with state-of-art similarity measurements (MI and NMI) in terms of registration accuracy and robustness to initial condition, as indicated by experiments conducted on two datasets that are collected in various urban scenes, with low-cost and high-end sensors.

Crack evaluation is a vital component in the pavement condition survey. However, researchers have had difficulty automatically mimicking the in-field evaluation practice to achieve a fully automated pavement crack evaluation due to the diversity of real-world crack patterns and the complexity of crack definitions in an agency's survey protocol. The multi-scale crack analysis model based on a Crack Fundamental Element is developed to topologically represent complex crack patterns and provide systematic crack properties to support crack classification. Utilizing this model, this study implements an automatic crack classification and quantification method for use on the state highways in Georgia, United States. Both image-based validation and field validation are conducted with the pavement experts from the Georgia Department of Transportation (GDOT) on State Route 236 in Atlanta, and the proposed method achieves a classification accuracy of 92.2% for load cracking and 98.1% for B/T cracking.

The authors of "Detecting Curves with Unknown Endpoints and Arbitrary Topology Using Minimal Paths" which appeared in the October 2012 issue of this journal point out and correct an error that occurred in equation (14) of the paper.

Existing state-of-the-art minimal path techniques work well to extract simple open curves in images when both endpoints of the curve are given as user input or when one input is given and the total length of the curve is known in advance. Curves which branch require even further prior input from the user, namely, each branch endpoint. In this work, we present a novel minimal path-based algorithm which works on much more general curve topologies with far fewer demands on the user for initial input compared to prior minimal path-based algorithms. The two key novelties and benefits of this new approach are that 1) it may be used to detect both open and closed curves, including more complex topologies containing both multiple branch points and multiple closed cycles without requiring a priori knowledge about which of these types is to be extracted, and 2) it requires only a single input point which, in contrast to existing methods, is no longer constrained to be an endpoint of the desired curve but may in fact be ANY point along the desired curve (even an internal point). We perform quantitative evaluation of the algorithm on 48 images (44 pavement crack images, 1 catheter tube image, and 3 retinal images) against human supplied ground truth. The results demonstrate that the algorithm is indeed able to extract curve-like objects accurately from images with far less prior knowledge and less user interaction compared to existing state-of-the-art minimal path-based image processing algorithms. In the future, the algorithm can be applied to other 2D curve-like objects and it can be extended to detect 3D curves.

©2010. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms.