• The first work to apply eye-movement attention mechanisms in the task of image cross-view matching. • Gaze information from the human brain is utilized to guide model training and learning; they are not required during testing. • The fusion of the eye-movement and self-attention mechanisms using sub-image block-level can significantly guide model to focus more on the key image regions. • The implementation of image adaptive cropping and resolution enhancement strategies effectively removes the interference of redundant information and reduces the waste of computational resources. The goal of cross-view matching between street and aerial images is to retrieve aerial-view images that correspond to a given street-view image from a database of GPS-tagged aerial images. This task relies on image cross-view matching technology, focusing on the extraction and alignment of features representing the same location in both image types. The significant differences in perspective and appearance between street-view images and aerial-view images present a challenge. Aerial-view images cover a broader area, while street-view images focus on specific locations, creating information asymmetry that complicates the matching process. To tackle these challenges, this paper proposes a gaze-guided hybrid attention Vision Transformer, which uses gaze information to guide the model to focus on and align task-related features. Furthermore, inspired by the human visual cognitive process of "focus and zoom," we develop a hybrid attention module alongside an image adaptive cropping and resolution enhancement module. The hybrid attention module utilizes gaze information to guide the model to focus on relevant regions, while the image adaptive cropping strategy uses gaze information to guide the model to eliminate irrelevant regions. Techniques for improving image resolution allow for the magnification of important regions, thereby aiding the extraction of fine-grained features. We evaluate the model's performance on benchmark datasets and conduct ablation study experiments to assess the contributions of each module. Experimental results show that the method achieves a top-1 accuracy of 75.56% on the CVACT dataset, representing state-of-the-art performance. This study provides valuable insights into incorporating human experience into computational models, particularly through gaze-guided learning of visual task networks to enhance model performance.

For the safety assessment of autonomous driving systems (ADS), simulation testing has become an important complementary technique to physical road testing. In essence, simulation testing is a scenario-driven approach, whose effectiveness is highly dependent on the quality of given simulation scenarios. Moreover, simulation scenarios should be encoded into well-formatted files, otherwise, ADS simulation platforms cannot take them as inputs. Without large public datasets of simulation scenario files, both industry and academic applications of ADS simulation testing are hindered.

As highlighted in authoritative standards (e.g., ISO21448), traffic law compliance is a fundamental prerequisite for the commercialization of autonomous driving systems (ADS). Hence, manufacturers are in severe need of techniques to detect harsh driving situations in which the target ADS would violate traffic laws. To achieve this goal, existing works commonly resort to searching-based simulation testing, which continuously adjusts the scenario configurations (e.g., add new vehicles) of initial simulation scenarios and hunts for critical scenarios. Specifically, they apply pre-defined heuristics on each mutated scenario to approximate the likelihood of triggering ADS traffic violations, and accordingly perform searching scheduling. However, with those comparably more critical scenarios in hand, they fail to offer deterministic guidance on which and how scenario configurations should be further mutated to reliably trigger the target ADS misbehaviors. Hence, they inevitably suffer from meaningless efforts to traverse the huge scenario search space. In this work, we propose VioHawk, a novel simulation-based fuzzer that hunts for scenarios that imply ADS traffic violations. Our key idea is that, traffic law regulations can be formally modeled as hazardous/non-hazardous driving areas on the map at each timestamp during ADS simulation testing (e.g., when the traffic light is red, the intersection is marked as hazardous areas). Following this idea, VioHawk works by inducing the autonomous vehicle to drive into the law-specified hazardous areas with deterministic mutation operations. We evaluated the effectiveness of VioHawk in testing industry-grade ADS (i.e., Apollo). We constructed a benchmark dataset that contains 42 ADS violation scenarios against real-world traffic laws. Compared to existing tools, VioHawk can reproduce 3.1X~13.3X more violations within the same time budget, and save 1.6X~8.9X the reproduction time for those identified violations. Finally, with the help of VioHawk, we identified 9+8 previously unknown violations of real-world traffic laws on Apollo 7.0/8.0.