Cooperation among human-driven vehicles (HVs) is essential for traffic safety and efficiency. However, the emergence of autonomous vehicles (AVs) has prompted a new question: Will HVs still cooperate with AVs? Prior studies and narrative reviews yielded inconsistent findings. To answer this question, we conducted the first systematic review and meta-analysis of HV–AV cooperation, synthesizing evidence from 24 articles, 27 samples, 32 effect sizes, and 5,778 participants. Results revealed that people drive less cooperatively when interacting with AVs than with HVs (Hedges’ g = − 0.19, 95% CI [ − 0.31, − 0.07]). The meta-regression revealed a significant link between cooperative driving and the year of publication, with more recent studies showing more cooperation; other moderators (e.g., data collection methods) were not significant. We discuss the implications of less cooperation for AV development, traffic regulations, and human–AI cooperation, and current challenges in theory, replicability, and ecological validity, in addition to offering recommendations for future research.

While autonomous vehicles (AVs) could transform transportation, their “black box” nature often leaves occupants unaware of the rationale for their actions. Providing explanations can enhance transparency and facilitate widespread AV adoption. This study investigated scenario (when) and human factors (who) that influence the Demand of Explanations (DoE) to ensure explanations are provided when needed, followed by exploring the reasons (why) behind these demands. We conducted an online experimental study among 440 participants, who viewed 36 simulated driving scenarios, varying in AV actions, driving styles, time/weather and traffic environments. Results of multilevel and qualitative analysis showed that: (1) DoE was significantly higher when AVs drove aggressively, in urban areas, during turning and merging, and in nighttime or rain; (2) participants who had lower trust in AVs and older adults significantly demanded more explanations; and (3) safety and traffic rules were the primary reasons for seeking explanations.

Stereo matching achieves significant progress with iterative algorithms like RAFT-Stereo and IGEV-Stereo. However, these methods struggle in ill-posed regions with occlusions, textureless, or repetitive patterns, due to a lack of global context and geometric information for effective iterative refinement. To enable the existing iterative approaches to incorporate global context, we propose the Global Regulation and Excitation via Attention Tuning (GREAT) framework which encompasses three attention modules. Specifically, Spatial Attention (SA) captures the global context within the spatial dimension, Matching Attention (MA) extracts global context along epipolar lines, and Volume Attention (VA) works in conjunction with SA and MA to construct a more robust cost-volume excited by global context and geometric details. To verify the universality and effectiveness of this framework, we integrate it into several representative iterative stereo-matching methods and validate it through extensive experiments, collectively denoted as GREAT-Stereo. This framework demonstrates superior performance in challenging ill-posed regions. Applied to IGEV-Stereo, among all published methods, our GREAT-IGEV ranks first on the Scene Flow test set, KITTI 2015, and ETH3D leaderboards, and achieves second on the Middlebury benchmark. Code is available at https://github.com/JarvisLee0423/GREAT-Stereo.

Multi-camera systems are essential in movies, live broadcasts, and other media. The selection of the appropriate camera for every moment has a decisive impact on production quality and audience preferences. Learning-based multi-camera view recommendation frameworks have been explored to assist professionals in decision making. This work explores how two standard cinematography practices could be incorporated into the learning pipeline: (1) not staying on the same camera for too long and (2) introducing a scene from a wider shot and gradually progressing to narrower ones. In these regards, we incorporate (1) the duration of the displaying camera and (2) camera identity as temporal and camera embedding in a transformer architecture, thereby implicitly guiding the model to learn the two practices from professional-labeled data. Experiments show that the proposed framework outperforms the baseline by 14.68% in six-way classification accuracy. Ablation studies on different approaches to embedding the temporal and camera information further verify the efficacy of the framework.

Safety validation, which assesses the safety of an autonomous system's motion planning decisions, is critical for the safe deployment of autonomous vehicles. Existing input validation techniques from other machine learning domains, such as image classification, face unique challenges in motion planning due to its contextual properties, including complex inputs and one-to-many mapping. Furthermore, current output validation methods in autonomous driving primarily focus on open-loop trajectory prediction, which is ill-suited for the closed-loop nature of motion planning. We introduce REDOUBT, the first systematic safety validation framework for autonomous vehicle motion planning that employs a duo mechanism, simultaneously inspecting input distributions and output uncertainty. REDOUBT identifies previously overlooked unsafe modes arising from the interplay of In-Distribution/Out-of-Distribution (OOD) scenarios and certain/uncertain planning decisions. We develop specialized solutions for both OOD detection via latent flow matching and decision uncertainty estimation via an energy-based approach. Our extensive experiments demonstrate that both modules outperform existing approaches, under both open-loop and closed-loop evaluation settings. Our codes are available at: https://github.com/sgNicola/Redoubt.

Recent advances in computer vision algorithms and video streaming technologies have facilitated the development of edge-server-based video analytics systems, enabling them to process sophisticated real-world tasks, such as traffic surveillance and workspace monitoring. Meanwhile, due to their omnidirectional recording capability, 360-degree cameras have been proposed to replace traditional cameras in video analytics systems to offer enhanced situational awareness. Yet, we found that providing an efficient 360-degree video analytics framework is a non-trivial task. Due to the higher resolution and geometric distortion in 360-degree videos, existing video analytics pipelines fail to meet the performance requirements for end-to-end latency and query accuracy. To address these challenges, we introduce the innovative ST-360 framework specifically designed for 360-degree video analytics. This framework features a spatial–temporal filtering algorithm that optimizes both data transmission and computational workloads. Evaluation of the ST-360 framework on a unique dataset of 360-degree first-responders videos reveals that it yields accurate query results with a 50% reduction in end-to-end latency compared to state-of-the-art methods.

Multi-camera systems are indispensable in movies, TV shows, and other media. Selecting the appropriate camera at every timestamp has a decisive impact on production quality and audience preferences. Learning-based view recommendation frameworks can assist professionals in decision-making. However, they often struggle outside of their training domains. The scarcity of labeled multi-camera view recommendation datasets exacerbates the issue. Based on the insight that many videos are edited from the original multi-camera videos, we propose transforming regular videos into pseudo-labeled multi-camera view recommendation datasets. Promisingly, by training the model on pseudo-labeled datasets stemming from videos in the target domain, we achieve a 68% relative improvement in the model’s accuracy in the target domain and bridge the accuracy gap between in-domain and never-before-seen domains.

In a real-life meeting environment, individuals often demonstrate a remarkable ability to selectively focus their attention on specific visual information. This ability allows them to naturally concentrate on a specific region of interest while tuning out others. Understanding and exploiting such selective attention remains unexplored in a user-centric teleconferencing system, where there is a potential to customize video streaming and foveated rendering based on the viewer’s attention. This paper proposes a novel user-centric scene analysis module that fully leverages the power of selective attention for online meeting scenarios and recognizes the unequal importance of individual pixels in the videos. The module determines the user’s selective attention through the meeting contexts. The contextual representation of the meeting is modeled as a combination of two primary components: proactive user interaction within the system and passive real-time analysis of high-level visual semantics from the scenes. As the meeting progresses, the interactive scene analysis module dynamically updates its contextual representation, offering a dual advantage: (a) Videos can be selectively and adaptively streamed within a user’s attention, resulting in bandwidth savings of up to 78 percent. (b) The module enhances the overall quality of the user experience by facilitating higher user interactivity, particularly in meeting-related tasks such as screen sharing, privacy-preserving user blocking, background removal, automatic user attention shift detection, etc. Our interactive scene analysis module makes significant progress toward enabling an efficient, immersive, and intelligent teleconferencing system.

The emergence of 360-video streaming systems has brought about new possibilities for immersive video experiences while requiring significantly higher bandwidth than traditional 2D video streaming. Viewport prediction is used to address this problem, but interesting storylines outside the viewport are ignored. To address this limitation, we present SAVG360, a novel viewport guidance system that utilizes global content information available on the server side to enhance streaming with the best saliency-captured storyline of 360-videos. The saliency analysis is performed offline on the media server with powerful GPU, and the saliency-aware guidance information is encoded and shared with clients through the Saliency-aware Guidance Descriptor. This enables the system to proactively guide users to switch between storylines of the video and allow users to follow or break guided storylines through a novel user interface. Additionally, we present a viewing mode prediction algorithms to enhance video delivery in SAVG360. Evaluation of user viewport traces in 360-videos demonstrate that SAVG360 outperforms existing tiled streaming solutions in terms of overall viewport prediction accuracy and the ability to stream high-quality 360 videos under bandwidth constraints. Furthermore, a user study highlights the advantages of our proactive guidance approach over predicting and streaming of where users look.

360-degree video has become increasingly popular in content consumption. However, finding the viewing direction for important content within each frame poses a significant challenge. Existing approaches rely on either viewer input or algorithmic determination to select the viewing direction, but neither mode consistently outperforms the other in terms of content-importance. In this paper, we propose 360TripleView, the first view management system for 360-degree video that automatically infers and utilizes the better view mode for each frame, ultimately providing viewers with higher content-importance views. Through extensive experiments and a user study, we demonstrate that 360TripleView achieves over 90% accuracy in inferring the better mode and significantly enhances content-importance compared to existing methods.

First responders operate in hazardous working conditions with unpredictable risks. To better prepare for demands of the job, first responder trainees conduct training exercises that are being recorded and reviewed by the instructors, who check for objects indicating risks within the video recordings (e.g., firefighter with an unfastened gas mask). However, the traditional reviewing process is inefficient due to unanalyzed video recordings and limited situational awareness. For better reviewing experience, a latency-aware Viewing and Query Service (VQS) should be provided. The VQS should support object searching, which can be achieved using the video object detection algorithms. Meanwhile, the application of 360-degree cameras facilitates an unlimited field of view of the training environment. Yet, this medium represents a major challenge because low-latency high-accuracy 360-degree object detection is difficult due to higher resolution and geometric distortion. In this paper, we present the Responders-360 system architecture designed for 360-degree object detection. We propose a Dynamic Selection algorithm that optimizes computation resources while yielding accurate 360-degree object inference. The results, using a unique dataset collected from a firefighting training institute, show that the Responders-360 framework achieves 4x speedup and 25% memory usage reduction compared with the state-of-the-art methods.

360-degree video is becoming an integral part of our content consumption through both video on demand and live broadcast services. However, live broadcast is still challenging due to the huge network bandwidth cost if all 360-degree views are delivered to a large viewer population over diverse networks. In this paper, we present 360BroadView, a viewer management approach to viewport prediction in 360-degree video live broadcast. We make some high-bandwidth network viewers be leading viewers to help the others (lagging viewers) predict viewports during 360-degree video viewing and save bandwidth. Our viewer management maintains the leading viewer population despite viewer churns during live broadcast, so that the system keeps functioning properly. Our evaluation shows that 360BroadView maintains the leading viewer population at a minimal yet necessary level for 97 percent of the time.

In IoT, what services from which nearby devices are available, must be discovered by a user's device (e.g., smartphone) before she can issue commands to access them. Service visibility scoping in large scale, heterogeneous enterprise environments has multiple unique features, e.g., proximity based interactions, differentiated visibility according to device natures and user attributes, frequent user churns thus revocation. They render existing solutions completely insufficient. We propose Argus, a distributed algorithm offering three-level, fine-grained visibility scoping in parallel: i) Level 1 public visibility where services are identically visible to everyone; ii) Level 2 differentiated visibility where service visibility depends on users’ non-sensitive attributes; iii) Level 3 covert visibility where service visibility depends on users’ sensitive attributes that are never explicitly disclosed. Extensive analysis and experiments show that: i) Argus is secure; ii) its Level 2 is 10x as scalable and computationally efficient as work using Attribute-based Encryption, Level 3 is 10x as efficient as work using Paring-based Cryptography; iii) it is fast and agile for satisfactory user experience, costing 0.25 s to discover 20 Level 1 devices, and 0.63 s for Level 2 or Level 3 devices.

Immersive virtual tours based on 360-degree cameras, showing famous outdoor scenery, are becoming more and more desirable due to travel costs, pandemics and other constraints. To feel immersive, a user must receive the view accurately corresponding to her position and orientation in the virtual space when she moves inside, and this requires cameras’ orientations to be known. Outdoor tour contexts have numerous, ultra-sparse cameras deployed across a wide area, making camera pose estimation challenging. As a result, pose estimation techniques like SLAM, which require mobile or dense cameras, are not applicable. In this paper we present a novel strategy called 360ViewPET, which automatically estimates the relative poses of two stationary, ultra-sparse (15 meters apart) 360-degree cameras using one equirectangular image taken by each camera. Our experiments show that it achieves accurate pose estimation, with a mean error as low as 0.9 degree.

Video cameras are pervasively deployed in city scale for public good or community safety (i.e. traffic monitoring or suspected person tracking). However, analyzing large scale video feeds in real time is data intensive and poses severe challenges to today's network and computation systems. We present CrossRoI, a resource-efficient system that enables real time video analytics at scale via harnessing the videos content associations and redundancy across a fleet of cameras. CrossRoI exploits the intrinsic physical correlations of cross-camera viewing fields to drastically reduce the communication and computation costs. CrossRoI removes the repentant appearances of same objects in multiple cameras without harming comprehensive coverage of the scene. CrossRoI operates in two phases - an offline phase to establish cross-camera correlations, and an efficient online phase for real time video inference. Experiments on real-world video feeds show that CrossRoI achieves 42% ~ 65% reduction for network overhead and 25% ~ 34% reduction for response delay in real time video analytics applications with more than 99% query accuracy, when compared to baseline methods. If integrated with SotA frame filtering systems, the performance gains of CrossRoI reaches 50% ~ 80% (network overhead) and 33% ~ 61% (end-to-end delay).

Scalable, fine-grained access control for Internet-of-Things is needed in enterprise environments, where tens of thousands of users need to access smart objects which have a similar or larger order of magnitude. Existing solutions offer all-or-nothing access, or require all access to go through a cloud backend, greatly impeding access granularity, robustness and scale. In this paper, we propose Heracles, an IoT access control system which achieves robust, fine-grained access control and responsive execution at enterprise scale. Heracles adopts a capability-based approach using secure, unforgeable tokens that describe the authorizations of users, to either individuals or collections of objects in single or bulk operations. It has a 3-tier architecture to provide centralized policy and distributed execution desired in enterprise environments. Extensive analysis and performance evaluation on a testbed prove that Heracles achieves fine-grained access control and responsive execution at enterprise scale. Compared with systems using access control list, Heracles eliminates or reduces by 10x-100x the updating overhead under frequent changes of subject memberships and policies. Besides, Heracles achieves responsive execution: it takes 0.57 second to access 18 objects which are scattered 1-9 hops away, and execution on a 1-hop or 2-hop object needs only 0.07 or 0.13 second respectively.

In IoT it is common that before a command can execute on a smart object, certain preconditions (on possibly other objects) should be met first to ensure safety or efficiency. Existing work has realized automatic precondition execution: when a user issues a command, her device automatically finds out all the precondition commands, and executes them in the correct order. However, security issues have not been considered: it assumes that a user device honestly follows the order it has been told to send commands to objects, and objects trust users thus do not check whether the preconditions are indeed met. In this paper we propose two strategies to enforce precondition execution order: 1) Snowball relying on signed declarations from precondition objects; 2) Onion using disposable access tokens encrypted by a trustworthy server. Our extensive analysis and experiments on a 20-node testbed show that both strategies are secure and reliable. Snowball has higher availability while Onion is more efficient and responsive: Onion uses 1.6/2.1 s to access 20 one-hop/multi-hop objects, 62%/54% of Snowball's time.

In IoT, what services from which nearby devices are available, must be discovered by a user's device (e.g., smartphone) before she can issue commands to access them. Service visibility scoping in large scale, heterogeneous enterprise environments has multiple unique features, e.g., proximity based interactions, differentiated visibility according to device natures and user attributes, frequent user churns thus revocation. They render existing solutions completely insufficient. We propose Argus, a distributed algorithm offering three-level, fine-grained visibility scoping in parallel: i) Level 1 public visibility where services are identically visible to everyone; ii) Level 2 differentiated visibility where service visibility depends on users' non-sensitive attributes; iii) Level 3 covert visibility where service visibility depends on users' sensitive attributes that are never explicitly disclosed. Extensive analysis and experiments show that: i) Argus is secure; ii) its Level 2 is 10x as scalable and computationally efficient as work using Attribute-based Encryption, Level 3 is 10x as efficient as work using Paring-based Cryptography; iii) it is fast and agile for satisfactory user experience, costing 0.25 s to discover 20 Level 1 devices, and 0.63 s for Level 2 or Level 3 devices.

Internet of Things in enterprise environments features large numbers of devices deployed in rooms, floors of possibly multiple buildings. Delivering user commands to control devices nearby and multiple hops away requires efficient and scalable routing in such environments. Existing work in ad-hoc, sensor or IoT network routing lacks good human accessibility and scalability. In this paper, we propose a peer-based protocol GraphiteRouting. All devices carry human-readable hierarchical string names for easy reference. It leverages devices' installation hierarchy for scalable hierarchical routing: most devices maintain only a few to dozens of routing entries for same-room devices, and a fraction of devices act as gateways for traffic from/to other rooms, floors or buildings. Also, it leverages users' operation patterns to less optimize infrequently used routes. Extensive analysis and performance evaluation on a 20-node testbed prove that GraphiteRouting is scalable: it has routing tables 10x- -1000x smaller than those in peer-based flat routing; also, upon device joining/leaving, its routing entries converge in less than 5 s, and forwarding a user command over 8 hops costs less than 0.3 s.

Scalable, fine-grained access control for Internet-of-Things is needed in enterprise environments, where thousands of subjects need to access possibly one to two orders of magnitude more objects. Existing solutions offer all-or-nothing access, or require all access to go through a cloud backend, greatly impeding access granularity, robustness and scale. In this paper, we propose Heracles, an IoT access control system that achieves robust, fine-grained access control at enterprise scale. Heracles adopts a capability-based approach using secure, unforgeable tokens that describe the authorizations of subjects, to either individual or collections of objects in single or bulk operations. It has a 3-tier architecture to provide centralized policy and distributed execution desired in enterprise environments, and delegated operations for responsiveness of resource-constrained objects. Extensive security analysis and performance evaluation on a testbed prove that Heracles achieves robust, responsive, fine-Qrained access control in large scale enterprise environments.

The proliferation and daily congregation of modern mobile devices have created abundant opportunities for peer edge devices to share valuable data with each other. The short contact durations, relatively small sharing sizes, and uncertain data availability, demand agile, light weight peer based data sharing. In this paper, we propose Peer Data Sharing (PDS) that enables edge devices to discover which data exist in nearby peers, and retrieve interested data robustly and efficiently. PDS uses novel lingering queries, mixedcast and en-route message rewriting techniques to minimize redundant transmissions and maximize opportunistic overhearing thus caching in data discovery and retrieval. Extensive evaluations based on an Android prototype show that PDS discovers and retrieves almost 100% data in tens of seconds, and remains robust despite wireless contention, simultaneous consumer requests and user mobility.

Digital garage maps are the basis for future vehicle navigation services such as smart parking management that displays the availability of parking spaces. It can direct drivers to empty ones, avoiding any searching, circulating in large, complex parking structures. However, such maps are not currently available, making it impossible to deploy smart parking management. Conducting manual survey incurs tremendous amount of human efforts, and cannot scale to large numbers of garages. In this paper, we propose three algorithms, Sequential Merging, Points Clustering and Segments Matching that can automatically construct complete and accurate garage maps using data crowdsensed from drivers. Upon entering and leaving the garage, the driver's smartphone collects inertial data, which are used to generate the vehicle's trajectory. Our algorithms fuse together these trajectories to recreate the size, layout of the garage. We compare the performance of the three algorithms using different garages. We find that Points Clustering is robust to trajectory errors, with F-score above 0.95 for trajectory length error up to 2 meters, Segments Matching can handle partial trajectories with arbitrary start/end locations, and it constructs the same map using trajectories much shorter than those needed by the other two algorithms.