This paper describes Acies-OS, a content-centric platform for edge AI twinning and orchestration that allows easy deployment, re-configuration, and control of edge AI services, augmented by a digital twin. The work is motivated by the proliferation of edge AI in a plethora of IoT applications, ranging from home automation to military defense, and the emergence of digital twins that go beyond monitoring and emulation into configuration management and optimization of edge capabilities. While past work focused on either the edge capabilities themselves or the digital twin, this work focuses on their seamless interactions, offering abstractions that enable the digital twin to manage and optimize an increasingly diverse edge AI system. Acies-OS features a structured namespace, a thin client library with flexible pub/sub-based communication, health monitoring support, and a control plane for twin-based value-added analysis and optimization. To illustrate the use of Acies-OS, we implemented a multi-node multi-modality vehicle classification application and used Acies-OS to interface it to a digital twin. We then deployed the system in the field to showcase run-time twin-based optimizations of inference latency, classification accuracy, and robustness to failures in noisy and challenging conditions.

Digital Twins are evolving as a key component in modern systems with diverse applications like remote prognostics, optimizing run-time operation, anomaly detection, and more. The essential elements of a digital twin are a virtual representation, a physical asset, and the transfer of data/information between the two. IoT deployments are generally characterized by resource constraints, making synchronization of digital twins with IoT devices more challenging. There is a pressing need to optimize the bandwidth of the data transferred between the system and the twin, while ensuring that the twin is able to capture selected key aspects of the current operational state accurately. In this paper, we present TwinSync, a framework that can be utilized to construct flexible real-time representations of deployed IoT systems and efficiently synchronize relevant system states with the twin, over a communication bottleneck, within a configurable application-specific notion of error (henceforth referred to as approximate synchronization). Our approach is optimized to achieve data transfers utilizing less bandwidth without compromising the ability of the twin to replicate real-time system states within the specified approximate synchronization semantics. We evaluate the efficacy of TwinSync's synchronization by conducting both a synthetic analysis and a case study based on a real-life application prototype. Our evaluation indicates that using TwinSync can provide the same or greater accuracy (in many cases) while sending significantly fewer bytes than a bandwidth-insensitive synchronization approach. The result is attributed to a more judicial selection of data to transmit over bottlenecks, compared to bandwidth-insensitive approaches.

We demonstrate a unified approach to rigorous design of safety-critical autonomous systems using the VerifAI toolkit for formal analysis of AI-based systems. VerifAI provides an integrated toolchain for tasks spanning the design process, including modeling, falsification, debugging, and ML component retraining. We evaluate all of these applications in an industrial case study on an experimental autonomous aircraft taxiing system developed by Boeing, which uses a neural network to track the centerline of a runway. We define runway scenarios using the Scenic probabilistic programming language, and use them to drive tests in the X-Plane flight simulator. We first perform falsification, automatically finding environment conditions causing the system to violate its specification by deviating significantly from the centerline (or even leaving the runway entirely). Next, we use counterexample analysis to identify distinct failure cases, and confirm their root causes with specialized testing. Finally, we use the results of falsification and debugging to retrain the network, eliminating several failure cases and improving the overall performance of the closed-loop system.

With a goal to improve transportation safety, this paper proposes a collaborative driving framework based on assessments of both internal and external risks involved in vehicle driving. The internal risk analysis includes driver drowsiness detection and driver intention recognition that helps to understand the human driver's behavior. Steering wheel data and facial expression are used to detect the driver's drowsiness. Hidden Markov models are adapted to recognize the driver's intention using the vehicle's lane position, control, and state data. For the external risk analysis, a co-pilot utilizes a collision avoidance system to estimate the collision probability between the ego vehicle and other nearby vehicles. Based on the risk analyses, we design a novel collaborative driving scheme by fusing the control inputs from the human driver and the co-pilot to obtain the final control input for the ego vehicle under different circumstances. The proposed collaborative driving framework is validated in an assisted-driving testbed, which enables both autonomous and manual driving capabilities.

This work presents a strategy for coordinated multi-agent weeding under conditions of partial environmental information. The goal of this work is to demonstrate the feasibility of coordination strategies for improving the weeding performance of autonomous agricultural robots. We show that, given a sufficient number of agents, the algorithm can successfully weed fields with various initial seed bank densities, even when multiple days are allowed to elapse before weeding commences. Furthermore, the use of coordination between agents is demonstrated to strongly improve system performance as the number of agents increases, enabling the system to eliminate all the weeds in the field, as in the case of full environmental information, when the planner without coordination failed to do so. As a domain to test our algorithms, we have developed an open source simulation environment, Weed World, which allows real-time visualization of coordinated weeding policies, and includes realistic weed generation. In this work, experiments are conducted to determine the required number of agents and their required transit speed, for given initial seed bank densities and varying allowed days before the start of the weeding process.

This paper proposes a driver assistance system with a collaborative control framework between the human driver and a Collision Avoidance System (CAS) for vehicles on the road while considering the driver drowsiness status. Driver drowsiness detection is performed with two inputs: driver's facial data from a camera mounted in front of the driver and steering wheel data from the car controller system. We use the driver's drowsiness state as an input to the collaborative control framework in which the CAS algorithm runs in parallel with the human control and only intervenes under certain situations to assist the human driver. Experiments were performed on a simulated vehicle driving testbed to evaluate our drowsiness detection system and demonstrate the effectiveness of the proposed collaborative control framework.