Vision Transformers (ViTs), extensively pre-trained on large-scale datasets, have become fundamental to foundation models, enabling adaptation to diverse downstream tasks. Existing adaptation methods typically require direct data access, rendering them infeasible in privacy-sensitive domains where clients are often reluctant to share their data. A straightforward solution may be sending the pre-trained ViT to clients for local adaptation, which poses issues of model intellectual property and incurs heavy client computation overhead. To address these issues, we propose a novel split adaptation (SA) method that enables effective downstream adaptation while protecting data and models. SA, inspired by split learning (SL), segments the pre-trained ViT into a frontend and a backend, with only the frontend shared with the client for data representation extraction. But unlike regular SL, SA replaces frontend parameters with low-bit quantized values, preventing direct exposure of the model. SA allows the client to add bi-level noise to the frontend and the extracted data representations, ensuring data protection. Accordingly, SA incorporates data-level and model-level out-of-distribution enhancements to mitigate noise injection’s impact. Our SA focuses on the challenging few-shot adaptation and adopts patch retrieval augmentation for overfitting alleviation. Extensive experiments on multiple datasets validate SA’s superiority over state-of-the-art methods and demonstrate its defense against advanced data reconstruction attacks while preventing model leakage with minimal computation cost on the client side. The source codes can be found at https://github.com/conditionWang/Split_Adaptation.

Hardware/software co-design in the context of cyber-physical systems (CPS) [2] takes the form of co-designing (i) a control algorithm, and (ii) its software implementation on a distributed and heterogeneous architecture [5, 6]. In domains such as automotive CPS, this involves the design and implementation of multiple controllers on distributed automotive architectures consisting of different electronic control units (ECUs) and communication buses like CAN, FlexRay and automotive Ethernet [13]. Traditionally, following the principle of separation of concerns, control algorithms were designed independent of the implementation platform details. As a result, they made certain assumptions on delays experienced by control signals and assumed, for example, that all sensor inputs necessary for state estimation arrive at the same time. When trying to implement such controllers on an independently designed implementation platform, where many of the controller design or model-level assumptions are not satisfied, an iterative design process that involves testing and adjustments to the controllers and the implementation decisions became necessary. This led to the development of X-in-the-loop simulations, where X can be software and hardware at various stages of implementation, along with which the control algorithms are simulated [1, 14]. The aim is to ensure that the semantics or the behavior of the controller models (or algorithms) are preserved in the final implementation [12].

While large language models have demonstrated impressive performance across various domains and tasks, their security issues have become increasingly severe. Machine unlearning has emerged as a representative approach for model safety and security by removing the influence of undesired data on the target model. However, these methods do not sufficiently consider that unlearning requests in real-world scenarios are continuously emerging, especially in the context of LLMs, which may lead to accumulated model utility loss that eventually becomes unacceptable. Moreover, existing LLM unlearning methods often ignore previous data access limitations due to privacy concerns and copyright protection. Without previous data, the utility preservation during unlearning is much harder. To overcome these challenges, we propose the \OOO{} framework that includes an \underline{\textit{O}}rthogonal low-rank adapter (LoRA) for continually unlearning requested data and an \underline{\textit{O}}ut-\underline{\textit{O}}f-Distribution (OOD) detector to measure the similarity between input and unlearning data. The orthogonal LoRA achieves parameter disentanglement among continual unlearning requests. The OOD detector is trained with a novel contrastive entropy loss and utilizes a glocal-aware scoring mechanism. During inference, our \OOO{} framework can decide whether and to what extent to load the unlearning LoRA based on the OOD detector's predicted similarity between the input and the unlearned knowledge. Notably, \OOO{}'s effectiveness does not rely on any retained data. We conducted extensive experiments on \OOO{} and state-of-the-art LLM unlearning methods across three tasks and seven datasets. The results indicate that \OOO{} consistently achieves the best unlearning effectiveness and utility preservation, especially when facing continuous unlearning requests. The source codes can be found at \url{https://github.com/GCYZSL/O3-LLM-UNLEARNING}.

Large Language Models (LLMs) have shown significant promise in real-world decision-making tasks for embodied artificial intelligence, especially when fine-tuned to leverage their inherent common sense and reasoning abilities while being tailored to specific applications. However, this fine-tuning process introduces considerable safety and security vulnerabilities, especially in safety-critical cyber-physical systems. In this work, we propose the first comprehensive framework for **B**ackdoor **A**ttacks against **L**LM-based **D**ecision-making systems (BALD) in embodied AI, systematically exploring the attack surfaces and trigger mechanisms. Specifically, we propose three distinct attack mechanisms: *word injection*, *scenario manipulation*, and *knowledge injection*, targeting various components in the LLM-based decision-making pipeline. We perform extensive experiments on representative LLMs (GPT-3.5, LLaMA2, PaLM2) in autonomous driving and home robot tasks, demonstrating the effectiveness and stealthiness of our backdoor triggers across various attack channels, with cases like vehicles accelerating toward obstacles and robots placing knives on beds. Our word and knowledge injection attacks achieve nearly 100\% success rate across multiple models and datasets while requiring only limited access to the system. Our scenario manipulation attack yields success rates exceeding 65\%, reaching up to 90\%, and does not require any runtime system intrusion. We also assess the robustness of these attacks against defenses, revealing their resilience. Our findings highlight critical security vulnerabilities in embodied LLM systems and emphasize the urgent need for safeguarding these systems to mitigate potential risks.

Reinforcement learning (RL) with delays is challenging as sensory perceptions lag behind the actual events: the RL agent needs to estimate the real state of its environment based on past observations. State-of-the-art (SOTA) methods typically employ recursive, step-by-step forecasting of states. This can cause the accumulation of compounding errors. To tackle this problem, our novel belief estimation method, named Directly Forecasting Belief Transformer (DFBT), directly forecasts states from observations without incrementally estimating intermediate states step-by-step. We theoretically demonstrate that DFBT greatly reduces compounding errors of existing recursively forecasting methods, yielding stronger performance guarantees. In experiments with D4RL offline datasets, DFBT reduces compounding errors with remarkable prediction accuracy. DFBT’s capability to forecast state sequences also facilitates multi-step bootstrapping, thus greatly improving learning efficiency. On the MuJoCo benchmark, our DFBT-based method substantially outperforms SOTA baselines. Code is available at https://github.com/QingyuanWuNothing/DFBT.

From clinical healthcare to daily living, continuous sensor monitoring across multiple modalities has shown great promise for real-world intelligent decision-making but also faces various challenges. In this work, we argue for modeling such heterogeneous data sources under the multimodal paradigm and introduce a new framework, MAESTRO. We introduce MAESTRO, a novel framework that overcomes key limitations of existing multimodal learning approaches: (1) reliance on a single primary modality for alignment, (2) pairwise modeling of modalities, and (3) assumption of complete modality observations. These limitations hinder the applicability of these approaches in real-world multimodal time-series settings, where primary modality priors are often unclear, the number of modalities can be large (making pairwise modeling impractical), and sensor failures often result in arbitrary missing observations. At its core, MAESTRO facilitates dynamic intra- and cross-modal interactions based on task relevance, and leverages symbolic tokenization and adaptive attention budgeting to construct long multimodal sequences, which are processed via sparse cross-modal attention. The resulting cross-modal tokens are routed through a sparse Mixture-of-Experts (MoE) mechanism, enabling black-box specialization under varying modality combinations. We evaluate MAESTRO against 10 baselines on four diverse datasets spanning three applications, and observe average relative improvements of 4% and 8% over the best existing multimodal and multivariate approaches, respectively, under complete observations. Under partial observations—with up to 40% of missing modalities—MAESTRO achieves an average 9% improvement. Further analysis also demonstrates the robustness and efficiency of MAESTRO's sparse, modality-aware design for learning from dynamic time series.

Intelligent vehicles are systems tightly integrating computation, communication, and physical behavior. The recent proliferation of artificial intelligence, machine learning, the Internet of Things (IoT), and edge-fog–cloud computing envisions that intelligent vehicles are capable of innovative solutions to change our lifestyles. However, the potential benefits come along with new challenges and concerns on security and privacy. This special issue consists of 12 papers and covers broad research contributions, including 1) intrusion detection from in-vehicular networks to connected vehicles, drones, and global positioning systems; 2) authentication with matchmaking encryption, certificateless cryptography, and blockchains for Internet of Vehicles; 3) privacy protection with data sharing and cross-vehicle federated learning; and 4) secure data analysis supported by the cloud. The special issue seeks to assist theoretical analysis, system architecture design, emerging applications, and social impacts of intelligent vehicles.

We present REGLO, a novel methodology for repairing pretrained neural networks to satisfy global robustness and individual fairness properties. A neural network is said to be globally robust with respect to a given input region if and only if all the input points in the region are locally robust. This notion of global robustness also captures the notion of individual fairness as a special case. We prove that any counterexample to a global robustness property must exhibit a corresponding large gradient. For ReLU networks, this result allows us to efficiently identify the linear regions that violate a given global robustness property. By formulating and solving a suitable robust convex optimization problem, REGLO then computes a minimal weight change that will provably repair these violating linear regions.

In this paper, we introduce a novel approach to enhancing neural network-based Simultaneous Localization and Mapping (SLAM) through the integration of model compression techniques and customized hardware architecture that focuses on micro-architectural and dataflow optimizations to improve computational efficiency and performance. Experiments across different scenarios demonstrate that the proposed approach achieves significant improvement.

Anomaly detection in time-series data is crucial for identifying faults, failures, threats, and outliers across a range of applications. Recently, deep learning techniques have been applied to this topic, but they often struggle in real-world scenarios that are complex and highly dynamic, e.g., the normal data may consist of multiple distributions, and various types of anomalies may differ from the normal data to different degrees. In this work, to tackle these challenges, we propose Distribution-Augmented Contrastive Reconstruction (DACR). DACR generates extra data disjoint from the normal data distribution to compress the normal data’s representation space, and enhances the feature extractor through contrastive learning to better capture the intrinsic semantics from time-series data. Furthermore, DACR employs an attention mechanism to model the semantic dependencies among multivariate time-series features, thereby achieving more robust reconstruction for anomaly detection. Extensive experiments conducted on nine benchmark datasets in various anomaly detection scenarios demonstrate the effectiveness of DACR in achieving new state-of-the-art time-series anomaly detection.

Reinforcement learning (RL) is challenging in the common case of delays between events and their sensory perceptions. State-of-the-art (SOTA) state augmentation techniques either suffer from state space explosion or performance degeneration in stochastic environments. To address these challenges, we present a novel *Auxiliary-Delayed Reinforcement Learning (AD-RL)* method that leverages auxiliary tasks involving short delays to accelerate RL with long delays, without compromising performance in stochastic environments. Specifically, AD-RL learns a value function for short delays and uses bootstrapping and policy improvement techniques to adjust it for long delays. We theoretically show that this can greatly reduce the sample complexity. On deterministic and stochastic benchmarks, our method significantly outperforms the SOTAs in both sample efficiency and policy performance. Code is available at https://github.com/QingyuanWuNothing/AD-RL.

Most existing speech disfluency detection techniques only rely upon acoustic data.In this work, we present a practical multimodal disfluency detection approach that leverages available video data together with audio.We curate an audiovisual dataset and propose a novel fusion technique with unified weight-sharing modality-agnostic encoders to learn the temporal and semantic context.Our resilient design accommodates real-world scenarios where the video modality may sometimes be missing during inference.We also present alternative fusion strategies when both modalities are assured to be complete.In experiments across five disfluency-detection tasks, our unified multimodal approach significantly outperforms Audio-only unimodal methods, yielding an average absolute improvement of 10% (i.e., 10 percentage point increase) when both video and audio modalities are always available, and 7% even when video modality is missing in half of the samples.

Trajectory generation and trajectory prediction are two critical tasks in autonomous driving, which generate various trajectories for testing during development and predict the trajectories of surrounding vehicles during operation, respectively. In recent years, emerging data-driven deep learning-based methods have shown great promise for these two tasks in learning various traffic scenarios and improving average performance without assuming physical models. However, it remains a challenging problem for these methods to ensure that the generated/predicted trajectories are physically realistic. This challenge arises because learning-based approaches often function as opaque black boxes and do not adhere to physical laws. Conversely, existing model-based methods provide physically feasible results but are constrained by predefined model structures, limiting their capabilities to address complex scenarios. To address the limitations of these two types of approaches, we propose a new method that integrates kinematic knowledge into neural stochastic differential equations (SDE) and designs a variational autoencoder based on this latent kinematics-aware SDE (LK-SDE) to generate vehicle motions. Experimental results demonstrate that our method significantly outperforms both model-based and learning-based baselines in producing physically realistic and precisely controllable vehicle trajectories. Additionally, it performs well in predicting unobservable physical variables in the latent space.

Cross-domain retrieval (CDR) is finding increasingly broad applications across various domains. However, existing efforts have several major limitations, with the most critical being their reliance on accurate supervision. Recent studies thus focus on achieving unsupervised CDR, but they typically assume that the category spaces across domains are identical, an assumption that is often unrealistic in real-world scenarios. This is because only through dedicated and comprehensive analysis can the category composition of a data domain be obtained, which contradicts the premise of unsupervised scenarios. Therefore, in this work, we introduce the problem of **U**niversal **U**nsupervised **C**ross-**D**omain **R**etrieval (U^2CDR) for the first time and design a two-stage semantic feature learning framework to address it. In the first stage, a cross-domain unified prototypical structure is established under the guidance of an instance-prototype-mixed contrastive loss and a semantic-enhanced loss, to counteract category space differences. In the second stage, through a modified adversarial training mechanism, we ensure minimal changes for the established prototypical structure during domain alignment, enabling more accurate nearest-neighbor searching. Extensive experiments across multiple datasets and scenarios, including close-set, partial, and open-set CDR, demonstrate that our approach significantly outperforms existing state-of-the-art CDR methods and other related methods in solving U^2CDR challenges.

In environments with delayed observation, state augmentation by including actions within the delay window is adopted to retrieve Markovian property to enable reinforcement learning (RL). Whereas, state-of-the-art (SOTA) RL techniques with Temporal-Difference (TD) learning frameworks commonly suffer from learning inefficiency, due to the significant expansion of the augmented state space with the delay. To improve the learning efficiency without sacrificing performance, this work novelly introduces Variational Delayed Policy Optimization (VDPO), reforming delayed RL as a variational inference problem. This problem is further modelled as a two-step iterative optimization problem, where the first step is TD learning in the delay-free environment with a small state space, and the second step is behaviour cloning which can be addressed much more efficiently than TD learning. We not only provide a theoretical analysis of VDPO in terms of sample complexity and performance, but also empirically demonstrate that VDPO can achieve consistent performance with SOTA methods, with a significant enhancement of sample efficiency (approximately 50\% less amount of samples) in the MuJoCo benchmark.

Neural networks (NNs) playing the role of controllers have demonstrated impressive empirical performance on challenging control problems. However, the potential adoption of NN controllers in real-life applications has been significantly impeded by the growing concerns over the safety of these NN-controlled systems (NNCSs). In this work, we present POLAR-Express, an efficient and precise formal reachability analysis tool for verifying the safety of NNCSs. POLAR-Express uses Taylor model (TM) arithmetic to propagate TMs layer-by-layer across an NN to compute an overapproximation of the NN. It can be applied to analyze any feedforward NNs with continuous activation functions, such as ReLU, Sigmoid, and Tanh activation functions that cover the common benchmarks for NNCS reachability analysis. Compared with its earlier prototype POLAR, we develop a novel approach in POLAR-Express to propagate TMs more efficiently and precisely across ReLU activation functions, and provide parallel computation support for TM propagation, thus significantly improving the efficiency and scalability. Across the comparison with six other state-of-the-art tools on a diverse set of common benchmarks, POLAR-Express achieves the best verification efficiency and tightness in the reachable set analysis. POLAR-Express is publicly available athttps://github.com/ChaoHuang2018/POLAR_Tool.

Multi-agent applications have recently gained significant popularity. In many computer vision tasks, a network of agents, such as a team of robots with cameras, could work collaboratively to perceive the environment for efficient and accurate situation awareness. However, these agents often have limited computation, communication, and storage resources. Thus, reducing resource consumption while still providing an accurate perception of the environment becomes an important goal when deploying multi-agent systems. To achieve this goal, we identify and leverage the overlap among different camera views in multi-agent systems for reducing the processing, transmission and storage of redundant/unimportant video frames. Specifically, we have developed two collaborative multi-agent video fast-forwarding frameworks in distributed and centralized settings, respectively. In these frameworks, each individual agent can selectively process or skip video frames at adjustable paces based on multiple strategies via reinforcement learning. Multiple agents then collaboratively sense the environment via either 1) a consensus-based distributed framework calledDMVFthat periodically updates the fast-forwarding strategies of agents by establishing communication and consensus among connected neighbors, or 2) a centralized framework calledMFFNetthat utilizes a central controller to decide the fast-forwarding strategies for agents based on collected data. We demonstrate the efficacy and efficiency of our proposed frameworks on a real-world surveillance video dataset VideoWeb and a new simulated driving dataset CarlaSim, through extensive simulations and deployment on an embedded platform with TCP communication. We show that compared with other approaches in the literature, our frameworks achieve better coverage of important frames, while significantly reducing the number of frames processed at each agent.

Connected and autonomous vehicles (CAVs) can realize many revolutionary applications, but it is expected to have mixed-traffic including CAVs and human-driving vehicles (HVs) together for decades. In this paper, we target the problem of mixed-traffic intersection management and schedule CAVs to control the subsequent HVs. We develop a dynamic programming approach and a mixed integer linear programming (MILP) formulation to optimally solve the problems with the corresponding intersection models. We then propose an MILP-based approach which is more efficient and real-time-applicable than solving the optimal MILP formulation, while keeping good solution quality as well as outperforming the first-come-first-served (FCFS) approach. Experimental results and SUMO simulation indicate that controlling CAVs by our approaches is effective to regulate mixed-traffic even if the CAV penetration rate is low, which brings incentive to early adoption of CAVs.

Neural network-based driving planners have shown great promises in improving task performance of autonomous driving. However, it is critical and yet very challenging to ensure the safety of systems with neural network-based components, especially in dense and highly interactive traffic environments. In this work, we propose a safety-driven interactive planning framework for neural network-based lane changing. To prevent over-conservative planning, we identify the driving behavior of surrounding vehicles and assess their aggressiveness, and then adapt the planned trajectory for the ego vehicle accordingly in an interactive manner. The ego vehicle can proceed to change lanes if a safe evasion trajectory exists even in the predicted worst case; otherwise, it can stay around the current lateral position or return back to the original lane. We quantitatively demonstrate the effectiveness of our planner design and its advantage over baseline methods through extensive simulations with diverse and comprehensive experimental settings, as well as in real-world scenarios collected by an autonomous vehicle company.

The rapid advancement of edge and cloud computing platforms, vehicular ad-hoc networks, and machine learning techniques have brought both opportunities and challenges for next-generation connected and automated vehicles (CAVs). On the one hand, these technologies can enable vehicles to leverage more computing power from edge and cloud servers and to share information with each other and surrounding infrastructures for better situation awareness and more intelligent decision making. On the other hand, the more distributed computing process and the wireless nature of V2X (vehicle-to-everything) communication expose vulnerabilities to various disturbances and attacks. In this paper, we discuss the security and safety challenges for edge- and cloud-enabled CAVs, particularly when they are under environment interferences, execution errors, and malicious attacks, and we will introduce our recent work and future directions in developing system-driven, end-to-end methodologies and tools to address these challenges and ensure system resiliency under uncertainties.

Neural-network-based (NN-based) planners have been increasingly used to enhance the performance of planning for autonomous vehicles. However, it is often difficult for NN-based planners to balance efficiency and safety in complicated scenarios, especially under real-world communication disturbance. To tackle this challenge, we present a safety-guaranteed framework for NN-based planners in connected vehicle environments with communication disturbance. Given any NN-based planner with no safety-guarantee, the framework generates a robust compound planner embedding the NN-based planner to ensure overall system safety. Moreover, with the aid of an information filter for imperfect communication and an aggressive approach for the estimation of the unsafe set, the compound planner could achieve similar or better efficiency than the given NN-based planner. A comprehensive case study of unprotected left turn and extensive simulations demonstrate the effectiveness of our framework.

Speech disfluency research is pivotal to accommodating atypical speakers in mainstream conversational technology. However, the lack of publicly available labeled and unlabeled datasets is a significant bottleneck to such research. While many works use pseudo dysfluency data with proxy labels and formulate a self-supervised task, we see merit in using real-world data. In this work, we consolidate the corpora of publicly available speech disfluency datasets with and without labels and propose DisfluentSiam – an efficient siamese network-based small-scale pretraining pipeline using task-specific data from multiple domains with only 10M trainable parameters. We show that with DisfluentSiam, we achieve an average of 15% boost in performance across five types of dysfluency event detection compared to direct wav2vec 2.0 embeddings. In particular, with only 4-5 mins of labeled data for fine-tuning, the DisfluentSiam demonstrates the advantage of task-specific pretraining with up to 25% higher accuracy.

We present a missingness-aware fusion network (MAFN) to identify a person’s digital phenotype from continuously measured longitudinal multi-modal wearable data. This work is done as a part of Track 1 of e-Prevention: Person Identification and Relapse Detection from Continuous Recordings of Biosignals Signal Processing Grand Challenge at International Conference on Acoustics, Speech, & Signal Processing (ICASSP) 2023. MAFN achieves an accuracy of 91.36% on test data. Additionally, our experiments confirm findings from previous works that kinetic features derived from the accelerometer in-deed contain more discriminative features for person identification task.

Predicting the trajectories of surrounding objects is a critical task for self-driving vehicles and many other autonomous systems. Recent works demonstrate that adversarial attacks on trajectory prediction, where small crafted perturbations are introduced to history trajectories, may significantly mislead the prediction of future trajectories and induce unsafe planning. However, few works have addressed enhancing the robustness of this important safety-critical task. In this paper, we present a novel adversarial training method for trajectory prediction. Compared with typical adversarial training on image tasks, our work is challenged by more random input with rich context and a lack of class labels. To address these challenges, we propose a method based on a semi-supervised adversarial autoencoder, which models disentangled semantic features with domain knowledge and provides additional latent labels for the adversarial training. Extensive experiments with different types of attacks demonstrate that our Semi-supervised Semantics-guided Adversarial Training (SSAT1) method can effectively mitigate the impact of adversarial attacks by up to 73% and outperform other popular defense methods. In addition, experiments show that our method can significantly improve the system's robust generalization to unseen patterns of attacks. We believe that such semantics-guided architecture and advancement on robust generalization is an important step for developing robust prediction models and enabling safe decision making.

It is quite challenging to ensure the safety of reinforcement learning (RL) agents in an unknown and stochastic environment under hard constraints that require the system state not to reach certain specified unsafe regions. Many popular safe RL methods such as those based on the Constrained Markov Decision Process (CMDP) paradigm formulate safety violations in a cost function and try to constrain the expectation of cumulative cost under a threshold. However, it is often difficult to effectively capture and enforce hard reachability-based safety constraints indirectly with such constraints on safety violation cost. In this work, we leverage the notion of barrier function to explicitly encode the hard safety chance constraints, and given that the environment is unknown, relax them to our design of *generative-model-based soft barrier functions*. Based on such soft barriers, we propose a novel safe RL approach with bi-level optimization that can jointly learn the unknown environment and optimize the control policy, while effectively avoiding the unsafe region with safety probability optimization. Experiments on a set of examples demonstrate that our approach can effectively enforce hard safety chance constraints and significantly outperform CMDP-based baseline methods in system safe rates measured via simulations.

The robustness of deep neural networks in safety-critical systems has received significant interest recently, which measures how sensitive the model output is under input perturbations. While most previous works focused on the local robustness property, the studies of the global robustness property, i.e., the robustness in the entire input space, are still lacking. In this work, we formulate the global robustness certification problem for ReLU neural networks and present an efficient approach to address it. Our approach includes a novel interleaving twin-network encoding scheme and an over-approximation algorithm leveraging relaxation and refinement techniques. Its timing efficiency and effectiveness are evaluated and compared with other state-of-the-art global robustness certification methods, and demonstrated via case studies on practical applications.

Predicting the future trajectories of surrounding vehicles based on their history trajectories is a critical task in autonomous driving. However, when small crafted perturbations are introduced to those history trajectories, the resulting anomalous (or adversarial) trajectories can significantly mislead the future trajectory prediction module of the ego vehicle, which may result in unsafe planning and even fatal accidents. Therefore, it is of great importance to detect such anomalous trajectories of the surrounding vehicles for system safety, but few works have addressed this issue. In this work, we propose two novel methods for learning effective and efficient representations for online anomaly detection of vehicle trajectories. Different from general time-series anomaly detection, anomalous vehicle trajectory detection deals with much richer contexts on the road and fewer observable patterns on the anomalous trajectories themselves. To address these challenges, our methods exploit contrastive learning techniques and trajectory semantics to capture the patterns underlying the driving scenarios for effective anomaly detection under supervised and unsupervised settings, respectively. We conduct extensive experiments to demonstrate that our supervised method based on contrastive learning and unsupervised method based on reconstruction with semantic latent space can significantly improve the performance of anomalous trajectory detection in their corresponding settings over various baseline methods. We also demonstrate our methods' generalization ability to detect unseen patterns of anomalies.

Recently significant progress has been made in vehicle prediction and planning algorithms for autonomous driving. However, it remains quite challenging for an autonomous vehicle to plan its trajectory in complex scenarios when it is difficult to accurately predict its surrounding vehicles' behaviors and trajectories. In this work, to maximize performance while ensuring safety, we propose a novel speculative planning framework based on a prediction-planning interface that quantifies both the behavior-level and trajectory-level uncertainties of surrounding vehicles. Our framework leverages recent prediction algorithms that can provide one or more possible behaviors and trajectories of the surrounding vehicles with probability estimation. It adapts those predictions based on the latest system states and traffic environment, and conducts planning to maximize the expected reward of the ego vehicle by considering the probabilistic predictions of all scenarios and ensure system safety by ruling out actions that may be unsafe in worst case. We demonstrate the effectiveness of our approach in improving system performance and ensuring system safety over other baseline methods, via extensive simulations in SUMO on a challenging multi-lane highway lane-changing case study.

Human emotion understanding is pivotal in making conversational technology mainstream. We view speech emotion understanding as a perception task which is a more realistic setting. With varying contexts (languages, demographics etc.) different share of people perceive the same speech segment as a non-unanimous emotion. As part of the ACM Multimedia 2023 Computational Paralinguistics ChallengE (ComParE) in the EMotion Share track, we leverage their rich dataset of multilingual speakers and multi-label regression target of 'emotion share' or perception of that emotion. We demonstrate that the training scheme of different foundation models dictates their effectiveness for tasks beyond speech recognition, especially for non-semantic speech tasks like emotion understanding. This is a very complex task due to multilingual speakers, variability in the target labels, and inherent imbalance in the regression dataset. Our results show that HuBERT-Large with a self-attention-based light-weight sequence model provides 4.6% improvement over the reported baseline.

With the rapid development of information technologies, the computing, networking, and physical elements in industrial environments are becoming tightly amalgamated with each other, resulting in the formation of the so-called Industrial Cyber-Physical Systems (ICPS). These systems forge the core of current real-world networked industrial infrastructures, having a cyber-representation of physical assets through digitalization of data across the enterprise, along the value stream and process engineering life cycle, along the digital thread, and along the supply chain. Typical applications of ICPS include smart grids, digital factory, cognitive and collaborative robots, freight transportation, process control, plant-wide systems, medical monitoring, etc. ICPS often operate in an unpredictable and challenging environment, where various disturbances, such as unplanned natural events, human faults or malicious behaviors, software and hardware failures, etc., may occur during the automation process at runtime. Moreover, ICPS can exhibit strong reconfigurability and evolve structurally for many purposes. During this evolution, new and unforeseen possibilities in the service-oriented business process may appear among various ICPS components. In particular, new “emergent” behaviors may arise that need to be monitored, understood, managed and controlled. When there are significant uncertainties, such emergent behaviors could make the evolved ICPS unstable and unable to meet the quality/performance targets, even resulting in hazards. Well-designed machine-learning techniques have the potential to effectively address the uncertainties and disturbances in the automation of ICPS. They can also facilitate the automated discovery of valuable underlying rules and patterns to improve the performance of ICPS in all phases of their life cycles.

A weakly-hard fault model can be captured by an (m,k) constraint, where 0≤ m ≤ k , meaning that there are at most m bad events (faults) among any k consecutive events. In this article, we use a weakly-hard fault model to constrain the occurrences of faults in system inputs. We develop approaches to verify properties for all possible values of (m,k) , where k is smaller than or equal to a given K , in an exact and efficient manner. By verifying all possible values of (m,k) , we define weakly-hard requirements for the system environment and design a runtime monitor based on counting the number of faults in system inputs. If the system environment satisfies the weakly-hard requirements, then the satisfaction of desired properties is guaranteed; otherwise, the runtime monitor can notify the system to switch to a safe mode. This is especially essential for cyber-physical systems that need to provide guarantees with limited resources and the existence of faults. Experimental results with discrete second-order control, network routing, vehicle following, and lane changing demonstrate the generality and the efficiency of the proposed approaches.

Perceiving the environment for better and more efficient situational awareness is essential in applications such as wildlife surveillance, wildfire detection, crop irrigation, and building management. Energy-harvesting, intermittently-powered sensors have emerged as a zero maintenance solution for long-term environmental perception. However, these devices suffer from intermittent and varying energy supply, which presents three major challenges for executing perceptual tasks: (1) intelligently scaling computation in light of constrained resources and dynamic energy availability, (2) planning communication and sensing tasks, (3) and coordinating sensor nodes to increase the total perceptual range of the network. We propose an adaptive framework, AdaSens, which adapts the operations of intermittently-powered sensor nodes in a coordinated manner to cover as much as possible of the targeted scene, both spatially and temporally, under interruptions and constrained resources. We evaluate AdaSens on a real-world surveillance video dataset, VideoWeb, and show at least 16% improvement on the coverage of the important frames compared with other methods.

Federated learning (FL) has attracted growing attentions via data-private collaborative training on decentralized clients. However, most existing methods unrealistically assume object classes of the overall framework are fixed over time. It makes the global model suffer from significant catastrophic forgetting on old classes in real-world scenarios, where local clients often collect new classes continuously and have very limited storage memory to store old classes. Moreover, new clients with unseen new classes may participate in the FL training, further aggravating the catastrophic forgetting of global model. To address these challenges, we develop a novel Global-Local Forgetting Compensation (GLFC) model, to learn a global class-incremental model for alleviating the catastrophic forgetting from both local and global perspectives. Specifically, to address local forgetting caused by class imbalance at the local clients, we design a class-aware gradient compensation loss and a class-semantic relation distillation loss to balance the forgetting of old classes and distill consistent inter-class relations across tasks. To tackle the global forgetting brought by the non-i.i.d class imbalance across clients, we propose a proxy server that selects the best old global model to assist the local relation distillation. Moreover, a prototype gradient-based communication mechanism is developed to protect the privacy. Our model outperforms state-of-the-art methods by 4.4%~15.1% in terms of average accuracy on representative benchmark datasets. The code is available at https://github.com/conditionWang/FCIL.

In the current control design of safety-critical cyber-physical systems, formal verification techniques are typically applied after the controller is designed to evaluate whether the required properties (e.g., safety) are satisfied. However, due to the increasing system complexity and the fundamental hardness of designing a controller with formal guarantees, such an open-loop process of design-then-verify often results in many iterations and fails to provide the necessary guarantees. In this paper, we propose a correct-by-construction control learning framework that integrates the verification into the control design process in a closed-loop manner, i.e., design-while-verify. Specifically, we leverage the verification results (computed reachable set of the system state) to construct feedback metrics for control learning, which measure how likely the current design of control parameters can meet the required reach-avoid property for safety and goal-reaching. We formulate an optimization problem based on such metrics for tuning the controller parameters, and develop an approximated gradient descent algorithm with a difference method to solve the optimization problem and learn the controller. The learned controller is formally guaranteed to meet the required reach-avoid property. By treating verifiability as a first-class objective and effectively leveraging the verification results during the control learning process, our approach can significantly improve the chance of finding a control design with formal property guarantees, demonstrated in a set of experiments that use model-based or neural network based controllers.

The robustness of deep neural networks has received significant interest recently, especially when being deployed in safety-critical systems, as it is important to analyze how sensitive the model output is under input perturbations. While most previous works focused on the local robustness property around an input sample, the studies of the global robustness property, which bounds the maximum output change under perturbations over the entire input space, are still lacking. In this work, we formulate the global robustness certification for neural networks with ReLU activation functions as a mixed-integer linear programming (MILP) problem, and present an efficient approach to address it. Our approach includes a novel interleaving twin-network encoding scheme, where two copies of the neural network are encoded side-by-side with extra interleaving dependencies added between them, and an over-approximation algorithm leveraging relaxation and refinement techniques to reduce complexity. Experiments demonstrate the timing efficiency of our work when compared with previous global robustness certification methods and the tightness of our over-approximation. A case study of closed-loop control safety verification is conducted, and demonstrates the importance and practicality of our approach for certifying the global robustness of neural networks in safety-critical systems.

Trajectory generation and prediction are two in-terwoven tasks that play important roles in planner evaluation and decision making for intelligent vehicles. Most existing methods focus on one of the two and are optimized to directly output the final generated/predicted trajectories, which only contain limited information for critical scenario augmentation and safe planning. In this work, we propose a novel behavior-aware Trajectory Autoencoder (TAE) that explicitly models drivers' behavior such as aggressiveness and intention in the latent space, using semi-supervised adversarial autoencoder and domain knowledge in transportation. Our model addresses trajectory generation and prediction in a unified architecture and benefits both tasks: the model can generate diverse, controllable and realistic trajectories to enhance planner op-timization in safety-critical and long-tailed scenarios, and it can provide prediction of critical behavior in addition to the final trajectories for decision making. Experimental results demonstrate that our method achieves promising performance on both trajectory generation and prediction.

Connected vehicle (CV) applications promise to revolutionize our transportation systems, improving safety and traffic capacity while reducing environmental footprint. Many CV applications have been proposed towards these goals, with the US Department of Transportation (USDOT) recently initiating some designated deployment sites to enable experimentation and validation. While the focus of this initial development effort is on demonstrating the functionality of a range of proposed applications, recent attacks have demonstrated their vulnerability to application level attacks. In these attacks, a malicious actor operates within the application’s parameters but providing falsified information. This paper explores a framework that protects against such application-level attacks. Then, we analyze the impact of the attacks, showing that an individual attacker can have substantial effects on the safety and efficiency of traffic flow even in the presence of message security standards developed by USDOT, motivating the need for our defense. Our defense relies on physically modeling the vehicles and their interaction using dynamic models and state estimation filters as well as reinforcement learning. It combines these observations with knowledge of application rules and guidelines to capture logic deviations. We demonstrate that the resultant defense, called CVGuard, can accurately and promptly detect attacks, with low false positive rates over a range of attack scenarios for different CV applications.

introduction Share on Introduction to the Special Issue on Artificial Intelligence and Cyber-Physical Systems - Part 2 Authors: Jingtong Hu University of Pittsburgh, Pittsburgh, PA, USA University of Pittsburgh, Pittsburgh, PA, USAView Profile , Qi Zhu Northwestern University, Evanston, IL, USA Northwestern University, Evanston, IL, USAView Profile , Susmit Jha SRI International, Menlo Park, CA, USA SRI International, Menlo Park, CA, USAView Profile Authors Info & Claims ACM Transactions on Cyber-Physical SystemsVolume 6Issue 2April 2022 Article No.: 10pp 1–3https://doi.org/10.1145/3517045Published:19 July 2022Publication History 0citation57DownloadsMetricsTotal Citations0Total Downloads57Last 12 Months57Last 6 weeks8 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Federated learning (FL) is a promising approach for training decentralized data located on local client devices while improving efficiency and privacy. However, the distribution and quantity of the training data on the clients' side may lead to significant challenges such as class imbalance and non-IID (non-independent and identically distributed) data, which could greatly impact the performance of the common model. While much effort has been devoted to helping FL models converge when encountering non-IID data, the imbalance issue has not been sufficiently addressed. In particular, as FL training is executed by exchanging gradients in an encrypted form, the training data is not completely observable to either clients or server, and previous methods for class imbalance do not perform well for FL. Therefore, it is crucial to design new methods for detecting class imbalance in FL and mitigating its impact. In this work, we propose a monitoring scheme that can infer the composition of training data for each FL round, and design a new loss function -- Ratio Loss to mitigate the impact of the imbalance. Our experiments demonstrate the importance of acknowledging class imbalance and taking measures as early as possible in FL training, and the effectiveness of our method in mitigating the impact. Our method is shown to significantly outperform previous methods, while maintaining client privacy.

Future autonomous systems will employ sophisticated machine learning techniques for the sensing and perception of the surroundings and the making corresponding decisions for planning, control, and other actions. They often operate in highly dynamic, uncertain and challenging environment, and need to meet stringent timing, resource, and mission requirements. In particular, it is critical and yet very challenging to ensure the safety of these autonomous systems, given the uncertainties of the system inputs, the constant disturbances on the system operations, and the lack of analyzability for many machine learning methods (particularly those based on neural networks). In this paper, we will discuss some of these challenges, and present our work in developing automated, quantitative, and formalized methods and tools for ensuring the safety of autonomous systems in their design and during their runtime adaptation. We argue that it is essential to take a holistic approach in addressing system safety and other safety-related properties, vertically across the functional, software, and hardware layers, and horizontally across the autonomy pipeline of sensing, perception, planning, and control modules. This approach could be further extended from a single autonomous system to a multi-agent system where multiple autonomous agents perform tasks in a collaborative manner. We will use connected and autonomous vehicles (CAVs) as the main application domain to illustrate the importance of such holistic approach and show our initial efforts in this direction.

The infrastructure-vehicle cooperative autonomous driving approach relies on the cooperation between intelligent roads and intelligent vehicles. This approach is not only safer but also more economical compared to the traditional on-vehicle-only autonomous driving. In this paper, we introduce the real-world deployment experiences of infrastructure-vehicle cooperative autonomous driving by PerceptIn, where a three-stage development roadmap is taken: infrastructure-augmented autonomous driving (IAAD), infrastructure-guided autonomous driving (IGAD), and infrastructure-planned autonomous driving (IPAD). We then discuss the future research challenges and opportunities for such approach.

Neural networks are being increasingly applied to control and decision making for learning-enabled cyber-physical systems (LE-CPSs). They have shown promising performance without requiring the development of complex physical models; however, their adoption is significantly hindered by the concerns on their safety, robustness, and efficiency. In this work, we propose COCKTAIL, a novel design framework that automatically learns a neural network based controller from multiple existing control methods (experts) that could be either model-based or neural network based. In particular, COCKTAIL first performs reinforcement learning to learn an optimal system-level adaptive mixing strategy that incorporates the underlying experts with dynamically-assigned weights, and then conducts a teacher-student distillation with probabilistic adversarial training and regularization to synthesize a student neural network controller with improved control robustness (measured by a safe control rate metric with respect to adversarial attacks or measurement noises), control energy efficiency, and verifiability (measured by the computation time for verification). Experiments on three non-linear systems demonstrate significant advantages of our approach on these properties over various baseline methods.

Given that building loads consume roughly 40% of the energy produced in developed countries, smart buildings with local renewable resources offer a viable alternative towards achieving a greener future. Building temperature control strategies typically employ detailed physical models which require a significant amount of time, information and finesse. Even then, due to unknown building parameters and related inaccuracies, future power demands by the building loads are difficult to estimate. This creates unique challenges in the domain of microgrid economic power dispatch for satisfying building power demands through efficient control and scheduling of renewable and non-renewable local resources in conjunction with supply from the main grid. In this work, we estimate the real-time uncertainties in building loads using Gaussian Process (GP) learning and establish the effectiveness of run time model correction in the context of microgrid economic dispatch.

Future autonomous systems will rely on advanced sensors and deep neural networks for perceiving the environment, and then utilize the perceived information for system planning, control, adaptation, and general decision making. However, due to the inherent uncertainties from the dynamic environment and the lack of methodologies for predicting neural network behavior, the perception modules in autonomous systems often could not provide deterministic guarantees and may sometimes lead the system into unsafe states (e.g., as evident by a number of high-profile accidents with experimental autonomous vehicles). This has significantly impeded the broader application of machine learning techniques, particularly those based on deep neural networks, in safety-critical systems. In this paper, we will discuss these challenges, define open research problems, and introduce our recent work in developing formal methods for quantitatively bounding the output uncertainty of perception neural networks with respect to input perturbations, and leveraging such bounds to formally ensure the safety of system control. Unlike most existing works that only focus on either the perception module or the control module, our approach provides a holistic end-to-end framework that bounds the perception uncertainty and addresses its impact on control.

Building loads consume roughly 40% of the energy produced in developed countries, a significant part of which is invested towards building temperature-control infrastructure. Therein, renewable resource-based microgrids offer a greener and cheaper alternative. This communication explores the possible co-design of microgrid power dispatch and building HVAC (heating, ventilation and air conditioning system) actuations with the objective of effective temperature control under minimised operating cost. For this, we attempt control designs with various levels of abstractions based on information available about microgrid and HVAC system models using the Deep Reinforcement Learning (DRL) technique. We provide control architectures that consider model information ranging from completely determined system models to systems with fully unknown parameter settings and illustrate the advantages of DRL for the design prescriptions.

Data quantity and quality are crucial factors for data-driven learning methods. In some target problem domains, there are not many data samples available, which could significantly hinder the learning process. While data from similar domains may be leveraged to help through domain adaptation, obtaining high-quality labeled data for those source domains themselves could be difficult or costly. To address such challenges on data insufficiency for classification problem in a target domain, we propose a weak adaptation learning (WAL) approach that leverages unlabeled data from a similar source domain, a low-cost weak annotator that produces labels based on task-specific heuristics, labeling rules, or other methods (albeit with inaccuracy), and a small amount of labeled data in the target domain. Our approach first conducts a theoretical analysis on the error bound of the trained classifier with respect to the data quantity and the performance of the weak annotator, and then introduces a multi-stage weak adaptation learning method to learn an accurate classifier by lowering the error bound. Our experiments demonstrate the effectiveness of our approach in learning an accurate classifier with limited labeled data in the target domain and unlabeled data in the source domain.

In the development of advanced driver-assistance systems (ADAS) and autonomous vehicles, machine learning techniques that are based on deep neural networks (DNNs) have been widely used for vehicle perception. These techniques offer significant improvement on average perception accuracy over traditional methods, however have been shown to be susceptible to adversarial attacks, where small perturbations in the input may cause significant errors in the perception results and lead to system failure. Most prior works addressing such adversarial attacks focus only on the sensing and perception modules. In this work, we propose an end-to-end approach that addresses the impact of adversarial attacks throughout perception, planning, and control modules. In particular, we choose a target ADAS application, the automated lane centering system in OpenPilot, quantify the perception uncertainty under adversarial attacks, and design a robust planning and control module accordingly based on the uncertainty analysis. We evaluate our proposed approach using both public dataset and production-grade autonomous driving simulator. The experiment results demonstrate that our approach can effectively mitigate the impact of adversarial attack and can achieve 55% ~ 90% improvement over the original OpenPilot.

While connected vehicle (CV) applications have the potential to revolutionize traditional transportation system, cyber and physical attacks on them may lead to disastrous consequences. In this work, we propose an efficient dual cyber-physical blockchain framework to build trust and secure communication for CV applications. Our approach incorporates blockchain technology and physical sensing capabilities of vehicles to quickly react to attacks in a large-scale vehicular network, with low resource overhead. We explore the application of our framework to three CV applications, i.e., highway merging, intelligent intersection management, and traffic network with route choices. Simulation results demonstrate the effectiveness of our blockchain-based framework in defending against spoofing attacks, bad mouthing attacks, and Sybil and voting attacks. We also provide analysis to show the timing and resource efficiency of our framework.

The emerging vehicular edge computing (VEC) technology has the potential to bring revolutionary development to vehicular ad hoc network (VANET). However, the edge computing servers (ECSs) are subjected to a variety of security threats. One of the most dangerous types of security attacks is the Sybil attack, which can create fabricated virtual vehicles (called Sybil vehicles) to significantly overload ECSs' limited computation resources and thus disrupt legitimate vehicles' edge computing applications. In this paper, we present a novel Sybil attack detection system on ECSs that is based on the design of a credibility enhanced temporal graph convolutional network. Our approach can identify the malicious vehicles in a dynamic traffic environment while preserving the legitimate vehicles' privacy, particularly their local position information. We evaluate our proposed approach in the SUMO simulator. The results demonstrate that our proposed detection system can accurately identify most Sybil vehicles while maintaining a low error rate.

As a fundamental component in the autonomous driving technology stack, High Definition Maps (HD map) provide high-precision descriptions of the environment. It enables extremely accurate perception and localization while improving the efficiency of path planning. However, the HD map's extremely large data volume poses great challenges for the real-time and safety requirements of autonomous driving. Based on our real-world deployment experiences, we first demonstrate how the existing data transmission mechanism is weak in supporting HD map services. To address this problem, we propose an HD map data service mechanism on top of Vehicle-to-Infrastructure (V2I) data transmission under a tight time and energy budget. By this mechanism, the selected road side unit (RSU) nodes cooperate on map provisioning tasks and transmit HD map data proportionately. Furthermore, we model the real-time map data service into a partial knapsack problem and develop a greedy data transmission algorithm. Experimental results confirm that the proposed mechanism can ensure the real-time HD map data service meanwhile meeting the energy limits.

Recent applications of the Internet of Things and cyber-physical systems require the integration of many sensing and control tasks into resource-constrained embedded devices. Such tasks can often tolerate a bounded number of timing violations. The concept of weakly hard real-time systems can effectively improve resource efficiency without sacrificing system safety. However, the existing studies have limitations on their practical use due to the restrictions imposed on the task timing behavior, high analysis complexity, and the lack of multicore support. In this article, we propose a new job-class-level fixed-priority preemptive scheduler and its schedulability analysis framework for weakly hard real-time tasks. Our proposed scheduler employs the meet-oriented classification of jobs of a task in order to reduce the worst-case temporal interference imposed on other tasks. Under this approach, each job is associated with a “job-class” that is determined by the number of deadlines previously met (with a bounded number of consecutively missed deadlines). This approach allows decomposing the complex weakly hard schedulability problem into two subproblems that are easier to solve: 1) analyzing the response time of a job with each job-class, which can be done by an extension of the existing task-level analysis and 2) finding possible job-class patterns, which can be modeled as a simple reachability tree. We also present a semipartitioned task allocation method for multicore platforms, which enhances the schedulability of weakly hard tasks under the proposed scheduling framework. Experimental results indicate that our scheduler outperforms the prior work in terms of task schedulability and analysis time complexity. We have also implemented a prototype of a job-class-level scheduler in the Linux kernel running on Raspberry Pi with acceptably small-runtime overhead.

introduction Share on Introduction to the Special Issue on Artificial Intelligence and Cyber-Physical Systems: Part 1 Authors: Jingtong Hu University of Pittsburgh, Pittsburgh, PA, USA University of Pittsburgh, Pittsburgh, PA, USAView Profile , Qi Zhu Northwestern University, Evanston, IL, USA Northwestern University, Evanston, IL, USAView Profile , Susmit Jha SRI International, Menlo Park, CA, USA SRI International, Menlo Park, CA, USAView Profile Authors Info & Claims ACM Transactions on Cyber-Physical SystemsVolume 5Issue 4October 2021 Article No.: 33pp 1–3https://doi.org/10.1145/3471164Online:22 September 2021Publication History 1citation77DownloadsMetricsTotal Citations1Total Downloads77Last 12 Months77Last 6 weeks8 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

During the operation of many real-time safety-critical systems, there are often strong needs for adapting to a dynamic environment or evolving mission objectives, e.g., increasing sampling and control frequencies of some functions to improve their performance under certain situations. However, a system's ability to adapt is often limited by tight resource constraints and rigid periodic execution requirements. In this work, we present a cross-layer approach to improve system adaptability by allowing proactive skipping of task executions, so that the resources can be either saved directly or re-allocated to other tasks for their performance improvement. Our approach includes three novel elements: (1) formal methods for deriving the feasible skipping choices of control tasks with safety guarantees at the functional layer, (2) a schedulability analysis method for assessing system feasibility at the architectural layer under allowed task job skippings, and (3) a runtime adaptation algorithm that efficiently explores job skipping choices and task priorities for meeting system adaptation requirements while ensuring system safety and timing correctness. Experiments demonstrate the effectiveness of our approach in meeting system adaptation needs.

The majority of today's power-hungry datacenters are physically co-located with office rooms in mixed-use buildings (MUBs). The heating, ventilation, and air conditioning (HVAC) system within each MUB is often shared or partially-shared between datacenter rooms and office zones, for removing the heat generated by computing equipment and maintaining desired room temperature for building tenants. To effectively reduce the total energy cost of MUBs, it is important to leverage the scheduling flexibility in both the HVAC system and the datacenter workload. In this work, we formulate both HVAC control and datacenter workload scheduling as a Markov decision process (MDP), and propose a deep reinforcement learning (DRL) based algorithm for minimizing the total energy cost while maintaining desired room temperature and meeting datacenter workload deadline constraints. Moreover, we also develop a heuristic DRL-based algorithm to enable interactive workload allocation among geographically distributed MUBs for further energy reduction. The experiment results demonstrate that our regular DRL-based algorithm can achieve up to 26.9 percent cost reduction for a single MUB, when compared with a baseline strategy. Our heuristic DRL-based algorithm can reduce the total energy cost by an additional 5.5 percent, when intelligently allocating interactive workload for multiple geographically distributed MUBs.

We introduce SAW, a tool for safety analysis of weakly-hard systems, in which traditional hard timing constraints are relaxed to allow bounded deadline misses for improving design flexibility and runtime resiliency. Safety verification is a key issue for weakly-hard systems, as it ensures system safety under allowed deadline misses. Previous works are either for linear systems only, or limited to a certain type of nonlinear systems (e.g., systems that satisfy exponential stability and Lipschitz continuity of the system dynamics). In this work, we propose a new technique for infinite-time safety verification of general nonlinear weakly-hard systems. Our approach first discretizes the safe state set into grids and constructs a directed graph, where nodes represent the grids and edges represent the reachability relation. Based on graph theory and dynamic programming, our approach can effectively find the safe initial set (consisting of a set of grids), from which the system can be proven safe under given weakly-hard constraints. Experimental results demonstrate the effectiveness of our approach, when compared with the state-of-the-art. An open source implementation of our tool is available at https://github.com/551100kk/SAW . The virtual machine where the tool is ready to run can be found at https://www.csie.ntu.edu.tw/~r08922054/SAW.ova .

Control schemes for autonomous systems are often designed in a way that anticipates the worst case in any situation. At runtime, however, there could exist opportunities to leverage the characteristics of specific environment and operation context for more efficient control. In this work, we develop an online intermittent-control framework that combines formal verification with model-based optimization and deep reinforcement learning to opportunistically skip certain control computation and actuation to save actuation energy and computational resources without compromising system safety. Experiments on an adaptive cruise control system demonstrate that our approach can achieve significant energy and computation savings.

Future autonomous systems will employ complex sensing, computation, and communication components for their perception, planning, control, and coordination, and could operate in highly dynamic and uncertain environment with safety and security assurance. To realize this vision, we have to better understand and address the challenges from the "unknowns" - the unexpected disturbances from component faults, environmental interference, and malicious attacks, as well as the inherent uncertainties in system inputs, model inaccuracies, and machine learning techniques (particularly those based on neural networks). In this work, we will discuss these challenges, propose our approaches in addressing them, and present some of the initial results. In particular, we will introduce a cross-layer framework for modeling and mitigating execution uncertainties (e.g., timing violations, soft errors) with weakly-hard paradigm, quantitative and formal methods for ensuring safe and time-predictable application of neural networks in both perception and decision making, and safety-assured adaptation strategies in dynamic environment.

Many safety-critical real-time systems operate under harsh environment and are subject to soft errors caused by transient or intermittent faults. It is critical and yet often very challenging to apply fault tolerance techniques in these systems, due to resource limitations and stringent constraints on timing and functionality. In this work, we leverage the concept of weakly-hard constraints, which allows task deadline misses in a bounded manner, to improve system's capability to accommodate fault tolerance techniques while ensuring timing and functional correctness. In particular, we a) quantitatively measure control cost under different deadline hit/miss scenarios and identify weak-hard constraints that guarantee control stability; b) employ typical worst-case analysis (TWCA) to bound the number of deadline misses and approximate system control cost; c) develop an event-based simulation method to check the task execution pattern and evaluate system control cost for any given solution; and d) develop a meta-heuristic algorithm that consists of heuristic methods and a simulated annealing procedure to explore the design space. Our experiments on an industrial case study and synthetic examples demonstrate the effectiveness of our approach.

Neural networks have been increasingly applied to control in learning-enabled cyber-physical systems (LE-CPSs) and demonstrated great promises in improving system performance and efficiency, as well as reducing the need for complex physical models. However, the lack of safety guarantees for such neural network based controllers has significantly impeded their adoption in safety-critical CPSs. In this work, we propose a controller adaptation approach that automatically switches among multiple controllers, including neural network controllers, to guarantee system safety and improve energy efficiency. Our approach includes two key components based on formal methods and machine learning. First, we approximate each controller with a Bernstein-polynomial based hybrid system model under bounded disturbance, and compute a safe invariant set for each controller based on its corresponding hybrid system. Intuitively, the invariant set of a controller defines the state space where the system can always remain safe under its control. The union of the controllers' invariants sets then define a safe adaptation space that is larger than (or equal to) that of each controller. Second, we develop a deep reinforcement learning method to learn a controller switching strategy for reducing the control/actuation energy cost, while with the help of a safety guard rule, ensuring that the system stays within the safe space. Experiments on a linear adaptive cruise control system and a non-linear Van der Pol's oscillator demonstrate the effectiveness of our approach on energy saving and safety enhancement.

Autonomous vehicles (AVs) are expected to be an integral part of the next generation of transportation systems, where they will share the transportation network with human-driven vehicles during the transition period. In this work, we model the interactions between vehicles (two AVs or an AV and a human-driven vehicle) in a lane changing process by leveraging the Stackelberg game. We explicitly model driving attitudes for both vehicles involved in lane changing. We design five cases, in which the two vehicles have different levels of knowledge, and make different assumptions, about the driving attitude of the rival. We conduct theoretical analysis and simulations for different cases in two lane changing scenarios, namely changing lanes from a higher-speed lane to a lower-speed lane, and from a lower-speed lane to a higher-speed lane. We use four metrics (fuel consumption, discomfort, minimum distance gap and lane change success rate) to investigate how the performance of a single vehicle and that of the system will be influenced by the level of information sharing, and whether a vehicle trajectory optimized based on selfish criteria can provide system-level benefits.

In many intelligent systems, a network of agents collaboratively perceives the environment for better and more efficient situation awareness. As these agents often have limited resources, it could be greatly beneficial to identify the content overlapping among camera views from different agents and leverage it for reducing the processing, transmission and storage of redundant/unimportant video frames. This paper presents a consensus-based distributed multi-agent video fast-forwarding framework, named DMVF, that fast-forwards multi-view video streams collaboratively and adaptively. In our framework, each camera view is addressed by a reinforcement learning based fast-forwarding agent, which periodically chooses from multiple strategies to selectively process video frames and transmits the selected frames at adjustable paces. During every adaptation period, each agent communicates with a number of neighboring agents, evaluates the importance of the selected frames from itself and those from its neighbors, refines such evaluation together with other agents via a system-wide consensus algorithm, and uses such evaluation to decide their strategy for the next period. Compared with approaches in the literature on a real-world surveillance video dataset VideoWeb, our method significantly improves the coverage of important frames and also reduces the number of frames processed in the system.

In practice, safety-critical cyber-physical systems (CPS) are often implemented using high quality-of-service (QoS) resources to provide maximum performance in all scenarios. Such implementations are oblivious to the changing criticality levels of CPS based on their physical dynamics (e.g., steady or transient state). Considering that high-QoS resources are constrained for cost-sensitive CPS, such criticality-oblivious implementations are highly inefficient. Towards a tighter dimensioning of these resources, state-of-the-art approaches have considered multi-QoS resources and studied criticality-aware dynamic resource allocation along the lines of mixed-criticality systems. However, these approaches have high implementation overheads. Moreover, in safety-critical domains like automotive and avionics, certification of such dynamic policies is challenging and the implementation platforms typically do not support dynamic reconfiguration. To address these challenges, we present GoodSpread that uses a static scheduling strategy and offers the same performance guarantees while saving resources (more than 50 % in certain cases) compared to the existing dynamic schemes. The main idea here is to spread the high-QoS resources as uniformly as possible over time in order to accommodate the uncertainty of when the criticality level might change. Our proposed strategy studies the physical dynamics to determine the spread factor, i.e., how often the high-QoS resources need to be provisioned. We further propose an extensibility-driven optimization approach to obtain a static schedule that will accommodate future workloads on the remaining resources with maximum flexibility.

Instance segmentation is a promising yet challenging topic in computer vision. Recent approaches such as Mask R-CNN typically divide this problem into two parts - a detection component and a mask generation branch, and mostly focus on the improvement of the detection part. In this paper, we present an approach that extends Mask R-CNN with five novel techniques for improving the mask generation branch and reducing the conflicts between the mask branch and the detection component in training. These five techniques are independent to each other and can be flexibly utilized in building various instance segmentation architectures for increasing the overall accuracy. We demonstrate the effectiveness of our approach with tests on the COCO dataset.

In this article, we present a layer-wise refinement method for neural network output range analysis. While approaches such as nonlinear programming (NLP) can directly model the high nonlinearity brought by neural networks in output range analysis, they are known to be difficult to solve in general. We propose to use a convex polygonal relaxation (overapproximation) of the activation functions to cope with the nonlinearity. This allows us to encode the relaxed problem into a mixedinteger linear program (MILP), and control the tightness of the relaxation by adjusting the number of segments in the polygon. Starting with a segment number of 1 for each neuron, which coincides with a linear programming (LP) relaxation, our approach selects neurons layer by layer to iteratively refine this relaxation. To tackle the increase of the number of integer variables with tighter refinement, we bridge the propagation-based method and the programming-based method by dividing and sliding the layerwise constraints. Specifically, given a sliding number s, for the neurons in layer l, we only encode the constraints of the layers between l - s and l. We show that our overall framework is sound and provides a valid overapproximation. Experiments on deep neural networks demonstrate significant improvement on output range analysis precision using our approach compared to the state-of-the-art.

The rapid development of vehicular network and autonomous driving technologies provides opportunities to significantly improve transportation safety and efficiency. One promising application is centralized intelligent intersection management, where an intersection manager accepts requests from approaching vehicles (via vehicle-to-infrastructure communication messages) and schedules the order for those vehicles to safely crossing the intersection. However, communication delays and packet losses may occur due to the unreliable nature of wireless communication or malicious security attacks (e.g., jamming and flooding), and could cause deadlocks and unsafe situations. In our previous work, we considered these issues and proposed a delay-tolerant intersection management protocol for intersections with a single lane in each direction. In this work, we address key challenges in efficiency and deadlock when there are multiple lanes from each direction, and propose a delay-tolerant protocol for general multi-lane intersection management. We prove that this protocol is deadlock free, safe, and satisfies the liveness property. Furthermore, we extend the traffic simulation suite SUMO with communication modules, implement our protocol in the extended simulator, and quantitatively analyze its performance with the consideration of communication delays. Finally, we also model systems that use smart traffic lights with various back-pressure scheduling methods in SUMO, including the basic back-pressure control, the capacity-aware back-pressure control, and the adaptive max-pressure control. We then compare our delay-tolerant intelligent intersection protocol with smart traffic lights that use the three back-pressure scheduling methods, in the case of a network of interconnected intersections. Simulation results demonstrate that our approach significant outperforms the smart traffic lights under normal operation (i.e., when the communication delay is not too large).

Weakly-hard systems are real-time systems that can tolerate occasional deadline misses in a bounded manner. Compared with traditional systems with hard deadline constraints, they provide more scheduling flexibility, and thus expand the design space for system configuration and reconfiguration. A key question for such a system is precisely to what degree it can tolerate deadline misses while still meeting its functional requirements. In this paper, we provide a formal treatment to the verification problem of a general class of weakly-hard systems. We discuss relaxation and over-approximation techniques for managing the complexity of reachability analysis, and develop algorithms based upon these for verifying the safety of weakly-hard systems. Experiments demonstrate the effectiveness of our approach in understanding the impact of and guiding the selection among different weakly-hard constraints.

Neural networks are widely used in many applications ranging from classification to control. While these networks are composed of simple arithmetic operations, they are challenging to formally verify for properties such as reachability due to the presence of nonlinear activation functions. In this paper, we make the observation that Lipschitz continuity of a neural network not only can play a major role in the construction of reachable sets for neural-network controlled systems but also can be systematically controlled during training of the neural network. We build on this observation to develop a novel verification-aware knowledge distillation framework that transfers the knowledge of a trained network to a new and easier-to-verify network. Experimental results show that our method can substantially improve reachability analysis of neural-network controlled systems for several state-of-the-art tools.

For many embedded systems, such as automotive electronic systems, security has become a pressing challenge. Limited resources and tight timing constraints often make it difficult to apply even lightweight authentication and intrusion detection schemes, especially when retrofitting existing designs. Moreover, traditional hard deadline assumption is insufficient to describe control tasks that have certain degrees of robustness and can tolerate some deadline misses while satisfying functional properties such as stability. In this work, we explore feasible weakly-hard constraints on control tasks, and then leverage the scheduling flexibility from those allowed misses to enhance system's capability for accommodating security monitoring tasks. We develop a co-design approach that 1) sets feasible weakly-hard constraints on control tasks based on quantitative analysis, ensuring the satisfaction of control stability and performance requirements; and 2) optimizes the allocation, priority, and period assignment of security monitoring tasks, improving system security while meeting timing constraints (including the weakly-hard constraints on control tasks). Experimental results on an industrial case study and a set of synthetic examples demonstrated the significant potential of leveraging weakly-hard constraints to improve security and the effectiveness of our approach in exploring the design space to fully realize such potential.

Many cyber-physical applications including sensing and control operations can tolerate a certain degree of timing violations as long as the number of the violations are predictably bounded. The notion of weakly-hard real-time systems has been studied to capture this effect, but existing work reveals limitations for practical use due the restrictions imposed on timing model and the high complexity of analysis. In this paper, we propose a new job-class-level fixed-priority preemptive scheduler and its schedulability analysis framework for sporadic tasks with weakly-hard real-time constraints. Our proposed scheduler employs the meet-oriented classification of jobs of a task in order to reduce the worst-case temporal interference imposed on other tasks. Under this approach, each job is associated with a "job-class" that is determined by the number of deadlines previously met (with a bounded number of consecutively-missed deadlines). This approach also allows decomposing the complex weakly-hard schedulability problem into two sub-problems that are easier to solve: (1) analyzing the response time of a job with each job-class, which can be done by an extension of the existing task-level analysis, and (2) finding possible job-class patterns, which can be modeled as a simple reachability tree. Experimental results indicate that our scheduler outperforms prior work in terms of task schedulability and analysis time complexity. We have also implemented a prototype of a job-class-level scheduler in the Linux kernel running on Raspberry Pi with acceptably-small runtime overhead.

Multicore platforms are becoming increasingly popular in real-time systems. One of the major challenges in designing multicore real-time systems is ensuring consistent and timely access to shared resources. Lock-based protection mechanisms such as MPCP and MSRP have been proposed to guarantee mutually exclusive access in multicore systems at the expense of blocking. In this article, we consider partitioning and scheduling in multicore real-time systems with resource sharing. We first propose a resource-aware task partitioning algorithm for systems with lock-based protection. Wait-free methods, which ensure consistent access to shared memory resources with negligible blocking at the expense of additional memory space, are a suitable alternative when the shared resource is a communication buffer. We propose several approaches to solve the joint problem of task partitioning and the selection of a data consistency mechanism (lock-based or wait-free). The problem is first formulated as an Integer Linear Programming (ILP). For large systems where an ILP solution is not scalable, we propose two heuristic algorithms. Experimental results compare the effectiveness of the proposed approaches in finding schedulable systems with low memory cost and show how the use of wait-free methods can significantly improve schedulability.

Applying neural networks as controllers in dynamical systems has shown great promises. However, it is critical yet challenging to verify the safety of such control systems with neural-network controllers in the loop. Previous methods for verifying neural network controlled systems are limited to a few specific activation functions. In this work, we propose a new reachability analysis approach based on Bernstein polynomials that can verify neural-network controlled systems with a more general form of activation functions, i.e., as long as they ensure that the neural networks are Lipschitz continuous. Specifically, we consider abstracting feedforward neural networks with Bernstein polynomials for a small subset of inputs. To quantify the error introduced by abstraction, we provide both theoretical error bound estimation based on the theory of Bernstein polynomials and more practical sampling based error bound estimation, following a tight Lipschitz constant estimation approach based on forward reachability analysis. Compared with previous methods, our approach addresses a much broader set of neural networks, including heterogeneous neural networks that contain multiple types of activation functions. Experiment results on a variety of benchmarks show the effectiveness of our approach.

Hybrid electric vehicles employ a hybrid propulsion system to combine the energy efficiency of electric motor and a long driving range of internal combustion engine, thereby achieving a higher fuel economy as well as convenience compared with conventional ICE vehicles. However, the relatively complicated powertrain structures of HEVs necessitate an effective power management policy to determine the power split between ICE and EM. In this work, we propose a deep reinforcement learning framework of the HEV power management with the aim of improving fuel economy. The DRL technique is comprised of an offline deep neural network construction phase and an online deep Q-learning phase. Unlike traditional reinforcement learning, DRL presents the capability of handling the high dimensional state and action space in the actual decision-making process, making it suitable for the HEV power management problem. Enabled by the DRL technique, the derived HEV power management policy is close to optimal, fully model-free, and independent of a prior knowledge of driving cycles. Simulation results based on actual vehicle setup over real-world and testing driving cycles demonstrate the effectiveness of the proposed framework on optimizing HEV fuel economy.

For many applications with limited computation, communication, storage and energy resources, there is an imperative need of computer vision methods that could select an informative subset of the input video for efficient processing at or near real time. In the literature, there are two relevant groups of approaches: generating a "trailer" for a video or fast-forwarding while watching/processing the video. The first group is supported by video summarization techniques, which require processing of the entire video to select an important subset for showing to users. In the second group, current fast-forwarding methods depend on either manual control or automatic adaptation of playback speed, which often do not present an accurate representation and may still require processing of every frame. In this paper, we introduce FastForwardNet (FFNet), a reinforcement learning agent that gets inspiration from video summarization and does fast-forwarding differently. It is an online framework that automatically fast-forwards a video and presents a representative subset of frames to users on the fly. It does not require processing the entire video, but just the portion that is selected by the fast-forward agent, which makes the process very computationally efficient. The online nature of our proposed method also enables the users to begin fast-forwarding at any point of the video. Experiments on two real-world datasets demonstrate that our method can provide better representation of the input video (about 6%-20% improvement on coverage of important frames) with much less processing requirement (more than 80% reduction in the number of frames processed).

In this special discussion session on machine learning, the panel members discuss various issues related to building secure and low power neuromorphic systems. The security of neuromorphic systems may be discussed in term of the reliability of the model, trust in the model, and security of the underlying hardware. The low power aspect of neuromorphic computing systems may be discussed in terms of adaptation of new devices and technologies, the adaptation of new computational models, development of heterogeneous computing frameworks, or dedicated engines for processing neuromorphic models. This session may include discussion on the design space of such supporting hardware, exploring tradeoffs between power/energy, security, scalability, hardware area, performance, and accuracy.

Connected vehicle applications such as autonomous intersections and intelligent traffic signals have shown great promises in improving transportation safety and efficiency. However, security is a major concern in these systems, as vehicles and surrounding infrastructures communicate through ad-hoc networks. In this paper, we will first review security vulnerabilities in connected vehicle applications. We will then introduce and discuss some of the defense mechanisms at network and system levels, including (1) the Security Credential Management System (SCMS) proposed by the United States Department of Transportation, (2) an intrusion detection system (IDS) that we are developing and its application on collaborative adaptive cruise control, and (3) a partial consensus mechanism and its application on lane merging. These mechanisms can assist to improve the security of connected vehicle applications.

Smart buildings in the future are complex cyber-physical-human systems that involve close interactions among embedded platform (for sensing, computation, communication and control), mechanical components, physical environment, building architecture, and occupant activities. The design and operation of such buildings require a new set of methodologies and tools that can address these heterogeneous domains in a holistic, quantitative and automated fashion. In this paper, we will present our design automation methods for improving building energy efficiency and offering comfortable services to occupants at low cost. In particular, we will highlight our work in developing both model-based and data-driven approaches for building automation and control, including methods for co-scheduling heterogeneous energy demands and supplies, for integrating intelligent building energy management with grid optimization through a proactive demand response framework, for optimizing HVAC control with deep reinforcement learning, and for accurately measuring in-building temperature by combining prior modeling information with few sensor measurements based upon Bayesian inference.

With rapid advancement of advanced driver assistance systems (ADAS) and autonomous driving functions, modern vehicles have become ever more intelligent than before. Sophisticated machine learning techniques have being developed for vehicle perception, planning and control. However, this also brings significant challenges to the design, implementation and validation of automotive systems, stemming from the fast-growing functional complexity, the adoption of advanced architectural components such as multicore CPUs and GPUs, the dynamic and uncertain physical environment, and the stringent requirements on various system metrics such as safety, security, reliability, performance, fault tolerance, extensibility, and cost. To address these challenges, new design methodologies, algorithms and tools are greatly needed. This paper will discuss the challenges in designing next-generation connected and autonomous vehicles, and the need of design automation techniques to tackle them.

Cyber-physical system (CPS) analysis and design are challenging due to the intrinsic heterogeneity of those systems. Today, CPSs are often designed by leveraging existing solutions and by adding cyber components to an existing physical system, thus decomposing the design into two separate phases. In this paper, we argue that the codesign of the cyber and physical components would expose solutions that are better under all aspects, such as safety, efficiency, security, performance, reliability, fault tolerance, and extensibility. To do so, automated codesign tools are a necessity due to the complexity of the problems at hand. In the paper, we will discuss the key needs and challenges in developing modeling, simulation, synthesis, validation, and verification tools for CPS codesign, present promising codesign approaches from our teams and others, and point out where additional research is needed.

Cyber-physical systems (CPSs) are characterized by the seamless integration and close interaction of cyber components (e.g., sensors, computation nodes, communication networks) and physical processes (e.g., mechanical devices, physical environment, humans). The cyber components monitor, analyze, and control the physical processes, and react to their changes through feedback loops. A classic example of CPSs is autonomous vehicles. These vehicles collect information of the surrounding physical environment via heterogeneous sensors such as cameras, radar, and LIDAR; process and analyze the multi-modal information at real time with advanced computing devices such as GPUs, application-specific SoCs and multicore CPUs; automatically make planning and control decisions; and continuously actuate the corresponding mechanical components. The cyber components of autonomous vehicles are much more intelligent and complex than those of traditional vehicles, and interact more directly and closely with the physical environment.

Aiming at high performance, more and more Cyber-Physical Systems (CPSs) adopt Multiprocessor System-on-Chips (MPSoCs) as computation units. However, due to increasing integration of transistors on a die, the power densities together with performance variations of MPSoC chips have been increasing dramatically. Consequently, the MPSoC-based CPSs might become unsustainable and unreliable. Although various Task Allocation and Scheduling (TAS) heuristics have been proposed to minimize the hotspot time (i.e., duration of thermal emergency) and energy consumption of MPSoC designs, few of them can guarantee the highest performance yield under process variations without violating energy, thermal and timing constraints. To address these challenges, this paper proposes a novel energy- and thermal-aware TAS evaluation and optimization framework. Based on statistical model checking techniques, our approach enables accurate modeling and reasoning of the performance yield of real-time MPSoC designs under joint energy and thermal constraints. To enable system-level design space exploration, we propose a regression analysis-based method that can drastically reduce the overall exploration efforts. Experimental results show that our fully-automated approach can not only allow accurate sustainability-oriented reasoning of TAS solutions under specified thermal and energy constraints, but also enable the quick search of optimal TAS solutions on different MPSoC architectures with the highest performance yield.

Buildings account for nearly 40% of the total energy consumption in the United States, about half of which is used by the HVAC (heating, ventilation, and air conditioning) system. Intelligent scheduling of building HVAC systems has the potential to significantly reduce the energy cost. However, the traditional rule-based and model-based strategies are often inefficient in practice, due to the complexity in building thermal dynamics and heterogeneous environment disturbances. In this work, we develop a data-driven approach that leverages the deep reinforcement learning (DRL) technique, to intelligently learn the effective strategy for operating the building HVAC systems. We evaluate the performance of our DRL algorithm through simulations using the widely-adopted EnergyPlus tool. Experiments demonstrate that our DRL-based algorithm is more effective in energy cost reduction compared with the traditional rule-based approach, while maintaining the room temperature within desired range.

Increasingly more software-based applications are being developed and deployed in modern vehicles. As a result, the extensibility of a system design has become an important issue in order to accommodate more future applications and update of existing ones on one hand and reduce the effort and cost of re-design, test and validation on the other. In this paper, we discuss the extensibility-driven design in the automotive E/E architecture. We explain the motivation for such a design objective and discuss the definition of extensibility metric and extensibility-driven design methods under two different setting, namely the system based on CAN bus and FlexRay bus. Based on these two examples, we illustrate the importance and advantages of extensibility-driven design in the automotive E/E architecture.

The recent breakthroughs of deep reinforcement learning (DRL) technique in Alpha Go and playing Atari have set a good example in handling large state and actions spaces of complicated control problems. The DRL technique is comprised of (i) an offline deep neural network (DNN) construction phase, which derives the correlation between each state-action pair of the system and its value function, and (ii) an online deep Q-learning phase, which adaptively derives the optimal action and updates value estimates. In this paper, we first present the general DRL framework, which can be widely utilized in many applications with different optimization objectives. This is followed by the introduction of three specific applications: the cloud computing resource allocation problem, the residential smart grid task scheduling problem, and building HVAC system optimal control problem. The effectiveness of the DRL technique in these three cyber-physical applications have been validated. Finally, this paper investigates the stochastic computing-based hardware implementations of the DRL framework, which consumes a significant improvement in area efficiency and power consumption compared with binary-based implementation counterparts.

With the fast development of autonomous driving and vehicular communication technologies, intelligent transportation systems that are based on VANET (Vehicular Ad-Hoc Network) have shown great promise. For instance, through V2V (Vehicle-to-Vehicle) and V2I (Vehicle-to-Infrastructure) communication, intelligent intersections allow more fine-grained control of vehicle crossings and significantly enhance traffic efficiency. However, the performance and safety of these VANET-based systems could be seriously impaired by communication delays and packet losses, which may be caused by network congestion or by malicious attacks that target communication timing behavior. In this paper, we quantitatively model and analyze some of the timing and security issues in transportation networks with VANET-based intelligent intersections. In particular, we demonstrate how communication delays may affect the performance and safety of a single intersection and of multiple interconnected intersections, and present our delay-tolerant intersection management protocols. We also discuss the issues of such protocols when the vehicles are non-cooperative and how they may be addressed with game theory.

The design of automotive electronic systems needs to address a variety of important objectives, including safety, performance, fault tolerance, reliability, security, extensibility, etc. To obtain a feasible design, timing constraints must be satisfied and latencies of certain functional paths should not exceed their deadlines. From functionality perspective, soft errors caused by transient or intermittent faults need to be detected and recovered with fault tolerance techniques. Moreover, during the lifetime of a vehicle design or even the same car, updates are often needed to add new features or fix bugs in existing ones. It is therefore critical to improve the design extensibility for accommodating such updates without incurring major redesign and re-verification cost. In this work, we discuss the metrics for measuring latency, fault tolerance and extensibility, and present a simulated annealing based algorithm to search the design space with respect to them. Experimental results on industrial and synthetic examples demonstrate clear trade-offs among these objectives, and hence the importance of quantitatively analyzing such trade-offs and exploring the design space with automation tools.

With the rapid advancement of autonomous driving and vehicular communication technology, intelligent intersection management has shown great promise in improving transportation efficiency. In a typical intelligent intersection, an intersection manager communicates with autonomous vehicles wirelessly and schedules their crossing of the intersection. Previous system designs, however, do not address the possible communication delays due to network congestion or security attacks, and could lead to unsafe or deadlocked systems. In this work, we propose a delay- tolerant protocol for intelligent intersection management, and develop a modeling, simulation and verification framework for analyzing the protocol's safety, liveness and performance. Experiments demonstrate the advantages of our proposed protocol over traditional traffic light control, and more importantly, demonstrate the importance and effectiveness of using this framework to address timing (delay) in vehicular network applications. This work is the first step towards a comprehensive delay-aware design and verification framework for practical vehicular network applications.

The hybrid architecture analysis and design language (AADL) has been proposed to model the interactions between embedded control systems and continuous physical environment. However, the worst-case performance analysis of hybrid AADL designs often leads to overly pessimistic estimations, and is not suitable for accurate reasoning about overall system performance, in particular when the system closely interacts with an uncertain external environment. To address this challenge, this paper proposes a statistical model checking-based framework that can perform quantitative evaluation of uncertainty-aware hybrid AADL designs against various performance queries. Our approach extends hybrid AADL to support the modeling of environment uncertainties. Furthermore, we propose a set of transformation rules that can automatically translate AADL designs together with designers' requirements into networks of priced timed automata and performance queries, respectively. Comprehensive experimental results on the movement authority scenario of Chinese train control system level 3 demonstrate the effectiveness of our approach.

A cyber-physical system (CPS) is an integration of computation with physical processes whose behavior is defined by both computational and physical parts of the system. In this paper, we present a view of the challenges and opportunities for design automation of CPS. We identify a combination of characteristics that define the challenges unique to the design automation of CPS. We then present selected promising advances in depth, focusing on four foundational directions: combining model-based and data-driven design methods; design for human-in-the-loop systems; component-based design with contracts, and design for security and privacy. These directions are illustrated with examples from two application domains: smart energy systems and next-generation automotive systems.

No abstract available.

The heating, ventilation and air conditioning (HVAC) system accounts for half of the energy consumption of a typical building. Additionally, the need for HVAC changes over hours and days as does the electric energy price. Level of comfort of the building occupants is, however, a primary concern, which tends to overwrite pricing. Dynamic HVAC control under a dynamic energy pricing model while meeting an acceptable level of occupants' comfort is thus critical to achieving energy efficiency in buildings in a sustainable manner. Finally, there is the possibility that the building is equipped with some renewable source of power such as solar panels mounted on the rooftop. The presence of a battery energy storage system in a target building would enable peak power shaving by adopting a suitable charge and discharge schedule for the battery, while simultaneously meeting building energy efficiency and user satisfaction. Achieving this goal requires detailed information (or predictions) about the amount of local power generation from the renewable source plus the power consumption load of the building. This paper addresses the coscheduling problem of HVAC control and battery management to achieve energy-efficient buildings, while also accounting for the degradation of the battery state-of-health during charging and discharging operations (which in turn determines the amortized cost of owning and utilizing a battery storage system)aa cč A time-of-use dynamic pricing scenario is assumed and various energy loss components are considered including power dissipation in the power conversion circuitry as well as the rate capacity effect in the battery. A global optimization framework targeting the entire billing cycle is presented and an adaptive co-scheduling algorithm is provided to dynamically update the optimal HVAC air flow control and the battery charging/discharging decision in each time slot during the billing cycle to mitigate the prediction error of unknown parameters. Experimental results show that the proposed algorithm achieves up to 15% in the total electric utility cost reduction compared with some baseline methods.

Petroleum Cyber-Physical System (CPS) marks the beginning of a new chapter of the oil and gas industry. Combining vast computational power with intelligent Computer Aided Design (CAD) algorithms, petroleum CPS is capable of precisely modeling the flow of fluids over the entire petroleum reservoir and leveraging the massive field data remotely collected at the production wells. It provides field operators with valuable insights into the geological structure and remaining reserves of the reservoir for optimizing their operational strategies. Despite such benefits, petroleum CPS is vulnerable to various cyberattacks that jeopardize the integrity of the field data collected at production wells. Given manipulated field data, CAD software would generate an inaccurate reservoir model which misleads the field operators.

Next-generation autonomous and semi-autonomous vehicles will not only precept the environment with their own sensors, but also communicate with other vehicles and surrounding infrastructures for vehicle safety and transportation efficiency. The design, analysis and validation of various vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) applications involve multiple layers, from V2V/V2I communication networks down to software and hardware of individual vehicles, and concern with stringent requirements on multiple metrics such as timing, security, reliability and fault tolerance. To cope with these challenges, we have been developing CONVINCE, a cross-layer modeling, exploration and validation framework for connected vehicles. The framework includes mathematical models, synthesis and validation algorithms, and a heterogeneous simulator for inter-vehicle communications and intra-vehicle software and hardware in a holistic environment. It explores various design options with respect to constraints and objectives on system safety, security, reliability, cost, etc. A V2V application is used in the case study to demonstrate the effectiveness of the proposed framework.

Computer vision algorithms are known to be extremely sensitive to the environmental conditions in which the data is captured, e.g., lighting conditions and target density. Tuning of parameters or choosing a completely new algorithm is often needed to achieve a certain performance level. In this paper, we focus on this problem and propose a framework to automatically choose the “best” algorithm-parameter combination (often referred to as the best algorithm for simplicity in this paper) for a certain input data. This necessitates developing a mechanism to switch among different algorithms and parameters as the nature of the input video changes. Specifically, our proposed algorithm calculates a similarity function between a test video segment and a training video segment. Similarity between training and test dataset indicates the same algorithm can be applied to both of them. We design a cost function with this similarity measure and a constraint on the number of switches. In the experiments, we apply our algorithm to the problem of pedestrian detection. We show how to adaptively select among 7 algorithm-parameter combinations and provide promising results on 3 publicly available datasets.

Buildings account for nearly 40% of energy consumption in the United States. To improve energy efficiency and reduce peak demand, intelligent building management systems can be developed to manage the energy consumption of flexible loads such as heating, ventilation, and air conditioning (HVAC) system an electric vehicle (EV) charging, as well as the usage of energy storage systems such as batteries. In the case where a building cluster is managed by the same institution, coordinating the energy consumption behavior across multiple buildings can provide further benefits in energy efficiency. In this paper, we first invest gate an integrated co-scheduling scheme that uses a joint formulation to optimize the control of HVAC systems, EV charging a d battery storage in multiple buildings for reducing the overall energy cost. Then, we further explore a more efficient heuristic scheme where the shared battery storage and EV charging demand are assigned to each building for separate building-level scheduling. Our experiments demonstrate the effectiveness of our co-scheduling scheme and separate-scheduling heuristic in reducing energy cost for building clusters.

For mixed-criticality (MC) systems, recent studies show that it can be important to provide continuous (albeit degraded) services for low-critical (LC) tasks even in the high running mode. In this paper, focusing on dual-criticality systems, we study a mode-switch fixed-priority (MS-FP) scheduler for a set of dual-rate mixed-criticality (DR-MC) tasks, where each LC task can have a pair of small and large periods to represent its service requirements in the low (LO) and high (HI) running modes, respectively. Moreover, DR-MC tasks may adjust their priorities at the mode-switch point for better system schedulability. By extending the response time analysis (RTA) technique for MC systems, we first derive the schedulability conditions for a set of DR-MC tasks under the MS-FP scheduler with mode transition being considered. Then, we investigate how to select periods and priorities of DR-MC tasks to optimize their control performance and formulate it as a Non-Linear Optimization problem. We propose an efficient heuristic for a simplified optimization problem based on Branch & Bound Search Tree (BBST) technique. The effectiveness of the proposed heuristic and the MS-FP scheduler with DR-MC task model is illustrated through one case study with four tasks and compared against the Ipopt solutions.

The sampling periods of real-time embedded control functions have a significant impact on control performance and system schedulability. Exploring period assignment for optimizing control performance while meeting schedulability constraints is very challenging, in particular for distributed systems where control loops share a network of computation and communication resources. In this work, we propose an efficient approach that approximates the performance of each control loop in the system with a piecewise linear function of its sampling period and end-to-end delay, and then optimizes the periods of tasks and messages by exploring the linear partitions of the approximated functions and solving a series of geometric programming (GP) formulations. Experiments on sample control models, an automotive industrial case study and a set of synthetic examples demonstrate the effectiveness and efficiency of our approach.

Buildings account for nearly 40 percent of the total energy consumption in the United States. As a critical step toward smart cities, it is essential to intelligently manage and coordinate the building operations to improve the efficiency and reliability of overall energy system. With the advent of smart meters and two-way communication systems, various energy consumptions from smart buildings can now be coordinated across the smart grid together with other energy loads and power plants. In this paper, we propose a comprehensive framework to integrate the operations of smart buildings into the energy scheduling of bulk power system through proactive building demand participation. This new scheme enables buildings to proactively express and communicate their energy consumption preferences to smart grid operators rather than passively receive and react to market signals and instructions such as time varying electricity prices. The proposed scheme is implemented in a simulation environment. The experiment results show that the proactive demand response scheme can achieve up to 10 percent system generation cost reduction and 20 percent building operation cost reduction compared with passive demand response scheme. The results also demonstrate that the system cost savings increase significantly with more flexible load installed and higher percentage of proactive customers participation level in the power network.

Security attacks may have disruptive consequences on cyber-physical systems, and lead to significant social and economic losses. Building secure cyber-physical systems is particularly challenging due to the variety of attack surfaces from the cyber and physical components, and often to limited computation and communication resources. In this paper, we propose a cross-layer design framework for resource-constrained cyber-physical systems. The framework combines control-theoretic methods at the functional layer and cybersecurity techniques at the embedded platform layer, and addresses security together with other design metrics such as control performance under resource and real-time constraints. We use the concept of interface variables to capture the interactions between control and platform layers, and quantitatively model the relation among system security, performance, and schedulability via interface variables. The general codesign framework is customized and refined to the automotive domain, and its effectiveness is demonstrated through an industrial case study and a set of synthetic examples.

Plug-in electric vehicles (PEVs) are considered the key to reducing the fossil fuel consumption and an important part of the smart grid. The plug-in electric vehicle-to-grid (V2G) technology in the smart grid infrastructure enables energy flow from PEV batteries to the power grid so that the grid stability is enhanced and the peak power demand is shaped. PEV owners will also benefit from V2G technology as they will be able to reduce energy cost through proper PEV charging and discharging scheduling. Moreover, power regulation service (RS) reserves have been playing an increasingly important role in modern power markets. It has been shown that by providing RS reserves, the power grid achieves a better match between energy supply and demand in presence of volatile and intermittent renewable energy generation. This paper addresses the problem of PEV charging under dynamic energy pricing, properly taking into account the degradation of battery state-of-health (SoH) during V2G operations as well as RS provisioning. An overall optimization throughout the whole parking period is proposed for the PEV and an adaptive control framework is presented to dynamically update the optimal charging/discharging decision at each time slot to mitigate the effect of RS tracking error. Experimental results show that the proposed optimal PEV charging algorithm minimizes the combination of electricity cost and battery aging cost in the RS provisioning power market.

Cyber-security has emerged as a pressing issue for transportation systems. Studies have shown that attackers can attack modern vehicles from a variety of interfaces and gain access to the most safety-critical components. Such threats become even broader and more challenging with the emergence of vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication technologies. Addressing the security issues in transportation systems requires comprehensive approaches that encompass considerations of security mechanisms, safety properties, resource constraints, and other related system metrics. In this work, we propose an integrated framework that combines hybrid modeling, formal verification, and automated synthesis techniques for analyzing the security and safety of transportation systems and carrying out design space exploration of both in-vehicle electronic control systems and vehicle-to-vehicle communications. We demonstrate the ideas of our framework through a case study of cooperative adaptive cruise control.

Demand response (DR) is an effective mechanism in improving power system efficiency and reducing energy cost for customers. However, DR processes might be vulnerable to cyber attacks from the usage of advanced metering infrastructure and wide-area network to exchange information. In this paper, we study potential attacks for a proactive demand participation scheme we recently proposed and for a conventional passive demand response scheme, particularly focusing on guideline price manipulation attacks. Our experiment results demonstrate that 1) guideline price manipulations may significantly lower the attacker's own electricity consumption cost while increasing other customers' cost, for both proactive and passive schemes; 2) such impact is less severe in the proactive scheme, i.e., the proactive demand participation scheme is more robust with respect to guideline price manipulation than the conventional DR.

Nested loops with regular iteration dependencies span a large class of applications ranging from string matching to linear system solvers. Wavefront parallelism is a well-known technique to enable concurrent processing of such applications and is widely being used on GPUs to benefit from their massively parallel computing capabilities. Wavefront parallelism on GPUs uses global barriers between processing of tiles to enforce data dependencies. However, such diagonal-wide synchronization causes load imbalance by forcing SMs to wait for the completion of the SM with longest computation. Moreover, diagonal processing causes loss of locality due to elements that border adjacent tiles.

Multicores are today used in automotive, controls and avionics systems supporting real-time functionality. When real-time tasks allocated on different cores cooperate through the use of shared communication resources, they need to be protected by mechanisms that guarantee access in a mutual exclusive way with bounded worst-case blocking time. Lock-based mechanisms such as MPCP and MSRP have been developed to fulfill this demand, and research papers are today tackling the problem of finding the optimal task placement in multicores while trying to meet the deadlines against blocking times. In this paper, we propose a resource-aware task allocation algorithm for systems that use MSRP to protect shared resources. Furthermore, we leverage the additional opportunity provided by wait-free methods as an alternative data consistency mechanism for the case that the shared resource is communication or state memory. An algorithm that performs both task allocation and data consistency mechanism (MSRP or wait-free) selection is proposed. The selective use of wait-free methods can significantly extend the range of schedulable systems at the cost of memory.

In this article, we address both security and safety requirements and solve security-aware design problems for the controller area network (CAN) protocol and time division multiple access (TDMA)-based protocols. To provide insights and guidelines for other similar security problems with limited resources and strict timing constraints, we propose a general security-aware design methodology to address security with other design constraints in a holistic framework and optimize design objectives. The security-aware design methodology is further applied to solve a security-aware design problem for vehicle-to-vehicle (V2V) communications with dedicated short-range communication (DSRC) technology. Experimental results demonstrate the effectiveness of our approaches in system design without violating design constraints and indicate that it is necessary to consider security together with other metrics during design stages.

As the building stock consumes 40% of the U.S. primary energy consumption, it is critically important to improve building energy efficiency. This involves reducing the total energy consumption of buildings, reducing the peak energy demand, and leveraging renewable energy sources, etc. To achieve such goals, hybrid energy supply has becoming popular, where multiple energy sources such as grid electricity, on-site fuel cell generators, solar, wind, and battery storage are scheduled together to improve energy efficiency.

The lack of accurate yet open to public simulation infrastructure has puzzled researchers in the memcomputing area for sometime. In this paper, we propose for the first time a full tool chain called MSim that supports the cycle-accurate microarchitecture level simulation for memcomputing studies. With MSim, the performance gains of utilizing memcomputing for arbitrary applications on user configurable computer system architectures can be evaluated in high accuracy. In addition, MSim provides flexible interfaces with pervasive object-oriented design, which makes it well-suited as a good base platform for researchers to explore new memcomputing technologies.

In this article, we propose a new lifetime task optimization technique for real-time embedded processors considering the electromigration-induced reliability. The new approach is based on a recently proposed physics-based electromigration (EM) model for more accurate EM assessment of a power grid network at the chip level. We apply the dynamic voltage and frequency scaling (DVFS) (by selecting the performance states or p-states of the tasks to manage the power) and thus the lifetime of the processor running different tasks over their periods. We consider both single-rate and multi-rate embedded systems with preemption. To model the mean-time-to-failure (MTTF) of a task for a given p-state, response surface modeling is applied. We then frame the reliability optimization problem as the continuous constrained nonlinear optimization problem in which the system EM-induced reliability is maximized subject to the timing constraints, which is further solved by simulated annealing method. Experimental results show that for low utilization systems, significant reliability improvement can be achieved with even smaller power consumption than existing reliability-ignore scheduling method. The proposed method can lead to near Pareto's front trade-off between the power/energy and the lifetime compared to the existing task scheduling method.

Cyber-security has become a critical issue for realtime distributed embedded systems in domains such as automotive, avionics, and industrial automation. However, in many of such systems, tight resource constraints and strict timing requirements make it difficult or even impossible to add security mechanisms after the initial design stages. To produce secure and safe systems with desired performance, security must be considered together with other objectives at the system level and from the beginning of the design. In this paper, we focus on security-aware design for Time Division Multiple Access (TDMA) based real-time distributed systems. The TDMA-based protocol we consider is an abstraction of many time-triggered protocols that are being adopted in various safety-critical systems for their more predictable timing behavior, such as FlexRay, Time-Triggered Protocol, and Time-Triggered Ethernet. To protect against attacks on TDMA-based real-time distributed systems, we apply a message authentication mechanism with time-delayed release of keys, which provides a good balance between security and computational overhead but needs sophisticated network scheduling to ensure that the increased latencies due to delayed key releases will not violate timing requirements. We propose formulations and an algorithm to optimize the task allocation, priority assignment, network scheduling, and key-release interval length during the mapping process, while meeting both security and timing requirements. Experimental results of an automotive case study and a synthetic example show the effectiveness and efficiency of our approach.

Building stock consumes 40% of primary energy consumption in the United States. Among various types of energy loads in buildings, HVAC (heating, ventilation, and air conditioning) and EV (electric vehicle) charging are two of the most important ones and have distinct characteristics. HVAC system accounts for 50% of the building energy consumption and typically operates throughout the day, while EV charging is an emerging major energy load that is hard to predict and may cause spikes in energy demand. To maximize building energy efficiency and grid stability, it is important to address both types of energy loads in a holistic framework. Furthermore, on the supply side, the utilization of multiple energy sources such as grid electricity, solar, wind, and battery storage provides more opportunities for energy efficiency, and should be considered together with the scheduling of energy loads. In this paper, we present a novel model predictive control (MPC) based algorithm to co-schedule HVAC control, EV scheduling and battery usage for reducing the total building energy consumption and the peak energy demand, while maintaining the temperature within the comfort zone for building occupants and meeting the deadlines for EV charging. Experiment results demonstrate the effectiveness of our approach under a variety of demand, supply and environment constraints.

Embedded software and electronics are major contributors of values in vehicles, and play a dominant role in vehicle innovations. The design of automotive embedded systems has become more and more challenging, with the rapid increase of system complexity and more requirements on various design objectives. Methodologies such as model-based design are being adopted to improve design quality and productivity through the usage of functional models. However, there is still a significant lack of design automation tools, in particular synthesis and optimization tools, that can turn complex functional specifications to correct and optimal software implementations on distributed embedded platforms. In this paper, we discuss some of the major technical challenges and the problems to be solved in automotive embedded systems design, especially for the synthesis and optimization of embedded software.

In the development of real-time embedded applications, especially those on systems-on-chip, an efficient use of RAM memory is as important as the effective scheduling of the computation resources. The protection of communication and state variables accessed by concurrent tasks must provide real-time schedulability guarantees while using the least amount of memory. Several schemes, including preemption thresholds, have been developed to improve schedulability and save stack space by selectively disabling preemption. However, the design synthesis problem is still open. In this article, we target the assignment of the scheduling parameters to minimize memory usage for systems of practical interest, including designs compliant with automotive standards. We propose algorithms either proven optimal or shown to improve on randomized optimization methods like simulated annealing.

Model-based design using synchronous reactive (SR) models is widespread for the development of embedded control software. SR models ease verification and validation, and enable the automatic generation of implementations. In SR models, synchronous finite state machines (FSMs) are commonly used to capture changes of the system state under trigger events. The implementation of a synchronous FSM may be improved by using multiple software tasks instead of the traditional single-task solution. In this work, we propose methods to quantitatively analyze task implementations with respect to a breakdown factor that measures the timing robustness, and an action extensibility metric that measures the capability to accommodate upgrades. We propose an algorithm to generate a correct and efficient task implementation of synchronous FSMs for these two metrics, while guaranteeing the schedulability constraints.

Cyber-security is a rising issue for automotive electronic systems, and it is critical to system safety and dependability. Current in-vehicles architectures, such as those based on the Controller Area Network (CAN), do not provide direct support for secure communications. When retrofitting these architectures with security mechanisms, a major challenge is to ensure that system safety will not be hindered, given the limited computation and communication resources. We apply Message Authentication Codes (MACs) to protect against masquerade and replay attacks on CAN networks, and propose an optimal Mixed Integer Linear Programming (MILP) formulation for solving the mapping problem from a functional model to the CAN-based platform while meeting both the security and the safety requirements. We also develop an efficient heuristic for the mapping problem under security and safety constraints. To the best of our knowledge, this is the first work to address security and safety in an integrated formulation in the design automation of automotive electronic systems. Experimental results of an industrial case study show the effectiveness of our approach.

Cyber-Physical Systems are integrations of computation and physical processes and as such, will be increasingly relevant to industry and people. The complexity of designing CPS resides in their heterogeneity. Heterogeneity manifest itself in modeling their functionality as well as in the implementation platforms that include a multiplicity of components such as microprocessors, signal processors, peripherals, memories, sensors and actuators often integrated on a single chip or on a small package such as a multi-chip module. We need a methodology, tools and environments where heterogeneity can be dealt with at all levels of abstraction and where different tools can be integrated. We present here Platform-Based Design as the CPS methodology of choice and metro II, a design environment that supports it. We present the metamodeling approach followed in metro II, how to couple the functionality and implementation platforms of CPS, and the simulation technology that supports the analysis of CPS and of their implementation. We also present examples of use and the integration of metro II with another popular design environment developed at Verimag, BIP.

In the development of some real-time embedded applications, especially systems-on-chip, an efficient use of RAM memory is as important as the effective scheduling of the computation resources. The design problem is to find a schedulable solution that fits within the memory budget. In a real-time concurrent system, preemption plays an important role in the exploration of these tradeoffs. Several schemes, including preemption thresholds and non-preemption groups, have been developed to improve schedulability and saving stack memory space by selectively disabling preemption. However, the design synthesis problem for such systems and protocols is still an open problem. We target at the efficient assignment of the scheduling parameters for systems scheduled according to these policies in several cases of practical interest, including those that are compliant with automotive standards.

The complexity and physical distribution of modern active safety, chassis, and powertrain automotive applications requires the use of distributed architectures. Complex functions designed as networks of function blocks exchanging signal information are deployed onto the physical HW and implemented in a SW architecture consisting of a set of tasks and messages. The typical configuration features priority-based scheduling of tasks and messages and imposes end-to-end deadlines. In this work, we present and compare formulations and procedures for the optimization of the task allocation, the signal to message mapping, and the assignment of priorities to tasks and messages in order to meet end-to-end deadline constraints and minimize latencies. Our formulations leverage worst-case response time analysis within a mixed integer linear optimization framework and are compared for performance against a simulated annealing implementation. The methods are applied for evaluation to an automotive case study of complexity comparable to industrial design problems.

We propose a system-level design flow for building automation and control (BAC) systems. The input to the design flow is a high level description of the control algorithms given in a model-based environment such as Simulink. The input specification is translated into an intermediate format, and then automatically refined into a distributed implementation. Refinement includes optimal mapping of the functional specification on a set of computation and communication resources, and software synthesis, which generates code for each component in the mapped design while guaranteeing semantic equivalence with the original specification. Experiments with a temperature control system are presented to illustrate the flow.

We consider a set of control tasks that must be executed on distributed platforms so that end-to-end latencies are within deadlines. We investigate how to allocate tasks to nodes, pack signals to messages, allocate messages to buses, and assign priorities to tasks and messages, so that the design is extensible and robust with respect to changes in task requirements. We adopt a notion of extensibility metric that measures how much the execution times of tasks can be increased without violating end-to-end deadlines. We optimize the task and message design with respect to this metric by adopting a mathematical programming front-end followed by postprocessing heuristics. The proposed algorithm as applied to industrial strength test cases shows its effectiveness in optimizing extensibility and a marked improvement in running time with respect to an approach based on randomized optimization.

We consider a set of control tasks that must be executed on distributed platforms so that end-to-end latencies are within deadlines. We investigate how to allocate tasks to nodes, pack signals to messages, allocate messages to buses, and assign priorities to tasks and messages, so that the design is robust with respect to changes in task requirements. The notion of extensibility is used to measure robustness. The extensibility metric measures how much the execution times of tasks can be increased without violating end-to-end deadlines. We optimize this metric by adopting a mathematical programming front-end followed by post-processing heuristics. The proposed algorithm as applied to industrial strength test cases shows its effectiveness in optimizing extensibility and a marked improvement in running time with respect to an approach based on randomized optimization.

The complexity and physical distribution of modern active-safety automotive applications requires the use of distributed architectures. These architectures consist of multiple electronic control units (ECUs) connected with standardized buses. The most common configuration features periodic activation of tasks and messages coupled with run-time priority-based scheduling. The correct deployment of applications on such architectures requires end-to-end latency deadlines to be met. This is challenging since deadlines must be enforced across a set of ECUs and buses, each of which supports multiple functionality. The need for accommodating legacy tasks and messages further complicates the scenario.

The complexity and physical distribution of modern active safety, chassis and powertrain automotive applications requires the use of distributed architectures. Complex functions designed as networks of function blocks exchanging signal information are deployed onto the physical HW and implemented in a SW architecture consisting of a set of tasks and messages. The typical configuration features priority-based scheduling of tasks and messages and imposes end- to-end deadlines. In this work, we optimize the task placement and the signal to message mapping and we automate the assignment of priorities to tasks and messages in order to meet end-to-end deadline constraints and minimize latencies. This is accomplished by leveraging worst case response time analysis within a mixed integer linear optimization framework. Our approach is applied to an automotive case study to prove its feasibility.

SAT sweeping is a method for simplifying an shape And/Inverter graph (AIG) by systematically merging graph vertices from the inputs towards the outputs using a combination of structural hashing, simulation, and SAT queries. Due to its robustness and efficiency, SAT sweeping provides a solid algorithm for Booleanreasoning in functional verification and logic synthesis. In previous work, SAT sweeping merges two vertices only if they are functionally equivalent. In this paper we present a significant extension of the SAT-sweeping algorithm that exploits local observability don't-cares (ODCs) to increase the number of vertices merged. We use a novel technique to bound the use of ODCs and thus the computational effort to find them, while still finding a large fraction of them. Our reported results based on a set of industrial benchmark circuits demonstrate that ODC-based SAT sweeping results in significantly more graph simplification with great benefit for Boolean reasoning with a moderate increase in computational effort.

In this work, we propose a semantics-driven synthesis flow, in which the semantics and the abstraction level are determined formally by using the concept of a common modeling domain between functionality and architecture. By doing so, a formal synthesis procedure can be defined and algorithms for automatic optimal mapping derived.

CAD tools have become more and more important for integrated circuit (IC) design since a complicated system can be designed into a single chip, called system-on-a-chip (SOC), in which physical design tool is an essential and critical part. We try to consider the via minimization problem as early as possible in physical design. We propose a routing method focusing on minimizing vias while considering mutability and wire-length constraint. That is, in the global routing phase, we minimize the number of bends, which is closely related to the number of vias. Previous work only dealt with very small nets, but our algorithm is general for the nets with any size. Experimental results show that our algorithm can greatly reduce the count of bends for various sizes of nets while meeting the constraints of congestion and wire-length.

With advances in fabrication technology of very/ultra large scale integrated circuit (VLSI/ULSI), we must face many new challenges. One of them is the interconnect effects, which may cause longer delay and heavier crosstalk. To solve this problem, many interconnect performance optimization algorithms have been proposed. However, when these algorithms are designed based on rectilinear interconnect architecture, the optimization capability is limited. Therefore, nonrectilinear interconnect architectures become a field of active research in which the octilinear interconnect architecture is the most promising one since it extends from the rectilinear case and greatly shortens the wire length. Meanwhile, an interconnect with less length is helpful to reduce wire capacitance, congestion, and routing area. In an interconnect architecture, the Steiner minimal tree (SMT) construction is one of the key problems. In this paper, we give two practical octilinear Steiner minimal tree (OSMT) construction algorithms based on octilinear spanning graphs (OSGs). The one with edge substitution (OST-E) has a worst-case running time of O(nlogn) and a similar performance as the recent work using batched greedy. The other one with triangle contraction (OST-T) has a small increase in the constant factor of running time and a better performance. These two are the fastest algorithms for octilinear Steiner tree construction so far. Experiments on both industrial and random test cases are conducted to compare with other programs. We also proposed the extension of our algorithms to any /spl lambda/ geometry.