Uncertainty Quantification (UQ) is crucial for ensuring the reliability of machine learning models deployed in real-world autonomous systems. However, existing approaches typically quantify task-level output prediction uncertainty without considering epistemic uncertainty at the multimodal feature fusion level, leading to sub-optimal outcomes. Additionally, popular uncertainty quantification methods, e.g., Bayesian approximations, remain challenging to deploy in practice due to high computational costs in training and inference. In this paper, we propose HyperDUM, a novel deterministic uncertainty method (DUM) that efficiently quantifies feature-level epistemic uncertainty by leveraging hyper-dimensional computing. Our method captures the channel and spatial uncertainties through channel and patch -wise projection and bundling techniques respectively. Multimodal sensor features are then adaptively weighted to mitigate uncertainty propagation and improve feature fusion. Our evaluations show that HyperDUM on average outperforms the state-of-the-art (SOTA) algorithms by up to 2.01%/1.27% in 3D Object Detection and up to 1.29% improvement over baselines in semantic segmentation tasks under various types of uncertainties. Notably, HyperDUM requires 2.36× less Floating Point Operations and up to 38.30× less parameters than SOTA methods, providing an efficient solution for real-world autonomous systems.

Autonomous vehicles (AVs) can operate in complex environments that require multiple types of sensors in their perception approaches. However, a single sensing configuration is not tenable in all environments, and adapting the perception approach to different domains is a challenging task. Both sensor fusion and ensemble learning methods utilize the diversity of multimodal sensor data (e.g., cameras, radar, lidar) to increase perception performance under challenging sensing conditions. In this paper, we conduct the first analysis examining how sensor diversity can impact performance across different AV perception tasks. We propose ensembles of multimodal models that leverage diversity and assess their ability to address the challenge of adaptation in AV perception. We introduce a novel, model-agnostic framework,DivFusE, that identifies the level of diversity that individual sensor modalities contribute to perception ensembles and employs adaptation based on this diversity. We benchmark our approach on five public datasets containing semantic segmentation and object detection tasks, highlighting scenarios where our proposed framework can improve perception performance by 70% (fuse it) and scenarios where fusion can reduce perception performance (lose it). Our findings indicate that the performance improvements of using multimodal ensembles is greater on objection detection compared to semantic segmentation, and that sensing conditions with higher (lower) levels of diversity further increase (decrease) performance. Ultimately, the analysis in this work highlights scenarios where these multimodal ensembles should and should not be deployed and can be used to develop more adaptive multimodal sensor fusion systems.

As machine learning (ML)-driven decisions proliferate, particularly in cases involving sensitive attributes, such as gender, race, and age, to name a few, the need for equity and impartiality has emerged as a fundamental concern. In situations demanding real-time decision-making, fairness objectives become more nuanced and complex: instantaneous fairness to ensure equity in every time slot and long-term fairness to ensure fairness over a period of time. There is a growing awareness that real-world systems operating over long periods require fairness over different timelines. Most existing approaches mainly address dynamic costs with time-invariant fairness constraints, often disregarding the challenges posed by time-varying fairness constraints. Time-varying fairness constraints require the learners to adapt their decisions to meet the changing constraints. However, long-term dynamics are hard to assess and accurately predicting the changes in constraints can be difficult. To bridge this gap, this work introduces a framework for ensuring long-term fairness within dynamic decision-making systems characterized by time-varying fairness constraints. We formulate the decision problem with fairness constraints over a period as a constrained online optimization problem. A novel online algorithm, named long-term fairness-aware online learning algorithm (LoTFair), is presented that solves the problem "on the fly." We demonstrate that long-term fairness for real-time decision making can be addressed flexibly and efficiently by LoTFair: it achieves overall fairness while maintaining performance over the long run.

Mixed-precision Deep Neural Networks (DNNs) provide an efficient solution for hardware deployment, especially under resource constraints, while maintaining model accuracy. Identifying the ideal bit precision for each layer, however, remains a challenge given the vast array of models, datasets, and quantization schemes, leading to an expansive search space. Recent literature has addressed this challenge, resulting in several promising frameworks. This paper offers a comprehensive overview of the standard quantization classifications prevalent in existing studies. A detailed survey of current mixed-precision frameworks is provided, with an in-depth comparative analysis highlighting their respective merits and limitations. The paper concludes with insights into potential avenues for future research in this domain.

In autonomous vehicles (AVs), early warning systems rely on collision prediction to ensure occupant safety. However, state-of-the-art methods using deep convolutional networks either fail at modeling collisions or are too expensive/slow, making them less suitable for deployment on AV edge hardware. To address these limitations, we propose SG2VEC, a spatiotemporalscene-graphembedding methodology that uses the graph neural network (GNN) and long short-term memory (LSTM) layers to predict future collisions via visual scene perception. We demonstrate that SG2VEC predicts collisions 8.11% more accurately and 39.07% earlier than the state-of-the-art method on synthesized data sets, and 29.47% more accurately on a challenging real-world collision data set. We also show that SG2VEC is better than the state of the art at transferring knowledge from synthetic data sets to real-world driving data sets. Finally, we demonstrate that SG2VEC performs inference$9.3\times $faster with an 88.0% smaller model, 32.4% less power, and 92.8% less energy than the state-of-the-art method on the industry-standard Nvidia DRIVE PX 2 platform, making it more suitable for implementation on the edge.

A neuro-cognitive inspired architecture based on the Hierarchical Temporal Memory (HTM) is proposed for anomaly detection and simultaneous data prediction in real-time for smart grid$\mu$PMU data. The key technical idea is that the HTM learns asparse distributed temporal representationof sequential data that turns out to be very useful for anomaly detection and simultaneous data prediction in real-time. Our results show that the proposed HTM can predict anomalies within 83–90 percent accuracy for three different application profiles, namelyStandard, Reward Few False Positive, Reward Few False Negativefor two different datasets. We show that the HTM is competitive to five state-of-the-art algorithms for anomaly detection. Moreover, for the multi-step prediction in the online setting, the same HTM achieves a low 0.0001 normalized mean square error, a low negative log-likelihood score of 1.5 and is also competitive to six state-of-the-art prediction algorithms. We demonstrate that the same HTM model can be used forboth the tasksand can learn online in one-pass, in an unsupervised fashion and adapt to changing statistics.The other state-of-the-art algorithms are either less accurate or are limited to one of the tasks or cannot learn online in one-pass, and adapt to changing statistics.

There is considerable evidence that evaluating the subjective risk level of driving decisions can improve the safety of Autonomous Driving Systems (ADS) in both typical and complex driving scenarios. In this paper, we propose a novel data-driven approach that uses scene-graphs as intermediate representations for modeling the subjective risk of driving maneuvers. Our approach includes a Multi-Relation Graph Convolution Network, a Long-Short Term Memory Network, and attention layers. To train our model, we formulate subjective risk assessment as a supervised scene classification problem. We evaluate our model on both synthetic lane-changing datasets and real-driving datasets with various driving maneuvers. We show that our approach achieves a higher classification accuracy than the state-of-the-art approach on both large (96.4% vs. 91.2%) and small (91.8% vs. 71.2%) lane-changing synthesized datasets, illustrating that our approach can learn effectively even from small datasets. We also show that our model trained on a lane-changing synthesized dataset achieves an average accuracy of 87.8% when tested on a real-driving lane-changing dataset. In comparison, the state-of-the-art model trained on the same synthesized dataset only achieved 70.3% accuracy when tested on the real-driving dataset, showing that our approach can transfer knowledge more effectively. Moreover, we demonstrate that the addition of spatial and temporal attention layers improves our model’s performance and explainability. Finally, our results illustrate that our model can assess the risk of various driving maneuvers more accurately than the state-of-the-art model (86.5% vs. 58.4%, respectively).

A digital twin is the virtual replica of a physical system. Digital twins are useful because they provide models and data for design, production, operation, diagnostics, and autonomy of machines and products. Hence, the digital twin has been projected as the key enabler of the Visions of Industry 4.0. The digital twin concept has become increasingly sophisticated and capable over time, enabled by many technologies. In this paper, we propose the cognitive digital twin as the next stage of advancement of a digital twin that will help realize the vision of Industry 4.0. Cognition, which is inspired by advancements in cognitive science, machine learning, and artificial intelligence, will enable a digital twin to achieve some critical elements of cognition, e.g., attention (selective focusing), perception (forming useful representations of data), memory (encoding and retrieval of information and knowledge), etc. Our main thesis is that cognitive digital twins will allow enterprises to creatively, effectively, and efficiently exploit implicit knowledge drawn from the experience of existing manufacturing systems and enable the transfer of higher performance decisions and control and improve the performance across the enterprise (at scale). Finally, we present open questions and challenges to realize these capabilities in a digital twin.

WiFi connectivity events, generated when a mobile device connects to WiFi access points can serve as a robust, passive, (almost) zero-cost indoor localization technology. The challenge is the coarse level localization it offers that limits its usefulness. We recently developed a novel data cleaning based approach, LOCATER , that exploits patterns in the network data to achieve accuracy as high as 90% at room level granularity making it possible to use network data to support a much larger class of applications. In this paper, we demonstrate one such application to help organizations track levels of occupancy, and potential exposure of the inhabitants of the buildings to others possibly infected on their premises. The system, entitled T-Cove, is in operational use at over 20 buildings at UCI and has now become part of the reopening procedure of the schools. The demonstration will highlight T-Cove functionalities over both live data and data captured in the past.

If machine failures can be detected preemptively, then maintenance and repairs can be performed more efficiently, reducing production costs. Many machine learning techniques for performing early failure detection using vibration data have been proposed; however, these methods are often power and data-hungry, susceptible to noise, and require large amounts of data preprocessing. Also, training is usually only performed once before inference, so they do not learn and adapt as the machine ages. In this article, we propose a method of performing online, real-time anomaly detection for predictive maintenance using hierarchical temporal memory (HTM). Inspired by the human neocortex, HTMs learn and adapt continuously and are robust to noise. Using the Numenta Anomaly Benchmark, we empirically demonstrate that our approach outperforms state-of-the-art algorithms at preemptively detecting real-world cases of bearing failures and simulated 3-D printer failures. Our approach achieves an average score of 64.71, surpassing state-of-the-art deep-learning (49.38) and statistical (61.06) methods.

We present our experiences in adapting and deploying TIPPERS1, a novel privacy-enabled IoT data collection and management system for smart spaces, to facilitate the monitoring of adherence to COVID-19 regulations in a university campus and a military facility.

We consider the scheduling of flexible electric power loads so as to minimize the peak load. We settle a conjecture of Alamdari et al.[1] regarding the performance of the bottom left decreasing height heuristic to schedule preemptable loads with known power requirement and duration and having the same earliest start time and the same deadline. Specifically, we show a tight bound of 4/3 - 1/(3D) relative to the minimum peak power load, where D is the duration of the scheduling interval. On benchmark strip packing data, the bottom left decreasing height heuristic generated schedules that required up to 45% less peak power than required by schedules generated using the next fit decreasing height heuristic. On electric and plug-in hybrid electric vehicles data, these two heuristics are very competitive.

This paper is focused on the development and analysis of suboptimal decision algorithms for a collection of robots that assist a remotely located operator in perimeter surveillance. The operator is tasked with the classification of incursions across the perimeter. whenever there is an incursion into the perimeter, an unattended ground sensor (UGS) in the vicinity, signals an alert. A robot services the alert by visiting the alert location, collecting information, e.g., photo and video imagery, and transmitting it to the operator. The accuracy of operator's classification depends on the volume and freshness of information gathered and provided by the robots at locations where incursions occur. There are two competing objectives for a robot: it needs to spend adequate time at an alert location to collect evidence for aiding the operator in accurate classification but it also needs to service other alerts as soon as possible, so that the evidence collected is relevant. The decision problem is to determine the optimal amount of time a robot must spend servicing an alert. The incursions are stochastic and their statistics are assumed to be known. This problem can be posed as a Markov Decision Problem. However, even for two robots and five UGS locations, the number of states is of the order of billions rendering exact dynamic programming methods intractable. Approximate dynamic programming (ADP) via linear programming (LP) provides a way to approximate the value function and derive suboptimal strategies. The novel feature of this paper is the derivation of a tractable lower bound via LP and the construction of a suboptimal policy whose performance improves upon the lower bound. An illustrative perimeter surveillance example corroborates the results derived in this paper. Note to Practitioners-In practice, one often encounters the curse of dimensionality in the application of dynamic programming to determine optimal policies for controlled Markov chains. This is true, in particular, for dynamic scheduling problems involving multiple robots/servers and queues of tasks that arrive in a stochastic fashion. The computation of value function, critical to the determination of optimal policies, is nearly impractical. Hence, one must settle for suboptimal policies. Two natural questions arise: (1) How does one construct a suboptimal policy? (2) How “good” is the constructed suboptimal policy? A common strategy to tackle the first problem is to approximate the value function and construct a suboptimal policy that is greedy with respect to the approximate value function. Typically, an approximate value function is constructed via a choice of basis functions. The question of how to choose the basis functions systematically for any problem is a difficult one; usually, the structure of the problem at hand is exploited in the construction of basis functions. The same approach is taken here and the state space is partitioned based on the reward structure and the optimal cost-to-go or value function is approximated by a constant over each partition. The second question is related to the first question in the sense that one needs to construct bounds for the performance of a suboptimal policy. In this paper, we construct upper and lower bounds for the value function (optimal performance) and use the lower bound as an approximate value function. Furthermore, we also show that the resulting suboptimal policy comes with a performance guarantee, in that it improves on the lower bound, it was derived from. Literature is replete with techniques for computing upper bounds; however, there is little work on lower bounds, which are also required for bounding the suboptimality of the policy. One encounters prohibitively large number of constraints in the case of computing an upper bound and has to deal with disjunctive linear inequalities in the case of a lower bound. The problem structure is exploited here to circumvent these difficulties. The upper and lower bounds to the value function developed in this paper could also be used to refine the partitions by identifying the partition with the largest difference between the upper and lower bounds; such a partition could be refined further using the structure of the problem. For practitioners, this could be a useful set of tools for generating suboptimal policies for any controlled Markov chain with a reward function that is amenable to state aggregation.

In this paper we consider the problem of scheduling energy consumption loads in the setting of smart electric grids. Each load is characterized as a "job" by a start (arrival) time and a deadline by which a certain amount of electric energy must be delivered to the load. A job may be preemptable, i.e. it can be interrupted or non-preemptable. Specifically, we focus on scheduling a mixture of preemptable and non-preemptable jobs with the same arrival time and deadline with the goal of minimizing the peak power. We study and modify the first fit decreasing height algorithm of the strip packing problem for this purpose. We derive a performance bound for the algorithm and prove its tightness. We test the performance of the algorithm extensively on a variety of datasets including real life household data.

We consider the scheduling of flexible electric loads in a smart grid so as to minimize peak power demand. Specifically, we focus on the case when the loads are preemptable, their power requirement and duration are known in advance, and they have the same earliest start time and the same deadline. Our main results are (a) when power requests are scheduled preemptively, the peak power demand can be reduced by up to 50% relative to when these requests are scheduled non-preemptively, (b) preemptive scheduling to minimize peak power demand is NP-hard, (c) schedules with minimum peak power demand may be constructed using integer linear programming, and (d) the next-fit decreasing height heuristic may be used to quickly obtain schedules whose peak power demand is at most two times that of the optimal schedule when all jobs are preemptable and at most three times the optimal when only some jobs are preemptable. Experimental results for the integer linear program and the heuristic are also presented. Our experiments indicate a significant reduction in peak power when preemption is exploited. For example, on our data sets recharging collections of electric and plug-in hybrid vehicles without preemption required up to 26% more peak power than when this was done preemptively.

Reducing the carbon footprint of data centers is becoming a primary goal of large IT companies. Unlike traditional energy sources, renewable energy sources are usually intermittent and unpredictable. How to better utilize the green energy from these renewable sources in data centers is a challenging problem. In this paper, we exploit the opportunities offered by geographical load balancing, opportunistic scheduling of delay-tolerant workloads, and thermal storage management in data centers to facilitate green energy integration and reduce the cost of brown energy usage. Moreover, bandwidth cost variations between users and data centers are considered. Specifically, this problem is first formulated as a stochastic program, and then, an online control algorithm based on the Lyapunov optimization technique, called Stochastic Cost Minimization Algorithm (SCMA), is proposed to solve it. The algorithm can enable an explicit trade-off between cost saving and workload delay. Numerical results based on real-world traces illustrate the effectiveness of SCMA in practice.

Reducing the carbon footprint of data centers is becoming a primary goal of large IT companies. Due to the intermittency and unpredictability of renewable energy sources such as wind and solar, it is quite challenging to utilize them in data centers. In this paper, we explore the opportunities offered by delay-tolerant workloads and thermal storage to facilitate the renewable energy integration in data centers and meanwhile, reduce the cost of using brown energy (i.e., energy from the utility grid). A stochastic optimization problem is formulated to tackle the stochastic renewable generation and workload arrival processes. Then, an online control algorithm based on the Lyapunov optimization approach is proposed to solve it. Simulation results based on the real-world traces show the effectiveness of the algorithm in practice.

In his seminal work, Jon Kleinberg considers a small-world network model consisting of a k-dimensional lattice augmented with shortcuts. Under the assumption that the probability of a shortcut being present between two nodes u and v decays as a power, d(u,v) -\alpha, of the distance d(u,v) between them, Kleinberg shows that decentralized routing scheme such as greedy geographic routing is efficient if alpha=k and that there is no efficient decentralized routing algorithm if alpha\neq k. The results are extended to a continuum model recently, wherein the nodes are distributed as a homogeneous Poisson point process by Franceschetti and Meester, Draief and Ganesh. In our work, we extend the result further to a more realistic model constructed from a nonhomogeneous Poisson point process, wherein each node is connected to all its neighbors within some fixed radius, as well as possessing random shortcuts to more distant nodes. More importantly, we show that in nonhomogeneous cases, the necessary and sufficient condition for greedy geographic routing to be efficient is that the probability of a shortcut being present from node u to v should be inversely proportional to the number of nodes which are closer to u than v is. We also demonstrate some applications of our results to wireless networks.

We present a hierarchical structure and framework for the modeling, specification, analysis and design of logic controllers for a reconfigurable machining system (RMS). This hierarchical framework allows one to integrate controllers at various levels of coordination in the machining system. Our approach is modular and "object oriented." This allows reusability and rapid reconfigurability of the controller as the machining system is reconfigured. We utilize the concept of timed transition models (TTM) to model a RMS. To specify the desired controlled behavior of the RMS, we use the tools of real-time temporal language introduced by Manna and Pnueli (1995). In this framework, the controller analysis problem can be posed as the problem of verifying that certain logical formulae are valid. Such verification can be carried out using either theorem proving techniques or model checking. We present some analytical results on a problem of system reconfiguration. An iterative approach for designing a controller based on the analysis result mentioned above is also presented. Using this approach, we can design a controller for a given set of closed-loop properties which guarantees correctness of the closed-loop system. The paper also illustrates how the state explosion problem inherent in model checking can be handled effectively by performing modular verification.

Logic controllers for machining systems typically have three control modes: auto, hand and manual. In this paper, a unified formal representation of logic controllers with three control modes is provided using Petri nets (PNs). A modular logic controller structure is introduced and formalized for high-volume transfer lines. The modular logic controller consists of one control module for the mode decision and other control modules for station logic controllers. Each station control module is represented by connecting together operation modules, which are designed with respect to the fault recovery processes of operations; their connection algorithm is also provided. In our formal representation, each control module is represented by a live, safe and reversible PN. A condition for the modular logic controller to generate a correct control logic is provided: the operation causality condition. Using the modular structure of a logic controller, the control logic can be easily reconfigured and automatic code generation is possible.

The class of piecewise linear homeomorphisms (PLH) provides a convenient functional representation for many applications wherein an approximation to data is required that is invertible in closed form. In this paper we introduce the graph intersection (GI) algorithm for "learning" piecewise linear scalar functions in two settings: "approximation", where an "oracle" outputs accurate functional values in response to input queries; and "estimation", where only a fixed discrete data base of input-output pairs is available. We provide a local convergence result for the approximation version of the GI algorithm as well as a study of its numerical performance in the estimation setting. We conclude that PLH offers accuracy closed to that of a neural net while requiring, via our GI algorithm, far shorter training time and preserving desired invariant properties unlike any other presently popular basis family.

The logic controller for a machining system is a discrete event supervisory system. In high volume transfer lines, it consists of three control modes: auto, hand, and manual. A logic controller can achieve the goal of a machining system by its control logic. The paper addresses a formal Petri net representation of the control logic and its implementation. The control logic is represented by two Petri net models: one for the mode decision control logic and the other for the sequence control logic. Because the Petri net models are live, safe, and reversible, the actual logic control program can be generated directly from the Petri net models by using IEC1131-3 programming languages.

The machining systems considered are high volume transfer lines. The cycle time during normal operation is used as the performance metric. Normal operation is governed by the auto-cycle function of a logic controller. A modular logic controller is introduced and formalized for event-based functional properties analysis of transfer line. By adding time specifications, a timed modular logic controller is generated. The time-based cyclic behavior of high volume transfer lines is characterized using timed Petri nets. Two efficient algorithms to compute the cycle time and critical operations are developed. A method for improving the performance of a transfer line is introduced.

The machining systems considered are high volume transfer lines which are widely used in automotive manufacturing. In these machining systems, several machines linked together provide complete processing of a part. A logic controller is a discrete event supervisory system which controls parallel and synchronized sequences of elementary operations of each machine to achieve the goal of the machining system. Normal operation is governed by the auto-cycle function of a logic controller. A modular logic controller is introduced and formalized for high volume transfer lines. Its event-based functional properties are verified and its reconfigurability is considered. A live and safe marked graph can be directly transformed into a sequential function chart (which is one of the IEC 1131-3 languages) and, using this SFC representation, a modular logic controller can be implemented. The performance analysis of the modular logic controller is also introduced. The cycle time during normal operation is used as the performance metric of a transfer line. By adding time specifications to the model, a timed modular logic controller is generated. The time-based cyclic behavior of high volume transfer lines is thus characterized using timed Petri nets. Two efficient algorithms to compute the cycle time and critical operations are developed.