We describe a formulation of multi-agents operating within a Cyber Physical System, resulting in collaborative or adversarial games. We show that the non-determinism inherent in the communication medium between agents and the underlying physical environment give rise to environment evolution that is a probabilistic function of agents’ strategies. We name these emergent properties Cyber Physical Games and study its properties. We present an algorithmic model that determines the most likely system evolution, approximating Cyber Physical Games through Probabilistic Finite State Automata, which can be used to drive agent strategies toward best performance. We evaluate it on collaborative and adversarial versions of the Iterated Boolean Game, comparing theoretical results with simulated ones, showing that the algorithm can be successfully used to guide agent strategy, outperforming other methods. Results support the validity of the proposed model and suggest several required research directions to continue evolving our understanding of Cyber Physical System, as well as how to best design agents that must operate within such environments.

We present a hardware-accelerated SAT solver targeting processor/Field Programmable Gate Arrays (FPGA) SoCs. Our solution accelerates the most expensive subroutine of the Davis-Putnam-Logemann-Loveland (DPLL) algorithm, Boolean Constraint Propagation (BCP) through fine-grained FPGA parallelism. Unlike prior state-of-the-art solutions, our solver eliminates costly clause look-up operations by assigning clauses directly to clause processors on the FPGA and dividing large formulas into smaller partitions manageable by FPGA. Partitions are hot-swapped during runtime as required and the supported formula size is limited only by available external memory, not on-chip FPGA memory.

While deep learning has been very successful in computer vision, real world operating conditions such as lighting variation, background clutter, or occlusion hinder its accuracy across several tasks. Prior work has shown that hybrid models—combining neural networks and heuristics/algorithms—can outperform vanilla deep learning for several computer vision tasks, such as classification or tracking.We consider the case of object tracking, and evaluate a hybrid model (PhyOT) that conceptualizes deep neural networks as "sensors" in a Kalman filter setup, where prior knowledge, in the form of Newtonian laws of motion, is used to fuse sensor observations and to perform improved estimations. Our experiments combine three neural networks, performing position, indirect velocity and acceleration estimation, respectively, and evaluate such a formulation on two benchmark datasets: a warehouse security camera dataset that we collected and annotated and a traffic camera open dataset.Results suggest that our PhyOT can track objects in extreme conditions that the state-of-the-art deep neural networks fail while its performance in general cases does not degrade significantly from that of existing deep learning approaches. Results also suggest that our PhyOT components are generalizable and transferable.

The range of applications that make up the Internet-of-Things ecosystem continues to grow. New opportunities present themselves along with new design challenges concerning the efficiency, portability, and processing capabilities of future Internet-of-Things systems. Thus, improving the operational metrics of individual Internet-of-Things devices, particularly across the edge and fog layers, is of paramount importance.In this manuscript, we present an approach for decreasing data collection at the edge, thus reducing form factor and power consumption of edge devices. This is particularly relevant for our application of interest, wearable motion capture, where human comfort and operational longevity are of prime importance. Our approach extrapolates from reduced edge data by leveraging prior physiological knowledge of the captured entity at the computational stage in the fog. By delegating computation to the fog, we also demonstrate the possibility for expanded capture volumes (operational areas) for future wearable motion capture systems and motion capture systems in general.Our approach, when prototyped on a millimeter wave sensor edge device and two fog node platforms of different processing tiers, shows that prior knowledge can facilitate a reduction in capture data dimensionality (and an associated decrease in power consumption) with little to no accuracy degradation, when compared to a more data-intensive edge system (Microsoft Kinect).

This manuscript reports the results of an experiment to assess the effectiveness of multi-abstraction computing tools on promoting exploratory self-learning in engineering, using a custom Real-Time Operating System (RTOS) during remote learning. Our goals were to determine whether students respond to guided exploratory learning opportunities during remote learning, and whether these contributed to immediate and further student success.Students in a 3rd year course were given the custom multi-abstraction RTOS. The higher abstraction layer was taught directly, whilst students were directed towards exploratory self-learning among the lower abstraction layer (which includes details beyond prior knowledge or the content of the course). In a survey, students were asked about their perceptions of using higher and lower abstraction layers and how these impacted their learning. Responses were correlated with course grades.Students that engaged with the lower abstraction layer demonstrated higher success (3 percentage points above class average), engagement (22.5% reporting they were more engaged), accuracy of self-assessment (1.3 percentage points above class average), and learning beyond course’s expectations (14% reporting they learned beyond the course’s learning outcomes).

The shift towards reconfigurable systems-hardware and software that adapt themselves to an external context-promises to unlock unprecedented performance, power consumption, and quality of service. However, reconfiguration imposes several challenges on the design of cyber-physical systems. Current design practices, including software frameworks and programming languages, are ill-prepared for supporting reconfiguration. In this paper, we explore Asynchronous Graph Programming, a programming paradigm and an associated model of computation designed for efficient and automated parallelization across processing elements, efficient reconfiguration (i.e., mapping of computational tasks across processing elements), and combining synchronous and asynchronous I/O handling within the same conceptual programming model. We also introduce an analytical model of parallelization, unlocked by Asynchronous Graph Programming, that can inform reconfiguration strategies. We analyze the implications of our model through an analysis of reconfiguration scenarios given a program's characteristics; our analysis quantifies the benefits of reconfiguring software for higher levels of parallelism, given an amount of data left to process. We also introduce Horde, an open source Asynchronous Graph Programming interpreter, and use it to empirically validate the performance advantage of its automatic parallelism capabilities; in our experiments, automatic parallelization from one to four cores improves average case execution time by a factor of 2 and worst case execution time by a factor of 3.

Programming abstractions decrease the cognitive gap between program idealization and expression. In the software domain, this high-level expressive power is achieved through layered abstractions - virtual machines, compilers, operating systems - which translate, at design and runtime, programmer visible code into hardware-compatible code. While this paradigm is ideal for static, i.e., unmodifiable, hardware, several of these abstractions break down when programming configurable hardware. State of the art hardware/software co-design techniques (e.g., High Level Synthesis (HLS), Intermediate Fabrics) are, for the most part, ad hoc patches to the traditional abstraction stack, applicable only to specific toolchains or software components. In this paper, we survey current hardware design and hardware/software co-design abstractions, from the perspective of the design language/toolchain. We perform a systematic analysis of different design paradigms, including HLS, Domain Specific Languages (DSL), and new-generation Hardware Description Languages (HDL). We analyze how these paradigms differ in expressiveness, support for hardware/software interaction, hierarchy and modularity, HDL interoperability, and interface with the outside world.

Dataflow languages are widely used for programming real-time embedded systems. They offer high level abstraction above hardware, and are amenable to program analysis and optimisation. This paper addresses the challenge of verifying parallel program transformations in the context of dynamic dataflow models, where the scheduling behaviour and the amount of data each actor computes may depend on values only known at runtime. We present a Linear Temporal Logic (LTL) model checking approach to verify a dataflow program transformation, using three LTL properties to identify cyclostatic actors in dynamic dataflow programs. The workflow abstracts dataflow actor code to Fiacre specifications to search for counterexamples of the LTL properties using the Tina model checker. We also present a new refactoring tool for the Orcc dataflow programming environment, which applies the parallelising transformation to cyclostatic actors. Parallel refactoring using verified transformations speedily improves FPGA performance, e.g.15.4 × speedup with 16 actors.

Specialized FPGA implementations can deliver higher performance and greater power efficiency than embedded CPU or GPU implementations for real-time image processing. Programming challenges limit their wider use, because the implementation of FPGA architectures at the register transfer level is time consuming and error prone. Existing software languages supported by high-level synthesis (HLS), although providing a productivity improvement, are too general purpose to generate efficient hardware without the use of hardware-specific code optimizations. Such optimizations leak hardware details into the abstractions that software languages are there to provide, and they require knowledge of FPGAs to generate efficient hardware, such as by using language pragmas to partition data structures across memory blocks. This article presents a thorough account of the Rathlin image processing language (RIPL), a high-level image processing domain-specific language for FPGAs. We motivate its design, based on higher-order algorithmic skeletons, with requirements from the image processing domain. RIPL’s skeletons suffice to elegantly describe image processing stencils, as well as recursive algorithms with nonlocal random access patterns. At its core, RIPL employs a dataflow intermediate representation. We give a formal account of the compilation scheme from RIPL skeletons to static and cyclostatic dataflow models to describe their data rates and static scheduling on FPGAs. RIPL compares favorably to the Vivado HLS OpenCV library and C++ compiled with Vivado HLS. RIPL achieves between 54 and 191 frames per second (FPS) at 100MHz for four synthetic benchmarks, faster than HLS OpenCV in three cases. Two real-world algorithms are implemented in RIPL: visual saliency and mean shift segmentation. For the visual saliency algorithm, RIPL achieves 71 FPS compared to optimized C++ at 28 FPS. RIPL is also concise, being 5x shorter than C++ and 111x shorter than an equivalent direct dataflow implementation. For mean shift segmentation, RIPL achieves 7 FPS compared to optimized C++ on 64 CPU cores at 1.1, and RIPL is 10x shorter than the direct dataflow FPGA implementation.

Multithreading is a key feature for dealing with the complexity of current generation embedded devices. Real-Time Operating Systems (RTOSes) provide a higher level of abstraction that alleviates design complexity and time-to-market pressure, inducing, however, undesired latencies and unpredictable execution times. Research works have been focusing on improving performance, determinism or agnosticism, but none simultaneously tackled the three metrics. This work in progress paper presents RT-SHADOWS, a co-designed architecture that promotes configurability, determinism and agnosticism from the outset. A multi-thread ARM softcore processor was exploited and extended by offloading the scheduling-related features into the hardware layer. The hardware multi-thread support is transparent for the standard RTOS applications and independent of the Operating System (OS). Promising preliminary results demonstrate huge improvements on performance and determinism, with only a small cost on hardware.

The proliferation of highly-configurable FPGA technology has allowed the implementation of dedicated systems of diverse configurations and fueled the software to hardware migration paradigm. This work demonstrates how the hardware implementation of virtualization technology affects the system reliability at several levels of abstraction. By correlating faults between the physical and virtual, the reliability impact of hardware-assisted virtualization is shown, as well as how runtime faults are capable of breaching virtualization. ISA profiling is used to assess reliability at early design stages and how its use can serve as a robustness guideline for hardware and software designers is explained.

This paper presents an innovative Multi Processor System on Chip (MPSoC) architecture based on Service Oriented Architecture (SOA) concepts. The proposed architecture promotes the design flexibility and interoperability by setting an implementation agnostic services' integration interface and being reconfiguration aware by design. The concept is tried with an example of an embedded legacy support system, a Dynamic Binary Translation (DBT) engine based on three services. DBT can solve the x86 architecture virtualization limitations. A SOA based MPSoC endowed of autonomous dynamic reconfiguration capabilities is introduced and pointed as a research opportunity. The introduction in the architecture of a Service Module responsible for Partial Runtime Reconfiguration (PRTR) is expected to dynamically adjust the architecture to better match the workload.

The heterogeneity of device interfaces and the disparity between interconnection standards leads to difficulty in achieving seamless device interoperability. The Open Services Gateway Initiative (OSGi) framework attempts to provide a unified Residential Service gateway by abstracting from the device interface and providing transparent methods for Service Oriented Architectures. However, it is not yet compliant with all standards and unsuited for some. This paper presents the development of a Multi-Camera Home Network System, using devices which implement three different protocols. The OSGi gateway bundles to handle the interoperability are described including one OSGi bundle for ONVIF devices. A Residential Service OSGi gateway is also presented and the difficulties faced with the integration of video streaming on the framework are presented, as well as the solutions used to tackle the problem.