Capsule networks (CapsNet) are a category of deep learning neural networks (DNN) that address one of the main issues and deficiencies of convolutional neural networks (CNN); loss of spatial information in pooling layers. However, the main concern with CapsNets is their compute-intensive nature, which is mainly related to the vector-based calculations during dynamic routing and acts as a barrier for their deployment in real-time applications. To address the specific computing requirements of dynamic routing in capsule layers of CapsN ets, we develop a hardware accelerator for dynamic routing using Vitis HLS. In this paper, we present a hardware acceleration solution for capsule networks by integrating AMD Xilinx deep processing unit (DPU) and a custom accelerator for the capsule layer. Our results show significant improvement in throughput compared with the baseline implementation when CapsNet is implemented on Zynq UltraScale+ MPSoC ZCU102 usinz Vitis AI DPUs.

Previous works have identified that EV charging can be weaponised to attack the power grid. As a case study, we consider the newest charging protocol ISO 15118–20, which provides a high-level communication protocol for EV charging. We first highlight fundamental issues in ISO 15118–20 which prevent the development of security features within the existing standard: We show that an attacker can perform a downgrade attack on ISO 15118–20, and propose modifications to the standard to prevent this. We show how this can be used to enable the development of additional security features within the modified protocol. A proof of concept is developed to prove functionality, determine interoperability between various parties, verify that it meets the original standard's timing requirements, and does not impact charging speed nor noticeably affect the length of a charging session.

Embedded real-time systems are tightly integrated with their physical environment. Their correctness depends both on the outputs and timeliness of their computations. The increasing use of multi-core processors in such systems is pushing embedded programmers to be parallel programming experts. However, parallel programming is challenging because of the skills, experiences, and knowledge needed to avoid common parallel programming traps and pitfalls. This article proposes the ForeC synchronous multi-threaded programming language for the deterministic, parallel, and reactive programming of embedded multi-cores. The synchronous semantics of ForeC is designed to greatly simplify the understanding and debugging of parallel programs. ForeC ensures that ForeC programs can be compiled efficiently for parallel execution and be amenable to static timing analysis. ForeC’s main innovation is its shared variable semantics that provides thread isolation and deterministic thread communication. All ForeC programs are correct by construction and deadlock free because no non-deterministic constructs are needed. We have benchmarked our ForeC compiler with several medium-sized programs (e.g., a 2.274-line ForeC program with up to 26 threads and distributed on up to 10 cores, which was based on a 2.155-line non-multi-threaded C program). These benchmark programs show that ForeC can achieve better parallel performance than Esterel, a widely used imperative synchronous language for concurrent safety-critical systems, and is competitive in performance to OpenMP, a popular desktop solution for parallel programming (which implements classical multi-threading, hence is intrinsically non-deterministic). We also demonstrate that the worst-case execution time of ForeC programs can be estimated to a high degree of precision.

Modern cars are being equipped with powerful computational resources for autonomous driving systems (ADS) as one of their major parts to provide safer travels on roads. High accuracy and real-time requirements of ADS are addressed by HW/SW co-design methodology which helps in offloading the computationally intensive tasks to the hardware part. However, the limited hardware resources could be a limiting factor in complicated systems. This paper presents a dynamically reconfigurable system for ADS which is capable of real-time vehicle and pedestrian detection. Our approach employs different methods of vehicle detection in different lighting conditions to achieve better results. A novel deep learning method is presented for detection of vehicles in the dark condition where the road light is very limited or unavailable. We present a partial reconfiguration (PR) controller which accelerates the reconfiguration process on Zynq SoC for seamless detection in real-time applications. By partially reconfiguring the vehicle detection block on Zynq SoC, resource requirements is maintained low enough to allow for the existence of other functionalities of ADS on hardware which could complete their tasks without any interruption. Our presented system is capable of detecting pedestrian and vehicles in different lighting conditions at the rate of 50fps (frames per second) for HDTV (1080x1920) frame.

Many modern forms of asymmetric multiprocessing (AMP) architecture use hypervisors to increase software security by isolating the system software in virtual machines. However, efficient virtualisation depends on hardware support that is not available across all products. Within modern ARM architectures, the aforementioned software isolation can also be implemented using ARM TrustZone technology. This paper presents a TrustZone-based AMP architecture (TZ-AMP) that can consolidate multiple system software environments securely on devices that lack hardware virtualisation support. We evaluate our prototype on the ARMv7-A architecture, and demonstrate TrustZone-based context-switching performance in the order of microseconds, confirming that TZ-AMP maintains high performance while also achieving hardware-backed software security.

In an age of increasing camera-based surveillance, there is also an increase in privacy concerns being voiced by the surveilled. Camera surveillance systems are expanding beyond law enforcement and public safety applications towards "video analytics" scenarios that act as a source of data for business and planning decisions. It is necessary for us to consider how perceptions of privacy may change under different conditions, so that appropriate safeguards can be designed for the relevant systems. Through an on-line survey, we determine that privacy concerns are not universal, and that they are heavily dependent on the specific context being considered. We find a wide distribution of perception between the respondents, and identify some of the underlying factors that lead to differences in opinion between different contexts.

For video analytics and surveillance applications, person re-identification across multiple camera views remains an open problem. The challenge of being able to determine that two images of people are the same person based solely on their appearance can be difficult, even for human observers. Many recent re-identification methods use deep learning with supervised learning to discriminate between the identity classes. However, the requisite training data is generally not available in real-world scenarios. In this paper, we compare a number of fast classification methods for the purposes of re-identification, taking extracted and pre-processed feature vectors and classifying them into identity classes, focusing on one-shot and unsupervised learning algorithms. We present two novel one-shot learning methods, including Sequential K-means, a computationally efficient algorithm with competitive accuracy. We demonstrate this on an indoor person tracking dataset, and discuss parameter tuning in order to further improve the accuracy of the algorithm.

In industrial automation, safety-critical control systems need robust timing guarantees in addition to functional correctness. Unfortunately, devices that are typically used in this domain, such as Programmable Logic Controllers, often feature architectures that are not amenable to static timing analysis, for instance relying on general purpose microprocessors or embedded operating systems. As a result, designers often rely on timing values gained from simple measurement of running applications, an approach that only provides very weak guarantees at best. The synchronous approach for IEC 61499 Function Blocks, in contrast, has been demonstrated to be time predictable when run on appropriate hardware, such as simple microprocessors. However, simple microprocessors are often not fast or powerful enough for modern automation requirements. In this paper, we examine how the performance of synchronous IEC 61499 can be improved through the usage of the multi-core T-CREST architecture, data scratchpads, and an optimised compiler. Overall, our improvements resulted in 60% shorter worst-case execution times.

Pedestrian detection is one of the most challenging and vital tasks of driver assistance systems (DAS). Among several algorithms developed for human detection, histogram of oriented gradients (HOG) followed by support vector machine (SVM) has shown the most promising results. This paper presents a hardware accelerator for real-time pedestrian detection at different scales to fulfill the real-time requirements of DAS. It proposes an algorithmic modification to the conventional multi-scale object detection by means of HOG+SVM to increase the throughput and maintain the accuracy reasonably high. Our hardware accelerator detects pedestrians at the rate of 60 fps for HDTV (1080x1920) frame.

Time predictability is a first class requirement in safety critical system design. Techniques exist for the timing analysis of programs designed in memory managed languages, but these require detailed knowledge of memory allocation. Moreover, enforcing hard real-time guarantees for systems designed in such garbage collected languages is difficult, because of the so called collection pause - however incremental. This paper proposes a novel solution whereby a separate reference counting unit replaces the garbage collector. We present the underlying concepts of the Reference Counting Memory Management Unit (RCMMU) and its use for garbage collection. In addition, we present an implementation that includes a specialized memory arrangement that allows for object management transactions to occur concurrently with program execution, with no resultant impact upon program performance. The RCMMU has been targeted for use with Java but can be adopted to be used with other memory managed languages. The hardware-implemented RCMMU removes all overhead associated with garbage collection operations when compared against a software-only collector. Additionally, it simplifies the static worst-case execution time analysis of programs.

MPSoC-based embedded systems design is becoming increasingly complex. Not only do we need to satisfy multiple design objectives, we increasingly need to address potential security risks. In this work, we propose a security-aware systematic design approach which explores the design space, given a system-level application description, by generating potential architecture configurations of execution platform nodes that are interconnected using a NoC. We then perform automated security analysis to check the generated configurations against designer-specified security constraints. Following the analysis, we use an automated architecture configuration refinement process to generate a list of security additions that are inserted into the initial configuration so that the security constraints are satisfied. By performing this refinement on several candidate configuration options, we can explore the trade-off between resource cost and security. In this paper, we illustrate the proposed approach using a Smart Home Control System application.

Many complex industrial control systems need to meet stringent timing requirements. Ensuring that implementations can meet these requirements can be a very difficult task. IEC 61499 is an emerging standard for modelling and implementing large distributed industrial control systems. However, there is currently no established approach for executing IEC 61499 code in a distributed time-predictable manner. In this paper, we present a novel time-predictable runtime called RunSync for executing IEC 61499 on the Precision Timed (PRET) architecture FlexPRET. PRET architectures are designed to guarantee timing repeatability while preserving performance. They are often designed as RISC architectures which utilize multiple hardware threads to remove pipeline hazards. Determining the allocation of tasks to the hardware threads is a key problem when utilizing such architectures. Hence, in this paper it is demonstrated that through RunSync it is possible to dynamically map IEC 61499 tasks to hardware threads during runtime, while preserving determinism and remaining amenable to timing analysis. Following that, quantitative results are presented, showing the minimal overheads of RunSync compared to implementations of the existing synchronous approach. RunSync is thus demonstrated to be more performant with large IEC 61499 networks, and when there are more hardware resources to be allocated.

This paper presents a new approach to designing an indoor Wi-Fi energy harvester with a bridgeless converter that operates at low incident power. The design approach started with the analysis of the incident power available at a point of the harvester with certain harvesting distance, dimension, position, and height from the ground plane. Then, a simple single-plane antenna was proposed. Finally, a bridgeless converter circuit was proposed to boost the voltage at very low input power. The analysis showed typical calculated incident power at the antenna on the pathway of the transmitter is approximately 15μW/m2and in the case of non-line-of-sight, the values dropped down to about 2 μW/m2. The proposed system has been tested in an indoor environment where the measured signal strength is between -60 dBm to -50 dBm; and the output voltage and power at the output of the converter were measured as 3.86 mV and 0.15 nW, respectively, a 100 kΩ resistive load is connected for testing. An additional boost-buffer module was connected to the converter circuit for further voltage boost. It was found that the energy harvester stabilizes after 38 hours of operation and 1.8 V was measured at the output supercapacitor (1000 uF), meaning 1.67 J of energy has been stored for driving external low power devices.

We describe an efficient GPU algorithm which extracts multiple contours from an image. The algorithm uses crack codes to generate contours which sit logically between adjacent image values; it works scan line by scan line and it can generate multiple contours in parallel with an image streamed directly from a camera. Whilst specifically targeted at detecting object contours in stereo disparity maps, it can also be used for general segmentation with a trivial change to the code generating the crack code masks. Using a480 ALU 1.4 GHz nVidia GPU, it can generate ~ 25000 contours from a real 2048 × 768 resolution 128 level disparity map image in ~ 29 ms if the contours are further processed in the GPU (additional ~5 ms to calculate shape moments) or ~ 39 ms if contours are transferred to the host. This is ~ 40 times faster than an OpenCV CPU implementation.

Histogram of oriented gradients (HOG) is considered as the most promising algorithm in human detection, however its complexity and intensive computational load is an issue for real-time detection in embedded systems. This paper presents a hardware accelerator for HOG feature extractor to fulfill the requirements of real-time pedestrian detection in driver assistance systems. Parallel and deep pipelined hardware architecture with special defined memory access pattern is employed to improve the throughput while maintaining the accuracy of the original algorithm reasonably high. Adoption of efficient memory access pattern, which provides simultaneous access to the required memory area for different functional blocks, avoids repetitive calculation at different stages of computation, resulting in both higher throughput and lower power. It does not impose any further resource requirements with regard to memory utilization. Our presented hardware accelerator is capable of extracting HOG features for 60 fps (frame per second) of HDTV (1080x1920) frame and could be employed with several instances of support vector machine (SVM) classifier in order to provide multiple object detection.

In this paper we present an approach to efficiently mix Java with asynchronous and synchronous concurrency and execute it on a specialized Java processor extended with capabilities for concurrency and reactivity. A new processor, which uses JOP (Java Optimized Processor) as its base, executes concurrent programs that comply with Globally Asynchronous Locally Synchronous (GALS) formal model of computation by clearly distinguishing between concurrency and reactivity control flow and Java control flow. The new processor, called JOP-Plus, can be used for embedded and even real-time applications in which majority of code is written in Java and the overall programs specified and structured in SystemJ system-level concurrent programming language.

This paper presents a novel, high performance and low cost execution architecture for the system level GALS programming language SystemJ, which extends Java with synchronous reactive features present in Esterel and asynchronous constructs of CSP (Communicating Sequential Processes). The new architecture is based on JOP (Java Optimized Processor), which is a hardware implementation of the Java Virtual Machine (JVM). The JOP, inherently suited to data-driven transformational operations, is extended to efficiently execute the control constructs and control flow of SystemJ. The new core, which is called RJOP (Reactive JOP) supports efficient execution of both data dominated and control dominated embedded applications. It also maintains the time-predictable execution of the applications intended for real-time embedded systems and calculation of Worst Case Reaction Time (WCRT) as provided by the original core. The initial results indicate significant performance improvement and lower resource requirements over the existing architectures used for the SystemJ execution.

This paper presents GALS-JOP processor for efficient execution of programs written in SystemJ GALS programming language, which extends Java with both synchronous and asynchronous concurrency and directly supports the design of concurrent and reactive programs that comply with globally asynchronous locally synchronous (GALS) formal model of computation (MoC). In the first step, a Java optimized processor (JOP) is enhanced with a control processor (CP), which deals with concurrency and reactivity, to design an intermediate solution, called tandem processor, or TP-JOP, in which control processor and JOP work together to implement control flow and data operations of GALS programs, respectively. Then, JOP and the CP functionalities are merged into a single processor, GALS-JOP, which enriches JOP with some key constructs and abstractions for efficient implementation of SystemJ GALS programs. Experimental results demonstrate superiority of the new processor over all other approaches for implementation of SystemJ programs so far making it suitable for embedded systems.

In recent years, there has been a growing interest in IP-reuse for SoCs in order to bridge the gap between the silicon capacity and the design productivity. This research work investigates how our proposed methodology can be used to partition and schedule a JPEG encoder IP core onto an FPGA. We will also describe a novel Constraint Satisfaction Problem (CSP) formulations that are used in the proposed framework. At the same time, we will also demonstrate the effectiveness of CSP in the Bayesian Network-based framework.

Reactivity on external events is an important feature of almost all embedded systems. In this paper we present the design of a new, reactive embedded microprocessor called REMIC, that supports reactivity in a new way following the paradigm of synchronous system level language Esterel. The rationale for REMIC design, its novel features with the design details and some performance figures are presented to demonstrate its suitability for embedded systems. Besides single processor systems, REMIC can be easily combined into multiple processor architectures that support real concurrency.

Energy-efficient and low-power designs issues are becoming more important in modern processors. In this paper we propose an energy-efficient adaptive multiple-issue architecture for portable systems. The architecture utilizes a hardware-software cooperation approach to minimize hardware overhead while providing control flexibilities to programs and operating systems. Optimizations are made based on the program execution profile. Simulation results show that the adaptive architecture improves the performance by an average 20% with a maximum 8% reduction in energy consumption compared with the baseline model. The increase in power dissipation is restricted to less than 6%.