Detecting and tracking objects in image sequences is paramount for any video analytic. While object detectors have become increasingly robust, motion estimates based on the straightforward approach of running a detector and linking detections are prone to bounding box noise. An algorithm for the monocular estimation of the 3D motion of rigid objects is presented, combining an object detector, minimal camera calibration, and 2D optical flows observed on the image plane. The algorithm utilizes the 2D Delaunay triangulation over geometrically consistent optical flow tracks to identify regions common to the same object. The algorithm is evaluated on the special case of image sequences of vehicles on roads. Experiments on BrnoCompSpeed dataset show that speed estimates from both detector-based tracking (mean error of 1.62 km/h) and optical flow based tracking (mean error of 2.19 km/h) perform competitively on straight roads. An extensive empirical evaluation is also conducted on a new dataset containing synthetic and real world scenes that also include vehicle trajectories involving rotation. A naive baseline bounding box track based method obtained a mean error of 5.29 km/h for vehicle speed, but optical flow tracks performed significantly better with mean error of 1.65 km/h on the new dataset including rotation. With the new method, video analytics such as vehicle speed estimation and lane change detection can obtain precise information about the 3D trajectory of rigid objects in motion.

Abstract The evergrowing Internet of Things (IoT) ecosystem continues to impose new requirements and constraints on every device. At the edge, low-end devices are getting pressured by increasing workloads and stricter timing deadlines while simultaneously are desired to minimize their power consumption, form factor, and memory footprint. Field-Programmable Gate Arrays (FPGAs) emerge as a possible solution for the increasing demands of the IoT. Reconfigurable IoT platforms enable the offloading of software tasks to hardware, enhancing their performance and determinism. This paper presents ChamelIoT, an agnostic hardware operating systems (OSes) framework for reconfigurable IoT devices. The framework provides hardware acceleration for kernel services of different IoT OSes by leveraging the RISC-V open-source instruction set architecture (ISA). The ChamelIoT hardware accelerator can be deployed in a tightly- or loosely-coupled approach and implements the following kernel services: thread management, scheduling, synchronization mechanisms, and inter-process communication (IPC). ChamelIoT allows developers to run unmodified applications of three well-established OSes, RIOT, Zephyr, and FreeRTOS. The experiments conducted on both coupling approaches consisted of microbenchmarks to measure the API latency, the Thread Metric benchmark suite to evaluated the system performance, and tests to the FPGA resource consumption. The results show that the latency can be reduced up to 92.65% and 89.14% for the tightly- and loosely-coupled approaches, respectively, the jitter removed, and the execution performance increased by 199.49% and 184.85% for both approaches.

Abstract Real time detection of falls and unstable movement by elderly people is vital to their quality of life and safety. We present an edge processing device integrated with a cloud computation framework that can be used for activity profiling as well as trigger alerts for falls and unstable motion by elderly people at home. The proposed system uses fixed cameras to track and analyze each visible person in the scene, classifying their actions into nine ordinary activities, a fall, or unstable movement. An alert notification is sent to caregivers whenever a fall or unstable movement is detected. The major components of the system include an embedded device (NVIDIA JETSON TX2) and cloud-based storage and analysis infrastructure. The system is composed of modules for detecting, tracking and recognizing humans, a cascaded hierarchical classifier for nine ordinary activities and falls, and a long short-term memory (LSTM) module to predict unstable movement in video. The system is designed for accuracy, usability, and cost. A prototype system has been subjected to individual module tests along with a field test within a volunteer’s household. It achieved an accuracy of 91.6% for ordinary actions and falls with a recall of 97.02% for unstable motion. Future phases will expand deployment to multiple homes.

Road fatality rates are very high, especially in developing and middle-income countries. One of the main causes of road fatalities is not using motorcycle helmets. Active law enforcement may help increase compliance, but ubiquitous enforcement requires many police officers and may cause traffic jams and safety issues. In this paper, we demonstrate the effectiveness of computer vision and machine learning methods to increase helmet compliance through automated helmet violation detection. The system detects riders and passengers not wearing helmets and consists of motorcyclist detection, helmet violation classification, and tracking. The architecture of the system comprises a single GPU server and multiple computational clients that cooperate to complete the task, with communication over HTTP. In a real-world test, the system is able to detect 97% of helmet violations with a 15% false alarm rate. The client-server architecture reduces cost by 20-30% compared to a baseline architecture.

Computer Vision is an emerging field with diverse applications which encompasses many algorithms with heavy computations. Histogram of Oriented Gradients-Support Vector Machine (HOG-SVM) is one such versatile algorithm used for object detection and image classification despite it's heavy computation load. Processing such an algorithm in real time with adequate throughput is a challenging task for a general purpose processor. Moreover, an embedded CPU with very limited processing power could least cater such heavy processing. Therefore our research in general focuses on developing application specific architectures for hardware acceleration of computer vision algorithms. This paper presents a continuation of a series of research to hardware accelerate HOG-SVM algorithm on FPGA. In this paper we mainly present the high performance application specific architecture for hardware acceleration of HOG-SVM which was successful in achieving a high throughput of 240fps on HD frames of size 1920x1080 which is a significant improvement of performance compared to previous research. On the other-hand, both hardware utilization and power consumption are minimized. A mechanism based around Block RAM (BRAM) structures and deep pipelining are used as the key architectural techniques of achieving high performance. The proposed design was deployed on Zynq 7000 FPGA platform which contains a hardwired ARM CPU along with the programmable FPGA fabric. The accelerator is deployed on the FPGA and integrated with the ARM CPU using AXI memory interfaces. A hardware thread model and bare-metal device drivers were developed which encapsulate the behavior of the accelerator as a hardware thread to the applications running on the ARM CPU.

Mobility and ambient conditions are key factors in urban environments, affecting well-being and quality of life. In this context, sensors, smart mobility, networks, connectivity can play a significant and strategic role, being exploited with the aim of improving data and information available to public administration and to each citizen. In this way, they can be supported in having more sustainable and aware behaviours and in getting useful information and services, improving their daily activities. In this paper, we present a prototype of smart bike eco-system, designed with the aim of collecting, aggregating and sharing data about air pollution and about the urban environment, which can be exploited in a smart mobility context thanks to sensor and vehicular networks.

Most hypervisors today rely either on (1) full virtualization on high-end hardware (i.e., hardware with virtualization extensions), (2) paravirtualization, or (3) both. These, however, do not fulfill embedded systems' requirements, or require legacy software to be modified. Full virtualization on low-end hardware (i.e., hardware without virtualization extensions), on the other end, has none of those disadvantages. However, it is often claimed that it is not feasible due to an unacceptably high virtualization overhead. We were, nevertheless, unable to find real-world quantitative results supporting those claims.

Virtualization has been used as the de facto technology to allow multiple operating systems (virtual machines) to run on top of the same hardware platform. In the embedded systems domain, virtualization research has focused on the coexistence of real-time requirements with non-real-time characteristics. However, existent standard software-based virtualization solutions have been shown to negatively impact the overall system, especially in performance, memory footprint and determinism. This work in progress paper presents the implementation of an embedded virtualization architecture through commodity hardware. ARM TrustZone technology is exploited to implement a lightweight virtualization solution with low overhead and high determinism, corroborated by promising preliminary results. Research roadmap is also pointed and discussed.

In this paper, we propose two methods used in 3-D IC placement that efficiently exploit the die-to-die thermal coupling in the stack. First, through-silicon vias (TSVs) are spread on each die to reduce the local power density and vertically aligned across dies simultaneously to increase thermal conductivity to the heatsink. Second, we move high-power logic cells to the location that has higher conductivity to the heatsink while moving TSVs in the upper dies so that high-power cells are vertically overlapping below the TSVs. These methods are employed in a force-directed 3-D placement successfully and outperform several state-of-the-art placers published in recent literature. We obtain 3-D placement results with shorter routed wirelength at similar temperature. We also obtain 3-D placement results with lower temperatures at similar routed wirelengths.

Due to the scientific and technological advances of the last decades, embedded systems have become central to modern life, which has led to a new reality in which the embedded systems market has been growing with a monthly or even weekly emergence of new products with different applications across several domains. Focus on embedded systems design is driven by: (1) high and fast penetration in products and services due to the integration of networks, operating systems and database capabilities, (2) the economically strategic nature of the field, and (3) its status as a new, and relatively undefined, subject in academic environment. We'll present the development of the educational context for the new technologies, and show how our Integrated Master Curriculum in Industrial Electronics and Computer Engineering has been adapted to satisfy the needs of the industry and meet all the above mentioned challenges.

The proliferation of highly-configurable FPGA technology has allowed the implementation of dedicated systems of diverse configurations and fueled the software to hardware migration paradigm. This work demonstrates how the hardware implementation of virtualization technology affects the system reliability at several levels of abstraction. By correlating faults between the physical and virtual, the reliability impact of hardware-assisted virtualization is shown, as well as how runtime faults are capable of breaching virtualization. ISA profiling is used to assess reliability at early design stages and how its use can serve as a robustness guideline for hardware and software designers is explained.

This paper presents an innovative Multi Processor System on Chip (MPSoC) architecture based on Service Oriented Architecture (SOA) concepts. The proposed architecture promotes the design flexibility and interoperability by setting an implementation agnostic services' integration interface and being reconfiguration aware by design. The concept is tried with an example of an embedded legacy support system, a Dynamic Binary Translation (DBT) engine based on three services. DBT can solve the x86 architecture virtualization limitations. A SOA based MPSoC endowed of autonomous dynamic reconfiguration capabilities is introduced and pointed as a research opportunity. The introduction in the architecture of a Service Module responsible for Partial Runtime Reconfiguration (PRTR) is expected to dynamically adjust the architecture to better match the workload.

Through automated agricultural inspection, farmers can potentially achieve better productivity and accurately predict yields and crop quality. A variety of sensors can be used for agricultural inspection, but the cheapest and most information-rich is the video camera. We collect data in the field from a monocular camera fixed to a mobile inspection platform. For purposes of pineapple crop mapping and yield prediction, we propose an image processing framework for in-field fruit detection, tracking, and 3D reconstruction. We perform a series of experiments on feature point extraction using Harris, SIFT, and SURF features, feature point description using SIFT and SURF descriptors, feature point classification using SVMs, fruit region tracking using blob tracking, and 3D reconstruction using structure from motion and robust ellipsoid estimation techniques. We find that SURF feature points and descriptors provide the best tradeoff between processing time and classification accuracy and that the method is sufficiently accurate for fruit region detection. Our preliminary results for fruit region tracking and 3D fruit reconstruction are promising. We plan further work towards development of a useful aid to help farmers manage their farms.

In this paper we present the Statistical Retiming-based Timing Analysis (SRTA) algorithm. The goal is to compute the timing slack distribution for the nodes in the timing graph and identify the statistically critical paths under retiming, which are the paths with a high probability of becoming timing-critical after retiming. SRTA enables the designers to perform circuit optimization on these paths to reduce the probability of them becoming timing bottleneck if the circuit is retimed as a post-process. We provide a comparison among static timing analysis (= STA), statistical timing analysis (= SSTA), retiming-based timing analysis (= RTA), and our statistical retiming-based timing analysis (SRTA). Our results show that the placement optimization based on SRTA achieves the best performance results.

This paper presents the first multiobjective microarchitectural floorplanning algorithm for high-performance processors implemented in two-dimensional (2-D) and three-dimensional (3-D) ICs. The floorplanner takes a microarchitectural netlist and determines the dimension as well as the placement of the functional modules into single- or multiple-device layers while simultaneously achieving high performance and thermal reliability. The traditional design objectives such as area and wirelength are also considered. The 3-D floorplanning algorithm considers the following 3-D-specific issues: vertical overlap optimization and bonding-aware layer partitioning. The hybrid floorplanning approach combines linear programming and simulated annealing, which is shown to be very effective in obtaining high-quality solutions in a short runtime under multiobjective goals. This paper provides comprehensive experimental results on making tradeoffs among performance, thermal, area, and wirelength for both 2-D and 3-D ICs

Process variations in digital circuits make sequential circuit timing validation an extremely challenging task. In this paper, a Statistical Bellman-Ford (SBF) algorithm is proposed to compute the longest path length distribution for directed graphs with cycles. Our SBF algorithm efficiently computes the statistical longest path length distribution if there exist no positive cycles or detects one if the circuit is likely to have a positive cycle. An important application of SBF is Statistical Retiming based Timing Analysis (SRTA), where SBF is used to check for the feasibility of a given target clock period distribution for retiming. Our gate and wire delay distribution model considers several high-impact intra-die process parameters and accurately captures the spatial and reconvergent path correlations. The Monte Carlo simulation is used to validate the accuracy of our SBF algorithm. To the best of our knowledge, this is the first paper that propose the statistic version of the longest path algorithm for sequential circuits

In this paper, we present the first multi-objective microarchitectural floorplanning algorithm for designing highperformance, high-reliability processors in the early design phase. Our floorplanner takes a microarchitectural netlist and determines the placement of the functional modules while simultaneously optimizing for performance and thermal reliability. The traditional design objectives such as area and wirelength are also considered. Our multi-objective hybrid floorplanning approach combining Linear Programming and Simulated Annealing is shown to be fast and effective in obtaining high-quality solutions. We evaluate the trade-off of performance, temperature, area, and wirelength and provide comprehensive experimental results.

The integration of retiming and simultaneous supply/threshold voltage scaling has a potential to enable more rigorous total power reduction. However, such integration is a highly complex task due to its enormous solution space. This paper presents the first algorithm that performs retiming and simultaneous supply/threshold voltage scaling. In our three-step approach, low power retiming is first performed to reduce the clock period while taking the FF delay/power into consideration. Next, the subsequent voltage scaling makes the best possible supply/threshold voltage assignment under the given clock period constraint set by the retiming. Finally, a post-process further refines the voltage scaling solution by exploiting the remaining timing slack in the circuit. Related experiments show that our min-FF retiming plus simultaneous Vdd/Vth scaling approach reduces the total power consumption by 34% on average compared to the existing max-FF retiming plus Vdd Scaling approach.

Dataflow architectures provide an abundance of computing units that can be statically or dynamically configured to match the computing requirements of the given application. Wire delay has a reduced impact in dataflow architectures because only neighboring architectural entities are allowed to communicate within a single clock cycle. In this paper, the authors propose integer linear programming (ILP)-based placement and routing algorithms for mapping dataflow graphs (DFGs) to dataflow machines. The optimization process is guided by profiling information available from the compiler. The goal is to minimize the total execution time of the given application represented by a DFG under architectural constraints. A hierarchical method to handle the complexity of the initial ILP formulation is proposed. The profile-driven ILP algorithm reduces the total execution time of benchmark applications compared to the conventional wirelength-driven ILP approach. In addition, the ILP-based approach outperforms simulated annealing-based approach

As very large scale integration (VLSI) process technology migrates to nanoscale with a feature size of less than 100 nm, global wire delay is becoming a major hindrance in keeping the latency of intrachip communication within a single cycle, thus substantially decaying performance scalability. In addition, an effective microarchitectural floor planning algorithm can no longer ignore the dynamic communication patterns of applications. This article, using the profile information acquired at the microarchitecture level, proposes a "profile-guided microarchitectural floor planner" that considers both the impact of wire delay and the architectural behavior, namely, the intermodule communication, to reduce the latency of frequent routes inside a processor and to maintain performance scalability. Based on the simulation results here, the proposed profile-guided method shows a 5%-40% average instructions per cycle (IPC) improvement when the clock frequency is fixed. From the perspective of instruction throughput in billion instructions per second (BIPS), the floor planner is much more scalable than the conventional wirelength-based floor planner.

As wire delay increasingly becomes a significant performance bottleneck in monolithic architectures, there is a strong motivation to move to Dataflow Architectures. In this paper, we propose a set of placement algorithms for generic dataflow architectures. Our timing-driven and profile-driven placement algorithms respectively are targeting streaming and non-streaming applications. Compared to the conventional wirelength-driven algorithm, our timing-driven placer reduces the longest path delay by 23% and maximum slack by 26% at the cost of 10% increase in wirelength for streaming applications. In addition, our profile-driven placer reduces the total execution time of non-streaming applications by 17%. Lastly, our simultaneous timing/profile-driven placer reduces the total execution time of non-streaming applications by 13% on average.

Dataflow architectures provide an abundance of computing units that can be statically or dynamically configured to match the computing requirements of the given application. Wire delay has a reduced impact in dataflow architectures because only neighboring architectural entities are allowed to communicate within a single clock cycle. In this paper, we propose MILP-based placement and routing algorithms for mapping dataflow graphs to dataflow machines. The optimization process is guided by profiling information available from the compiler. Our goal is to minimize the total execution time of the given application represented by a dataflow graph under architectural constraints. We propose a hierarchical method to handle the complexity of the initial MILP formulation. Our profile-driven MILP algorithm reduces the total execution time of benchmark applications compared to the conventional wirelength-driven approach on average by 18%.

This paper introduces a compiler orchestrated prefetching system as a unified framework geared toward ameliorating the gap between processing speeds and memory access latencies. We focus the scope of the optimization on specific subsets of the program dependence graph that succinctly characterize the memory access pattern of both regular array-based applications and irregular pointer-intensive programs. We illustrate how program embedded precomputation via speculative execution can accurately predict and effectively prefetch future memory references with negligible overhead. The proposed techniques reduce the total running time of seven SPEC benchmarks and two OLDEN benchmarks by 27% on an Itanium 2 processor. The improvements are in addition to several state-of-the-art optimizations including software pipelining and data prefetching. In addition, we use cycle-accurate simulations to identify important and lightweight architectural innovations that further mitigate the memory system bottleneck. In particular, we focus on the notoriously challenging class of pointer-chasing applications, and demonstrate how they may benefit from a novel scheme of it sentineled prefetching. Our results for twelve SPEC benchmarks demonstrate that 45% of the processor stalls that are caused by the memory system are avoidable. The techniques in this paper can effectively mask long memory latencies with little instruction overhead, and can readily contribute to the performance of processors today.

As process technology migrates to deep submicron with feature size less than 100nm, global wire delay is becoming a major hindrance in keeping the latency of intra-chip communication within a single cycle, thus decaying the performance scalability substantially. An effective floorplanning algorithm can no longer ignore the information of dynamic communication patterns of applications. In this paper, using the profile information acquired at the architecture/microarchitecture level, we propose a "profile-guided microarchitectural floorplanner" that considers both the impact of wire delay and the architectural behavior, namely the inter-module communication, to reduce the latency of frequent routes inside a processor and to maintain performance scalability. Based on our simulation results, the profile-guided method shows a 5% to 40% average IPC improvement when clock frequency is fixed. From the perspective of instruction throughput(in BIPS), our floorplanner is much more scalable than a conventional wire length based floorplanner.