Recently, biologically inspired vision systems have been the focus of intense research effort to emulate the high energy-efficiency, performance and robustness of mammalian vision systems. However, previous vision accelerators have only focused on speeding up computationally intense portions of the system without exploiting effects seen in the human brain that demonstrate the task influence in the vision mechanism. In this paper, we propose a task-oriented two-level vision system which is composed of Saliency and SURF. To the best of our knowledge, our design is the first embedded system that utilizes task influence in the computation of visual attention and recognition. As a result, we show that the new system can achieve at most 12.75% accuracy improvement while saving 25% computation work.

This work proposes an adaptive architecture for a neuro-inspired vision pipeline that uses a data driven approach to dynamic configuration and optimization of the underlying computational fabric. The micro-architecture allows the adaptation mechanism to be tuned for changing system objectives that cross-cut accuracy, power consumption, and throughput constraints.

Video and image content has begun to play a growing role in many applications, ranging from video games to autonomous self-driving vehicles. In this paper, we present accelerators for gist-based scene recognition, saliency-based attention, and HMAX-based object recognition that have multiple uses and are based on the current understanding of the vision systems found in the visual cortex of the mammalian brain. By integrating them into a two-level hierarchical system, we improve recognition accuracy and reduce computational time. Results of our accelerator prototype on a multi-FPGA system show real-time performance and high recognition accuracy with large speedups over existing CPU, GPU and FPGA implementations.

Empowered by the massive growth of camera enabled mobile devices; mobile applications that allow users to perceive and experience the world in richer and more engaging ways have emerged at tremendous pace. As more complex perception algorithms are developed to take advantage of higher resolution imagery, future mobile applications will require application specific accelerators to maintain performance required for interactive user experiences. A key challenge in these accelerator-rich mobile platforms will be guaranteeing the off-chip memory bandwidth required by each accelerator. Device integration techniques such as Package on Package and Wide-IO seek to tackle the memory wall problem by reducing bottlenecks at the I/O interfaces. However, less effort has been focused on solving the bandwidth problem by dynamically leveraging the individual and collective bandwidth characteristics of accelerators operating concurrently. This work investigates the off-chip bandwidth characteristics of accelerators in the context of embedded perceptual computing applications. A bandwidth aware feedback system is proposed that dynamically partitions available bandwidth among a set of accelerators at the expense of application accuracy. As a case study, the proposed adaption policy is applied to a biologically-inspired scene understanding application. Results indicate that the system maintains good accuracy while requiring only 25% of the original bandwidth.

The emerging Phase Change Memory (PCM), while having many advantages, suffers from slow write operations. This is mainly due to its asymmetric write characteristic, i.e., for two types of write operations of PCM, SET is much slower than RESET. Recent study has shown that proactively setting dirty memory lines to all ‘1’s can enable RESET-only writes when these lines are written back from the cache, which helps to reduce the effective write latency. Unfortunately, it results in higher write power demand. In this paper, we propose WoM-SET, a low power proactive-SET-based write strategy. By exploiting the WoM (write-once memory) code, we greatly reduce the number of RESETs per write and hence the write power demand. By applying our design only to write-intensive pages, we restrict the extra space requirement in WoM-SET. Our experiments show that WoM-SET achieves 40% RESET bit reduction, 40% write power reduction, and 12% energy-delay-product improvement over the PreSET scheme.

Compute-intensive applications are emerging in intelligent home, retail store and automotive industries. These applications are becoming more sophisticated with new features rich in audio, video, image, and machine learning capabilities that demand heavy computations. We present the EMERALD (EMERging Applications and algorithms for Low power Device) workload suite. We profile the workloads to show the hotspot functions that are candidates for hardware accelerators.

Conventional cache replacement policies use access information of each cache block for replacement decisions. We observe that there are many identical tags across different cache sets because programs exhibit spatial locality. The number of different tags in cache memory is significantly less than the total number of cache blocks in a cache. We propose a tag-based replacement that uses access frequency and recency of tags instead of cache blocks for the replacement decision. The tag-based replacement reduces the average miss rate of the baseline 1MB L2 cache by 15% over conventional LRU with 95% status bits reduction over conventional LRU. The performance improvement of a processor using the tag-based replacement is up to 40% with an average of 4.5% over LRU.

The widely used LRU replacement policy suffers from the following problems. First, LRU does not exploit fre-quency information of cache accesses. Second, LRU may experience cache thrashing when access to cache exhibits cyclic patterns and the cache capacity is less than the working set. Finally, LRU is expensive to implement in hardware. We propose a bubble replacement for low-level caches, where cache blocks in one set are arranged in a queue for replacement determination. An incoming block enters the queue from the bottom and exchanges its posi-tion with the block above when the block hits, therefore, both recency and frequency information of a program are exploited. A victim block can be chosen from either the bottom or the top block of the queue, which is controlled by a single-bit set-hit flag per set. Choosing the bottom block as the victim makes the bubble replacement resistant to less frequently used blocks from polluting the cache while choosing the top block as the victim makes the bub-ble replacement adaptable to changes in the working set. We also propose to divide the blocks in a cache set into groups where each group implements the bubble replace-ment (we name it the DC-Bubble) to resolve the problems of the bubble replacement. The victim block is chosen ran-domly from the bottom block of each group. The proposed DC-Bubble reduces the average MPKI of the baseline 1MB 16-way L2 cache by 14%, bridges 47% of the gap between LRU and the OPT, reduces the storage require-ment by 61% and simplifies the circuit design compared to LRU.

High associativity is important for level-two cache designs [9]. Implementing CAM-based Highly Associative Caches (CAM-HAC), however, is both costly in hardware and exhibits poor scalability. We propose to implement the CAM-HAC in macro-blocks to improve scalability. Each macro-block contains 128-row and 8-column of cache blocks. We name it Two dimensional Cache, or T-Cache. Each macro-block has an associativity equivalent to 128×8=1024-way. Twelve bits of the T-Cache’s tag are implemented by using CAM, while the remaining tag uses SRAM; Furthermore, random replacement is used in rows to balance cache sets usage while LRU is used in columns to select the victim from a row. The hardware complexity for replacement is reduced greatly compared to a traditional CAM-HAC using LRU solely. Experimental results show that the T-Cache achieves a 16% miss rate reduction over a traditional 8-way unified L2 cache. This translates into an average IPC improvement of 5% and as high as 18%. The T-Cache exhibits a 4% total memory access-related energy savings due to the reduction to applications’ execution time.

Level-one caches normally reside on a processor's critical path, which determines clock frequency. Therefore, fast access to level-one cache is important. Direct-mapped caches exhibit faster access time, but poor hit rates, compared with same sized set-associative caches because of nonuniform accesses to the cache sets. The nonuniform accesses generate more cache misses in some sets, while other sets are underutilized. We propose to increase the decoder length and, hence, reduce the accesses to heavily used sets without dynamically detecting the cache set usage information. We increase the access to the underutilized cache sets by incorporating a replacement policy into the cache design using programmable decoders. On average, the proposed techniques achieve as low a miss rate as a traditional 4-way cache on all 26 SPEC2K benchmarks for the instruction and data caches, respectively. This translates into an average IPC improvement of 21.5 and 42.4% for SPEC2K integer and floating-point benchmarks, respectively. The B-Cache consumes 10.5% more power per access, but exhibits a 12% total memory access-related energy savings as a result of the miss rate reductions, and, hence, the reduction to applications' execution time. Compared with previous techniques that aim at reducing the miss rate of direct-mapped caches, our technique requires only one cycle to access all cache hits and has the same access time of a direct-mapped cache.

We propose a low-leakage register file cell design based on the observation that the physical registers in a superscalar processor have very short life cycles. When a register is dead, we discharge its cells to '0' to greatly reduce the leakage current from the read bitlines to the ground. Our design has no impact to critical register read access path. Projected to future 45 nm technology, our design yields additional 38% and 47% leakage power savings on top of the existing low-leakage cell designs for 64-bit and 32-bit datapath, respectively. Taking into the account of dynamic energy savings due to the elimination of write '0' operations, our design saves nearly 20% of total energy.

Reducing energy consumption is an important issue for battery powered embedded computing systems. Content addressable memory (CAM)-based highly-associative caches (HAC) are widely used in low power embedded microprocessors. The CAM tag is costly in power, access time, and area. We have designed a low power highly associative cache (LPHAC) whose tag is partially implemented by using CAM, while the remaining tag is implemented by using SRAM. The experimental results from 10 MediaBench and all 26 SPEC2K benchmarks show the proposed LPHAC exhibits almost the identical miss rate as a traditional HAC. At the same time, it consumes 27% less per cache access power and 1.6% less area with faster access time.

Traditional level-one instruction caches and data caches for embedded systems typically have the same capacities. Configurable caches either shut down a part of the cache to suit applications needing a small cache or employ a large cache and high associativity for applications needing to reduce miss rate and energy. However, increasing associativity is energy-costly compared with increasing capacity. We have extended the traditional configurable cache and made the whole on-chip cache memory capacity available to both instruction and data caches. The capacity can then be co-allocated between the data and the instruction caches. Compared with way shutdown and way concatenation, the capacity co-allocation cache provides a better solution than increasing associativity. Four out of 17 benchmarks from Mibench benefit from the capacity co-allocation cache. Energy reduction can be up to 24%, with an average of 5%, compared to a traditional configurable cache.

Level one cache normally resides on a processor’s critical path, which determines the clock frequency. Directmapped caches exhibit fast access time but poor hit rates compared with same sized set-associative caches due to nonuniform accesses to the cache sets, which generate more conflict misses in some sets while other sets are underutilized. We propose a technique to reduce the miss rate of direct mapped caches through balancing the accesses to cache sets. We increase the decoder length and thus reduce the accesses to heavily used sets without dynamically detecting the cache set usage information. We introduce a replacement policy to direct-mapped cache design and increase the access to the underutilized cache sets with the help of programmable decoders. On average, the proposed balanced cache, or BCache, achieves 64.5% and 37.8% miss rate reductions on all 26 SPEC2K benchmarks for the instruction and data caches, respectively. This translates into an average IPC improvement of 5.9%. The B-Cache consumes 10.5% more power per access but exhibits 2% total memory access related energy saving due to the miss rate reductions and hence the reduction to applications’ execution time. Compared with previous techniques that aim at reducing the miss rate of direct-mapped caches, our technique requires only one cycle to access all cache hits and has the same access time of a direct-mapped cache.

Caches contribute to much of a microprocessor system's power and energy consumption. Numerous new cache architectures, such as phased, pseudo-set-associative, way predicting, reactive-associative, way-shutdown, way-concatenating, and highly-associative, are intended to reduce power and/or energy, but they all impose some performance overhead. We have developed a new cache architecture, called a way-halting cache, that reduces energy further than previously mentioned architectures, while imposing no performance overhead. Our way-halting cache is a four-way set-associative cache that stores the four lowest-order bits of all ways' tags into a fully associative memory, which we call the halt tag array. The lookup in the halt tag array is done in parallel with, and is no slower than, the set-index decoding. The halt tag array predetermines which tags cannot match due to their low-order 4 bits mismatching. Further accesses to ways with known mismatching tags are then halted, thus saving power. Our halt tag array has an additional feature of using static logic only, rather than dynamic logic used in highly associative caches, making our cache simpler to design with existing tools. We provide data from experiments on 29 benchmarks drawn from Powerstone, Mediabench, and Spec 2000, based on our layouts in 0.18 micron CMOS technology. On average, we obtained 55% savings of memory-access related energy over a conventional four-way set-associative cache. We show that savings are greater than previous methods, and nearly twice that of highly associative caches, while imposing no performance overhead and only 2% cache area overhead.

Energy consumption is a major concern in many embedded computing systems. Several studies have shown that cache memories account for about 50% of the total energy consumed in these systems. The performance of a given cache architecture is determined, to a large degree, by the behavior of the application executing on the architecture. Desktop systems have to accommodate a very wide range of applications and therefore the cache architecture is usually set by the manufacturer as a best compromise given current applications, technology, and cost. Unlike desktop systems, embedded systems are designed to run a small range of well-defined applications. In this context, a cache architecture that is tuned for that narrow range of applications can have both increased performance as well as lower energy consumption. We introduce a novel cache architecture intended for embedded microprocessor platforms. The cache has three software-configurable parameters that can be tuned to particular applications. First, the cache's associativity can be configured to be direct-mapped, two-way, or four-way set-associative, using a novel technique we call way concatenation . Second, the cache's total size can be configured by shutting down ways. Finally, the cache's line size can be configured to have 16, 32, or 64 bytes. A study of 23 programs drawn from Powerstone, MediaBench, and Spec2000 benchmark suites shows that the configurable cache tuned to each program saved energy for every program compared to a conventional four-way set-associative cache as well as compared to a conventional direct-mapped cache, with an average savings of energy related to memory access of over 40%.

Customizing a memory hierarchy to a particular application or applications is becoming increasingly common in embedded system design, with one benefit being reduced energy. Adding a victim buffer to the memory hierarchy is known to reduce energy and improve performance on average, yet victim buffers are not typically found in commercial embedded processors. One problem with such buffers is, while they work well on average, they tend to hurt performance for many applications. We show that a victim buffer can be very effective if it is considered as a parameter in designing a memory hierarchy, like the traditional cache parameters of total size, associativity, and line size. We describe experiments on PowerStone and MediaBench benchmarks, showing that having the option of adding a victim buffer to a direct-mapped cache can reduce memory-access energy by a factor of 3 in some cases. Furthermore, even when other cache parameters are configurable, we show that a victim buffer can still reduce energy by 43%. By treating the victim buffer as a parameter, meaning the buffer can be included or excluded, we can avoid performance overhead of up to 4% on some examples. We discuss the victim buffer in the context of both core-based and pre-fabricated platform based design approaches.

Memory access can account for about half of a microprocessor system's power consumption. Customizing a microprocessor cache's total size, line size and associativity to a particular program is well known to have tremendous benefits for performance and power. Customizing caches has until recently been restricted to core-based flows, in which a new chip will be fabricated. However, several configurable cache architectures have been proposed recently for use in pre-fabricated microprocessor platforms. Tuning those caches to a program is still however a cumbersome task left for designers, assisted in part by recent computer-aided design (CAD) tuning aids. We propose to move that CAD on-chip, which can greatly increase the acceptance of configurable caches. We introduce on-chip hardware implementing an efficient cache tuning heuristic that can automatically, transparently, and dynamically tune the cache to an executing program. We carefully designed the heuristic to avoid any cache flushing, since flushing is power and performance costly. By simulating numerous Powerstone and MediaBench benchmarks, we show that such a dynamic self-tuning cache can reduce memory-access energy by 45% to 55% on average, and as much as 97%, compared with a four-way set-associative base cache, completely transparently to the programmer.

Static energy dissipation in cache memories will constitute an increasingly larger portion of total microprocessor energy dissipation due to nanoscale technology characteristics and the large size of on-chip caches. We propose to reduce the static energy dissipation of an on-chip data cache by taking advantage of the frequent values (FV) that widely exist in a data cache memory. The original FV-based low-power cache design aimed at only reducing dynamic power, at the cost of a 5% slowdown. We propose a better design that reduces both static and dynamic cache power, and that uses a circuit design that eliminates performance overhead. A designer can utilize our architecture by simulating an application and then synthesizing the FVs into an application-specific cache design when values will not change, or by simulating and then writing to an FV-cache with configuration registers when values could change. Furthermore, we describe hardware that can dynamically determine FVs and write to the configuration registers completely transparently. Experiments on 11 Spec 2000 benchmarks show that, in addition to the dynamic power savings, 33% static energy savings for data caches can be achieved.

Caches contribute to much of a microprocessor system's power and energy consumption. We have developed a new cache architecture, called a way-halting cache, that reduces energy while imposing no performance overhead. Our way-halting cache is a four-way set-associative cache that stores the four lowest-order bits of all ways' tags into a fully associative memory, which we call the halt tag array. The lookup in the halt tag array is done in parallel with, and is no slower than, the set-index decoding. The halt tag array pre-determines which tags cannot match due to their low-order four bits mismatching. Further accesses to ways with known mismatching tags are then halted, thus saving power. Our halt tag array has an additional feature of using static logic only, rather than dynamic logic used in highly associative caches. We provide data from experiments on 17 benchmarks drawn from MediaBench and Spec 2000, based on our layouts in 0.18 micron CMOS technology. On average, 55% savings of memory-access related energy were obtained over a conventional four-way set-associative cache. We show that energy savings are greater than previous methods, and nearly twice that of highly-associative caches, while imposing no performance overhead and only 2% cache area overhead.

Memory accesses often account for about half of a microprocessor system's power consumption. Customizing a microprocessor cache's total size, line size, and associativity to a particular program is well known to have tremendous benefits for performance and power. Customizing caches has until recently been restricted to core-based flows, in which a new chip will be fabricated. However, several configurable cache architectures have been proposed recently for use in prefabricated microprocessor platforms. Tuning those caches to a program is still, however, a cumbersome task left for designers, assisted in part by recent computer-aided design (CAD) tuning aids. We propose to move that CAD on-chip, which can greatly increase the acceptance of tunable caches. We introduce on-chip hardware implementing an efficient cache tuning heuristic that can automatically, transparently, and dynamically tune the cache to an executing program. Our heuristic seeks not only to reduce the number of configurations that must be examined, but also traverses the search space in a way that minimizes costly cache flushes. By simulating numerous Powerstone and MediaBench benchmarks, we show that such a dynamic self-tuning cache saves on average 40% of total memory access energy over a standard nontuned reference cache.

Since the I/O pins of a CPU are a significant source of energy consumption, work has been done on developing encoding schemes for reducing switching activity on external buses. Modest reductions in switching can be achieved for data and address buses using a number of general purpose encoding schemes. However, by exploiting the characteristic of memory reference locality, switching activity on the address bus can be reduced by as much as 66%. Till now no characteristic has been identified that can be used to achieve similar reductions in switching activity on the data bus. We have discovered a characteristic of values transmitted over the data bus according to which a small number of distinct values, called frequent values , account for 32% of transmissions over the external data bus. Exploiting this characteristic we have developed an encoding scheme that we call the FV encoding scheme. To implement this scheme we have also developed a technique for dynamically identifying the frequent values which compares quite favorably with an optimal offline algorithm. Our experiments show that FV encoding of 32 frequent values yields an average reduction of 30% (with on-chip data cache) and 49% (without on-chip data cache) in data bus switching activity for SPEC95 and mediabench programs. Moreover the reduction in switching achieved by FV encoding is 2 to 4 times the reduction achieved by the bus-invert coding scheme and 1.5 to 3 times the reduction achieved by the adaptive method . The overall energy savings on data bus we attained considering the coder overhead is 29%.

Energy consumption is a major concern in many embedded computing systems. Several studies have shown that cache memories account for about 50% of the total energy consumed in these systems. The performance of a given cache architecture is largely determined by the behavior of the application using that cache. Desktop systems have to accommodate a very wide range of applications and therefore the manufacturer usually sets the cache architecture as a compromise given current applications, technology and cost. Unlike desktop systems, embedded systems are designed to run a small range of well-defined applications. In this context, a cache architecture that is tuned for that narrow range of applications can have both increased performance as well as lower energy consumption. We introduce a novel cache architecture intended for embedded microprocessor platforms. The cache can be configured by software to be direct-mapped, two-way, or four-way set associative, using a technique we call way concatenation, having very little size or performance overhead. We show that the proposed cache architecture reduces energy caused by dynamic power compared to a way-shutdown cache. Furthermore, we extend the cache architecture to also support a way shutdown method designed to reduce the energy from static power that is increasing in importance in newer CMOS technologies. Our study of 23 programs drawn from Powerstone, MediaBench and Spec2000 show that tuning the cache's configuration saves energy for every program compared to conventional four-way set-associative as well as direct mapped caches, with average savings of 40% compared to a four-way conventional cache.