Multimodal Sentiment Analysis (MSA) leverages heterogeneous modalities, such as language, vision, and audio, to enhance the understanding of human sentiment. While existing models often focus on extracting shared information across modalities or directly fusing heterogeneous modalities, such approaches can introduce redundancy and conflicts due to equal treatment of all modalities and the mutual transfer of information between modality pairs. To address these issues, we propose a Disentangled-Language-Focused (DLF) multimodal representation learning framework, which incorporates a feature disentanglement module to separate modality-shared and modality-specific information. To further reduce redundancy and enhance language-targeted features, four geometric measures are introduced to refine the disentanglement process. A Language-Focused Attractor (LFA) is further developed to strengthen language representation by leveraging complementary modality-specific information through a language-guided cross-attention mechanism. The framework also employs hierarchical predictions to improve overall accuracy. Extensive experiments on two popular MSA datasets, CMU-MOSI and CMU-MOSEI, demonstrate the significant performance gains achieved by the proposed DLF framework. Comprehensive ablation studies further validate the effectiveness of the feature disentanglement module, language-focused attractor, and hierarchical predictions.

In the context of cloud computing, services are held on cloud servers, where the clients send their data to the server and obtain the results returned by server. However, the computation, data and results are prone to tampering due to the vulnerabilities on the server side. Thus, verifying the integrity of computation is important in the client-server setting. The cryptographic method known as Zero-Knowledge Proof (ZKP) is renowned for facilitating private and verifiable computing. ZKP allows the client to validate that the results from the server are computed correctly without violating the privacy of the server’s intellectual property. Zero-Knowledge Succinct NonInteractive Argument of Knowledge (zkSNARKs), in particular, has been widely applied in various applications like blockchain and verifiable machine learning. Despite their popularity, existing zkSNARKs approaches remain highly computationally intensive. For instance, even basic operations like matrix multiplication require an extensive number of constraints, resulting in significant overhead. In addressing this challenge, we introduce $z k V C$, which optimizes the ZKP computation for matrix multiplication, enabling rapid proof generation on the server side and efficient verification on the client side. zkVC integrates optimized ZKP modules, such as Constraint-reduced Polynomial Circuit (CRPC) and Prefix-Sum Query (PSQ), collectively yielding a more than $\mathbf{1 2}$-fold increase in proof speed over prior methods. The code is available at https://github.com/UCF-Lou-Lab-PET/zkformer.

Hardware/software co-design in the context of cyber-physical systems (CPS) [2] takes the form of co-designing (i) a control algorithm, and (ii) its software implementation on a distributed and heterogeneous architecture [5, 6]. In domains such as automotive CPS, this involves the design and implementation of multiple controllers on distributed automotive architectures consisting of different electronic control units (ECUs) and communication buses like CAN, FlexRay and automotive Ethernet [13]. Traditionally, following the principle of separation of concerns, control algorithms were designed independent of the implementation platform details. As a result, they made certain assumptions on delays experienced by control signals and assumed, for example, that all sensor inputs necessary for state estimation arrive at the same time. When trying to implement such controllers on an independently designed implementation platform, where many of the controller design or model-level assumptions are not satisfied, an iterative design process that involves testing and adjustments to the controllers and the implementation decisions became necessary. This led to the development of X-in-the-loop simulations, where X can be software and hardware at various stages of implementation, along with which the control algorithms are simulated [1, 14]. The aim is to ensure that the semantics or the behavior of the controller models (or algorithms) are preserved in the final implementation [12].

Chiplet technology has recently emerged as a promising solution to improving chip performance through the modularization of complex designs and communication facilitated by high-speed interchiplet serial links. However, the increasing on-package routing density and data rates of these links introduce complex signal and power integrity challenges, surpassing those encountered in traditional large monolithic chips. Addressing these complexities with efficient analysis and design tools is crucial for maintaining design robustness. In this article, we propose SPIRAL+: signal-power integrity co-analysis framework for high-speed interchiplet serial links validation. The framework employs machine learning (ML) to construct transmitter models and utilizes an impulse response extraction method for modeling the channel and receiver. It then performs signal-power integrity co-analysis through a novel double-edge response-based method, leveraging the developed equivalent models. Additionally, an efficient ML model is crafted to accurately predict eye diagram metrics. The analysis provides valuable insights for design optimization. Experimental results show that SPIRAL+ achieves eye diagrams with a mean relative error of 0.07%–7.47%, while realizing a speedup of$31\times $–$326\times $over traditional commercial tools.

FPGA-based edge servers are used in many applications in smart cities, hospitals, retail, etc. Equipped with heterogeneous FPGA-based accelerator cards, the servers can be implemented with multiple tasks, including efficient video prepossessing, machine learning algorithm acceleration, etc. These servers are required to implement inference during the daytime while retraining the model during the night to adapt to new environments, domains, or new users. During the retraining, conventionally, the incoming data are transmitted to the cloud, and then the updated machine learning models will be transferred back to the edge server. Such a process is inefficient and cannot protect users’ privacy, so it is desirable for the models to be directly trained on the edge servers. Deploying convolutional neural network (CNN) training on heterogeneous resource-constrained FPGAs is challenging since it needs to consider both the complex data dependency of the training process and the communication bottleneck among different FPGAs. Previous multiaccelerator training algorithms select optimal scheduling strategies for data parallelism (DP), tensor parallelism (TP), and pipeline parallelism (PP). However, PP cannot deal with batch normalization (BN) which is an essential CNN operator, while purely applying DP and TP suffers from resource under-utilization and intensive communication costs. In this work, we propose MTrain, a novel multiaccelerator training scheduling strategy that transfers the training process into a multibranch workflow, thus independent suboperations of different branches are executed on different training accelerators in parallelism for better utilization and reduced communication overhead. Experimental results show that we can achieve efficient CNN training on heterogeneous FPGA-based edge servers with$1.07\times $–$2.21\times $speedup under 15-GB/s peer-to-peer bandwidth compared to the state-of-the-art work.

Recent rapid advancements in intelligent vehicular systems and deep learning techniques have led to the emergence of diverse applications utilizing high-quality automotive videos in the Internet-of-Vehicles (IoV), often assisted by uncrewed aerial vehicles (UAVs). These applications aim to provide convenience and security for users. However, transmitting automotive videos with high-quality and low-bit-rate poses a challenge due to the inherent lossiness of traditional compression codecs in current UAV-assisted IoV systems, thereby affecting the performance of subsequent tasks. To address this, we propose a spatial-temporal hybrid video compression framework (STHVC), which integrates Space-Time Super-Resolution (STSR) with conventional codecs to enhance the compression efficiency on automotive videos. In our hybrid design, the encoder generates a low-frame-rate and low-resolution version of the source video, which is then compressed using a traditional codec. During the decoding stage, an effective STSR network is developed to increase both the resolution and the frame rate, and mitigate compression artifacts for automotive videos simultaneously. Additionally, we introduce a rectified intermediate flow estimation technique (RecIFE) within the proposed STSR network to address the challenge of noisy and inaccurate motions during the compression pipeline. Extensive experiments on various benchmark datasets demonstrate that our approach achieves bit-rate reductions of 29.97% compared to H.265 (slow) and 31.27% compared to H.266, while also exhibiting superior restoration performance compared to other state-of-the-art learning-based approaches.

Chiplet has recently emerged as a promising solution to achieving further performance improvements by breaking down complex processors into modular components and communicating through high-speed inter-chiplet serial links. However, the ever-growing on-package routing density and data rates of such serial links inevitably lead to more complex and worse signal and power integrity issues than a large monolithic chip. This highly demands efficient analysis and validation tools to support robust design. In this paper, a signal-power integrity co-analysis framework for high-speed inter-chiplet serial links validation named SPIRAL is proposed. The framework first builds equivalent models for the links with a machine learning-based transmitter model and an impulse response based model for the channel and receiver: Then, the signal-power integrity is co-analyzed with a pulse response based method using the equivalent models. Experimental results show that SPIRAL yields eye diagrams with 0.82-1.85% mean relative error, while achieving $18-44 \times$ speedup compared to a commercial SPICE.

After a large language model (LLM) is deployed on edge devices, it is desirable for these devices to learn from user-generated conversation data to generate user-specific and personalized responses in real-time. However, user-generated data usually contains sensitive and private information, and uploading such data to the cloud for annotation is not preferred if not prohibited. While it is possible to obtain annotation locally by directly asking users to provide preferred responses, such annotations have to be sparse to not affect user experience. In addition, the storage of edge devices is usually too limited to enable large-scale fine-tuning with full user-generated data. It remains an open question how to enable on-device LLM personalization, considering sparse annotation and limited on-device storage. In this paper, we propose a novel framework to select and store the most representative data online in a self-supervised way. Such data has a small memory footprint and allows infrequent requests of user annotations for further fine-tuning. To enhance fine-tuning quality, multiple semantically similar pairs of question texts and expected responses are generated using the LLM. Our experiments show that the proposed framework achieves the best user-specific content-generating capability (accuracy) and fine-tuning speed (performance) compared with vanilla baselines. To the best of our knowledge, this is the very first on-device LLM personalization framework.

With the increase in the computation intensity of the chip, the mismatch between computation layer shapes and the available computation resource significantly limits the utilization of the chip. Driven by this observation, prior works discuss spatial accelerators or dataflow architecture to maximize the throughput. However, using spatial accelerators could potentially increase the execution latency. In this work, we first systematically investigate two execution models: (1) sequentially (temporally) launch one monolithic accelerator, and (2) spatially launch multiple accelerators. From the observations, we find that there is a latency throughput tradeoff between these two execution models, and combining these two strategies together can give us a more efficient latency throughput Pareto front. To achieve this, we propose spatial sequential architecture (SSR) and SSR design automation framework to explore both strategies together when deploying deep learning inference. We use the 7nm AMD Versal ACAP VCK190 board to implement SSR accelerators for four end-to-end transformer-based deep learning models. SSR achieves average throughput gains of 2.53x, 35.71x, and 14.20x under different batch sizes compared to the 8nm Nvidia GPU A10G, 16nm AMD FPGAs ZCU102, and U250. The average energy efficiency gains are 8.51x, 6.75x, and 21.22x, respectively. Compared with the sequential-only solution and spatial-only solution on VCK190, our spatial-sequential-hybrid solutions achieve higher throughput under the same latency requirement and lower latency under the same throughput requirement. We also use SSR analytical models to demonstrate how to use SSR to optimize solutions on other computing platforms, e.g., 14nm Intel Stratix 10 NX.

While Vision Transformers (ViTs) have shown consistent progress in computer vision, deploying them for real-time decision-making scenarios (< 1 ms) is challenging. Current computing platforms like CPUs, GPUs, or FPGA-based solutions struggle to meet this deterministic low-latency real-time requirement, even with quantized ViT models. Some approaches use pruning or sparsity to reduce model size and latency, but this often results in accuracy loss. To address the aforementioned constraints, in this work, we propose EQ-ViT, an end-to-end acceleration framework with novel algorithm and architecture co-design features to enable real-time ViT acceleration on AMD Versal Adaptive Compute Acceleration Platform (ACAP). The contributions are four-fold. First, we perform in-depth kernel-level performance profiling & analysis and explain the bottlenecks for existing acceleration solutions on GPU, FPGA, and ACAP. Second, on the hardware level, we introduce a new spatial and heterogeneous accelerator architecture, EQ-ViT architecture. This architecture leverages the heterogeneous features of ACAP, where both FPGA and artificial intelligence engines (AIEs) coexist on the same system-on-chip (SoC). Third, On the algorithm level, we create a comprehensive quantization-aware training strategy, EQ-ViT algorithm. This strategy concurrently quantizes both weights and activations into 8-bit integers, aiming to improve accuracy rather than compromise it during quantization. Notably, the method also quantizes nonlinear functions for efficient hardware implementation. Fourth, we design EQ-ViT automation framework to implement the EQ-ViT architecture for four different ViT applications on the AMD Versal ACAP VCK190 board, achieving accuracy improvement with 2.4%, and average speedups of 315.0x, 3.39x, 3.38x, 14.92x, 59.5x, 13.1x over computing solutions of Intel Xeon 8375C vCPU, Nvidia A10G, A100, Jetson AGX Orin GPUs, and AMD ZCU102, U250 FPGAs. The energy efficiency gains are 62.2x, 15.33x, 12.82x, 13.31x, 13.5x, 21.9x.

Federated learning (FL) has been widely adopted in IoT-enabled health monitoring on biosignals thanks to its advantages in data privacy preservation. However, the global model trained from FL generally performs unevenly across subjects since biosignal data is inherent with complex temporal dynamics. The morphological characteristics of biosignals with the same label can vary significantly among different subjects (i.e., inter-subject variability) while biosignals with varied temporal patterns can be collected on the same subject (i.e., intra-subject variability). To address the challenges, we present the Personalized Meta-Federated learning (PMFed) framework for personalized IoT-enabled health monitoring. Specifically, in the federated learning stage, a novel momentum-based model aggregating strategy is introduced to aggregate clients' models based on domain similarity in the meta-federated learning paradigm to obtain a well-generalized global model while speeding up the convergence. In the model personalizing stage, an adaptive model personalization mechanism is devised to adaptively tailor the global model based on the subject-specific biosignal features while preserving the learned cross-subject representations. We develop an IoT-enabled computing framework to evaluate the effectiveness of PMFed over three real-world health monitoring tasks. Experimental results show that the PMFed excels at detection performances in terms of F1 and accuracy by up to 9.4% and 8.7%, and reduces training overhead and throughput by up to 56.3% and 63.4% when compared with the SOTA federated learning algorithms.

DNNs are rapidly evolving from streamlined single-modality single-task (SMST) to multi-modality multi-task (MMMT) with large variations for different layers and complex data dependencies among layers. To support such models, hardware systems also evolved to be heterogeneous. The heterogeneous system comes from the prevailing trend to integrate diverse accelerators into the system for lower latency. FPGAs have high computation density and communication bandwidth and are configurable to be deployed with different designs of accelerators, which are widely used for various machine-learning applications. However, scaling from SMST to MMMT on heterogeneous FPGAs is challenging since MMMT has much larger layer variations, a massive number of layers, and complex data dependency among different backbones. Previous mapping algorithms are either inefficient or over-simplified which makes them impractical in general scenarios. In this work, we propose CHEF to enable efficient implementation of MMMT models in realistic heterogeneous FPGA clusters, i.e. deploying heterogeneous accelerators on heterogeneous FPGAs (A2F) and mapping the heterogeneous DNNs on the deployed heterogeneous accelerators (M2A). We propose CHEF-A2F, a two-stage accelerators-to-FPGAs deployment approach to co-optimize hardware deployment and accelerator mapping. In addition, we propose CHEF-M2A, which can support general and practical cases compared to previous mapping algorithms. To the best of our knowledge, this is the first attempt to implement MMMT models in real heterogeneous FPGA clusters. Experimental results show that the latency obtained with CHEF is near-optimal while the search time is 10000X less than exhaustively searching the optimal solution.

With the development of embedded systems and deep learning, it is feasible to combine them for offering various and convenient human-centered services, which is based on high-quality (HQ) videos. However, due to the limit of video traffic load and unavoidable noise, the visual quality of an image from an edge camera may degrade significantly, influencing the overall video and service quality. To maintain video stability, video quality enhancement (QE), aiming at recovering HQ videos from their distorted low-quality (LQ) sources, has aroused increasing attention in recent years. The key challenge for video QE lies in how to effectively aggregate complementary information from multiple frames (i.e., temporal fusion). To handle diverse motion in videos, existing methods commonly apply motion compensation before the temporal fusion. However, the motion field estimated from the distorted LQ video tends to be inaccurate and unreliable, thereby resulting in ineffective fusion and restoration. In addition, motion estimation for consecutive frames is generally conducted in a pairwise manner, which leads to expensive and inefficient computation. In this article, we propose a fast yet effective temporal fusion scheme for video QE by incorporating a novel Spatio-Temporal Deformable Convolution (STDC) to simultaneously compensate motion and aggregate temporal information. Specifically, the proposed temporal fusion scheme takes a target frame along with its adjacent reference frames as input to jointly estimate an offset field to deform the spatio-temporal sampling positions of convolution. As a result, complementary information from multiple frames can be fused within the STDC operation in one forward pass. Extensive experimental results on three benchmark datasets show that our method performs favorably to the state of the art in terms of accuracy and efficiency.

Dense matrix multiply (MM) serves as one of the most heavily used kernels in deep learning applications. To cope with the high computation demands of these applications, heterogeneous architectures featuring both FPGA and dedicated ASIC accelerators have emerged as promising platforms. For example, the AMD/Xilinx Versal ACAP architecture combines general-purpose CPU cores and programmable logic with AI Engine processors optimized for AI/ML. An array of 400 AI Engine processors executing at 1 GHz can provide up to 6.4 TFLOPS performance for 32-bit floating-point (FP32) data. However, machine learning models often contain both large and small MM operations. While large MM operations can be parallelized efficiently across many cores, small MM operations typically cannot. We observe that executing some small MM layers from the BERT natural language processing model on a large, monolithic MM accelerator in Versal ACAP achieved less than 5% of the theoretical peak performance. Therefore, one key question arises: How can we design accelerators to fully use the abundant computation resources under limited communication bandwidth for end-to-end applications with multiple MM layers of diverse sizes? We identify the biggest system throughput bottleneck resulting from the mismatch between the massive computation resources of one monolithic accelerator and the various MM layers of small sizes in the application. To resolve this problem, we propose the CHARM framework to compose multiple diverse MM accelerator architectures working concurrently on different layers within one application. CHARM includes analytical models that guide design space exploration to determine accelerator partitions and layer scheduling. To facilitate system designs, CHARM automatically generates code, enabling thorough onboard design verification. We deploy the CHARM framework on four different deep learning applications in FP32, INT16, and INT8 data types, including BERT, ViT, NCF, and MLP, on the AMD/Xilinx Versal ACAP VCK190 evaluation board. Our experiments show that we achieve 1.46 TFLOPS, 1.61 TFLOPS, 1.74 TFLOPS, and 2.94 TFLOPS inference throughput for BERT, ViT, NCF, and MLP in FP32 data type, respectively, which obtain 5.29 \(\times\) , 32.51 \(\times\) , 1.00 \(\times\) , and 1.00 \(\times\) throughput gains compared to one monolithic accelerator. CHARM achieves the maximum throughput of 1.91 TOPS, 1.18 TOPS, 4.06 TOPS, and 5.81 TOPS in the INT16 data type for the four applications. The maximum throughput achieved by CHARM in the INT8 data type is 3.65 TOPS, 1.28 TOPS, 10.19 TOPS, and 21.58 TOPS, respectively. We have open-sourced our tools, including detailed step-by-step guides to reproduce all the results presented in this article and to enable other users to learn and leverage CHARM framework and tools in their end-to-end systems: https://github.com/arc-research-lab/CHARM .

Contrastive learning (CL), a self-supervised learning approach, can effectively learn visual representations from unlabeled data. Given the CL training data, generative models can be trained to generate synthetic data to supplement the real data. Using both synthetic and real data for CL training has the potential to improve the quality of learned representations. However, synthetic data usually has lower quality than real data, and using synthetic data may not improve CL compared with using real data. To tackle this problem, we propose a data generation framework with two methods to improve CL training by joint sample generation and contrastive learning. The first approach generates hard samples for the main model. The generator is jointly learned with the main model to dynamically customize hard samples based on the training state of the main model. Besides, a pair of data generators are proposed to generate similar but distinct samples as positive pairs. In joint learning, the hardness of a positive pair is progressively increased by decreasing their similarity. Experimental results on multiple datasets show superior accuracy and data efficiency of the proposed data generation methods applied to CL. For example, about 4.0%, 3.5%, and 2.6% accuracy improvements for linear classification are observed on ImageNet-100, CIFAR-100, and CIFAR-10, respectively. Besides, up to 2× data efficiency for linear classification and up to 5× data efficiency for transfer learning are achieved.

Quantization has become an essential technique in compressing deep neural networks for deployment onto resource-constrained hardware. It is noticed that, the hardware efficiency of implementing quantized networks is highly coupled with the actual values to be quantized into, and therefore, with given bit widths, we can smartly choose a value space to further boost the hardware efficiency. For example, using weights of only integer powers of two, multiplication can be fulfilled by bit operations. Under such circumstances, however, existing quantization-aware training methods are either not suitable to apply or unable to unleash the expressiveness of very low bit-widths. For the best hardware efficiency, we revisit the quantization of convolutional neural networks and propose to address the training process from a weight-searching angle, as opposed to optimizing the quantizer functions as in existing works. Extensive experiments on CIFAR10 and ImageNet classification tasks are examined with implementations onto well-established CNN architectures, such as ResNet, VGG, and MobileNet, etc. It is shown the proposed method can achieve a lower accuracy loss than the state of arts, and/or improving implementation efficiency by using hardware-friendly weight values at the same time.

Model fairness (a.k.a., bias) has become one of the most critical problems in a wide range of AI applications. An unfair model in autonomous driving may cause a traffic accident if corner cases (e.g., extreme weather) cannot be fairly regarded; or it will incur healthcare disparities if the AI model misdiagnoses a certain group of people (e.g., brown and black skin). In recent years, there are emerging research works on addressing unfairness, and they mainly focus on a single unfair attribute, like skin tone; however, real-world data commonly have multiple attributes, among which unfairness can exist in more than one attribute, called "multi-dimensional fairness". In this paper, we first reveal a strong correlation between the different unfair attributes, i.e., optimizing fairness on one attribute will lead to the collapse of others. Then, we propose a novel Multi-Dimension Fairness framework, namely Muffin, which includes an automatic tool to unite off-the-shelf models to improve the fairness on multiple attributes simultaneously. Case studies on dermatology datasets with two unfair attributes show that the existing approach can achieve 21.05% fairness improvement on the first attribute while it makes the second attribute unfair by 1.85%. On the other hand, the proposed Muffin can unite multiple models to achieve simultaneously 26.32% and 20.37% fairness improvement on both attributes; meanwhile, it obtains 5.58% accuracy gain.

Dense matrix multiply (MM) serves as one of the most heavily used kernels in deep learning applications. To cope with the high computation demands of these applications, heterogeneous architectures featuring both FPGA and dedicated ASIC accelerators have emerged as promising platforms. For example, the AMD/Xilinx Versal ACAP architecture combines general-purpose CPU cores and programmable logic (PL) with AI Engine processors (AIE) optimized for AI/ML. An array of 400 AI Engine processors executing at 1 GHz can theoretically provide up to 6.4 TFLOPs performance for 32-bit floating-point (fp32) data. However, machine learning models often contain both large and small MM operations. While large MM operations can be parallelized efficiently across many cores, small MM operations typically cannot. In our investigation, we observe that executing some small MM layers from the BERT natural language processing model on a large, monolithic MM accelerator in Versal ACAP achieved less than 5% of the theoretical peak performance. Therefore, one key question arises: How can we design accelerators to fully use the abundant computation resources under limited communication bandwidth for end-to-end applications with multiple MM layers of diverse sizes?

Spintronic domain-wall memories (DWMs) offer improved memory density and energy compared to conventional memories, but are susceptible to shifting faults. We propose PIETT (Pinning,Insertion,Erasure, andTranslation-faultTolerance) for improved misalignment correction versus the state of the art. PIETT proposes a derived error correction combined with multi-domain access approach to detect and correct a minimum of three misalignment faults after an arbitrary shift distance. Moreover, the rate of both misalignment and pinning faults are characterized in DWM nanowires, demonstrating that pinning faults are a significant concern to DWM. As such, PIETT is the first method to combine correction of misalignment and pinning faults in random access DWMs. It also introduces novel PIETT Transverse Access Points (TAPs), which utilize a novel write access mode that can set/reset multiple domains in a single intrinsic operation and can store shift distance detection codes. By allowing checks between shifts of the intrinsic shift distance (e.g., 3 domains), using a single TAP per nanowire expands misalignment protection and determines the needed corrective shifts to correct faults in all nanowires. Two TAPs expands misalignment protection to correct misalignment by more than one position and detects pinning by detecting different shift distances at each extremity of the nanowire. PIETT leverages knowledge of pinned nanowire locations to guide a modified SECDED ECC with one additional parity bit stored in additional parity nanowires. Thus, PIETT in TAP mode can correct unlimited, potentially multi-position, misalignment faults and either up to three pinning faults or up to two pinning faults with up to one bit flip fault using scrubbing. PIETT provides 8 to 21 orders of magnitude improvement in mean-time-to-failure with similar or better area overhead and only a 1% system performance degradation compared to state of the art DWM misalignment correction.

The ubiquity of edge devices has led to a growing amount of unlabeled data produced at the edge. Deep learning models deployed on edge devices are required to learn from these unlabeled data to continuously improve accuracy. Self-supervised representation learning has achieved promising performances using centralized unlabeled data. However, the increasing awareness of privacy protection limits centralizing the distributed unlabeled image data on edge devices. While federated learning has been widely adopted to enable distributed machine learning with privacy preservation, without a data selection method to efficiently select streaming data, the traditional federated learning framework fails to handle these huge amounts of decentralized unlabeled data with limited storage resources on edge. To address these challenges, we propose a self-supervised on-device federated learning framework with coreset selection, which we call SOFed, to automatically select a coreset that consists of the most representative samples into the replay buffer on each device. It preserves data privacy as each client does not share raw data while learning good visual representations. Experiments demonstrate the effectiveness and significance of the proposed method in visual representation learning.

Intermittent systems powered by ambient energy harvesting are becoming popular for the benefits of an infinite lifetime and minimum maintenance requirements. Nonvolatile processors (NVPs) enable continual task executions under an unstable power supply with an efficient reactive checkpointing strategy. However, recovering concurrent peripherals in an intermittent system may incur significant overhead once power failures take place, and the recovery of interrupts also lacks discussion in existing works. Noticing the different optimization directions between responsive checkpointing within NVPs and proactive checkpointing required to recover concurrent peripherals, this paper proposes REMARK, an NVP architecture enabling hybrid checkpointing and efficient peripheral recovery. REMARK expands the current NVP structure with a hybrid backup/restore module, a peripheral handler and an interrupt handler, which addresses the recovery problem of both peripherals and interrupts efficiently. A REMARK chip is fabricated to verify the proposed architecture. Results show that the execution efficiency is improved by$13\times $compared with the state-of-the-art. With programming optimization, another 36.5% performance improvement can be achieved.

Along with the progress of AI democratization, neural networks are being deployed more frequently in edge devices for a wide range of applications. Fairness concerns gradually emerge in many applications, such as face recognition and mobile medical. One fundamental question arises: what will be the fairest neural architecture for edge devices? By examining the existing neural networks, we observe that larger networks typically are fairer. But, edge devices call for smaller neural architectures to meet hardware specifications. To address this challenge, this work proposes a novel Fairness- and Hardware-aware Neural architecture search framework, namely FaHaNa. Coupled with a model freezing approach, FaHaNa can efficiently search for neural networks with balanced fairness and accuracy, while guaranteed to meet hardware specifications. Results show that FaHaNa can identify a series of neural networks with higher fairness and accuracy on a dermatology dataset. Target edge devices, FaHaNa finds a neural architecture with slightly higher accuracy, 5.28X smaller size, 15.14% higher fairness score, compared with MobileNetV2; meanwhile, on Raspberry PI and Odroid XU-4, it achieves 5.75X and 5.79X speedup.

The complex nature of real-world problems calls for heterogeneity in both machine learning (ML) models and hardware systems. The heterogeneity in ML models comes from multi-sensor perceiving and multi-task learning, i.e., multi-modality multi-task (MMMT), resulting in diverse deep neural network (DNN) layers and computation patterns. The heterogeneity in systems comes from diverse processing components, as it becomes the prevailing method to integrate multiple dedicated accelerators into one system. Therefore, a new problem emerges: heterogeneous model to heterogeneous system mapping (H2H). While previous mapping algorithms mostly focus on efficient computations, in this work, we argue that it is indispensable to consider computation and communication simultaneously for better system efficiency. We propose a novel H2H mapping algorithm with both computation and communication awareness; by slightly trading computation for communication, the system overall latency and energy consumption can be largely reduced. The superior performance of our work is evaluated based on MAESTRO modeling, demonstrating 15%-74% latency reduction and 23%-64% energy reduction compared with existing computation-prioritized mapping algorithms. Code is publicly available at https://github.com/xyzxinyizhang/H2H.

Energy-harvesting wireless sensor nodes have found widespread adoption due to their low cost and small form factor. However, uncertainty in the available power supply introduces significant challenges in engineering communications between intermittently-powered nodes. We propose a constraint-based model for energy harvests that together with a hardware model can be used to enable consistent, opportunistic communication with worst-case latency guarantees. We show that greedy approaches that attempt communication whenever energy is available lead to prolonged latencies in real-world environments. Our approach offers bounded worst-case latency while providing a performance improvement over a conservative, offline approach planned around the worst-case energy harvest.

No abstract available.

No abstract available.

Collaborative learning enables distributed clients to learn a shared model for prediction while keeping the training data local on each client. However, existing collaborative learning methods require fully-labeled data for training, which is inconvenient or sometimes infeasible to obtain due to the high labeling cost and the requirement of expertise. The lack of labels makes collaborative learning impractical in many realistic settings. Self-supervised learning can address this challenge by learning from unlabeled data. Contrastive learning (CL), a self-supervised learning approach, can effectively learn visual representations from unlabeled image data. However, the distributed data collected on clients are usually not independent and identically distributed (non-IID) among clients, and each client may only have few classes of data, which degrades the performance of CL and learned representations. To tackle this problem, we propose a collaborative contrastive learning framework consisting of two approaches: feature fusion and neighborhood matching, by which a unified feature space among clients is learned for better data representations. Feature fusion provides remote features as accurate contrastive information to each client for better local learning. Neighborhood matching further aligns each client’s local features to the remote features such that well-clustered features among clients can be learned. Extensive experiments show the effectiveness of the proposed framework. It outperforms other methods by 11% on IID data and matches the performance of centralized learning.

The growth in data needs of modern applications has created significant challenges for modern systems leading to a “memory wall.” Spintronic Domain-Wall Memory (DWM), provides near-SRAM read/write performance, energy savings and non-volatility, potential for extremely high storage density, and does not have significant endurance limitations. However, DWM’s benefits cannot directly address data access latency and throughput limitations of memory bus bandwidth. Processing-inmemory (PIM) is a popular solution to reduce the demands of memory-to-processor communication by offloading computation directly to the memory. PIM has been proposed in multiple technologies including DRAM, Phase-change memory (PCM), resistive memory (ReRAM), and Spin-Transfer Torque Memory (STT-MRAM). DRAM PIM provides solutions for a restricted set of two operand bulk-bitwise operations. PIM in PCM and ReRAM raise concerns about their effective endurance and PIM in STT-MRAM has insufficient density for main-memory applications. We propose CORUSCANT, a DWM-based in-memory computing solution that leverages the properties of DWM nanowires and allows them to serve as polymorphic gates. While normally DWM is accessed by applying spin polarized currents orthogonal to the nanowire at access points to read individual bits, transverse access along the DWM nanowire allows the differentiation of the aggregate resistance of multiple bits in the nanowire, akin to a multi-level cell. CORUSCANT leverages this transverse reading to directly provide multi-operand bulk-bitwise logic. Leveraging this multi-operand concept enabled by transverse access, CORUSCANT provides techniques to conduct multi-operand addition and two operand multiplication much more efficiently than prior digital PIM solutions. CORUSCANT provides a 1.6 × speedup compared to the leading DRAM PIM technique for query applications that leverage bulk bitwise operations. Compared to the leading PIM technique for DWM, CORUSCANT improves performance by 6.9 ×, 2.3 × and energy by 5.5 ×, 3.4 × for 8-bit addition and multiplication, respectively. For arithmetic heavy benchmarks, CORUSCANT reduces access latency by 2.1 ×, while decreasing energy consumption by 25.2 × for a 10% area overhead versus non-PIM DWM.

The Internet of Things (IoT) has been widely applied in personal health monitoring on biosignals. Conventional detection methods in the field count on a variety of heuristic criteria by utilizing extracted features, which are carefully selected through extensive clinical trials and experts’ experiences. Recently, deep learning (DL) gains rapidly growing attention in health monitoring. The most significant advantage of DL-based methods is that DL could execute feature engineering automatically with only labeled data, which results in a great reduction in the expertise involved and manual works in the detection method’s design. However, individual differences among various patients (subjects) can lead to accuracy degradation of the pretrained deep model. Simply fine-tuning the deep model with the patient-specific data cannot alleviate the problem since the pretrained model may not generalize well to new data. To address the problem, we propose a metalearning-based personalization method to generate the personalized neural network for each patient to conduct patient-specific detection. Specifically, the proposed metalearning method leverages a novel patientwise training tasks formatting strategy to train the neural network that ends up with a well-generalized model initialization containing across-patient knowledge. The well-generalized model initialization would then be utilized to perform a quick adaptation to the specific patient’s data domain. In this way, a new patient could be immediately assigned with a personalized neural network using limited labeled data. Experimental results show that the proposed metalearning-based personalization method achieves 8.2%, 2.5%, and 6.4% higher accuracy when compared with the existing DL detection methods in VF detection, AF detection, and human activity recognition, respectively.

Detecting actions in videos have been widely applied in on-device applications, such as cars, robots, etc. Practical on-device videos are always untrimmed with both action and background. It is desirable for a model to both recognize the class of action and localize the temporal position where the action happens. Such a task is called temporal action location (TAL), which is always trained on the cloud where multiple untrimmed videos are collected and labeled. It is desirable for a TAL model to continuously and locally learn from new data, which can directly improve the action detection precision while protecting customers’ privacy. However, directly training a TAL model on the device is nontrivial. To train a TAL model which can precisely recognize and localize each action, tremendous video samples with temporal annotations are required. However, annotating videos frame by frame is exorbitantly time consuming and expensive. Although weakly supervised temporal action localization (W-TAL) has been proposed to learn from untrimmed videos with only video-level labels, such an approach is also not suitable for on-device learning scenarios. In practical on-device learning applications, data are collected in streaming. For example, the camera on the device keeps collecting video frames for hours or days, and the actions of nearly all classes are included in a single long video stream. Dividing such a long video stream into multiple video segments requires lots of human effort, which hinders the exploration of applying the TAL tasks to realistic on-device learning applications. To enable W-TAL models to learn from a long, untrimmed streaming video, we propose an efficient video learning approach that can directly adapt to new environments. We first propose a self-adaptive video dividing approach with a contrast score-based segment merging approach to convert the video stream into multiple segments. Then, we explore different sampling strategies on the TAL tasks to request as few labels as possible. To the best of our knowledge, we are the first attempt to directly learn from the on-device, long video stream. Experimental results on the THUMOS’14 dataset show that the performance of our approach is comparable to the current W-TAL state-of-the-art (SOTA) work without any laborious manual video splitting.

Energy harvesting (EH)-powered sensor nodes can achieve theoretically unlimited lifetime by scavenging energy from ambient power sources, such as radio-frequency (RF) and kinetic energy. The nodes can collect and transmit data wirelessly with the harvested energy. However, the transmission between two sensor nodes is successful only when both nodes have enough energy at the same time. While the receiver can be actively listening, it may deplete the energy long before the sender has accumulated enough energy. Thus, given the scarce, unpredictable, and unevenly distributed energy among sensor nodes, it is challenging to ensure efficient data transmission between them. To address this challenge, we propose a sensor node architecture with multiple radios, each with different energy consumption on the sender and receiver. A node can be put into sleep when charged up and wakes up for communication when it infers that both nodes have enough energy based on its observations. What is more, two nodes can cooperatively and dynamically select different radios according to the stored energy and historical information to maximize the data throughput. To achieve cooperative communication adaptively, the communication procedure is modeled as a cooperative Markov game with partial observability on each node, and multiagent reinforcement learning (MARL) is employed to achieve the best results. Experimental results on hardware prototype and by simulation show that the proposed approaches achieve up to 89.1% of the optimal throughput and significantly outperform other online algorithms.

introduction Share on Introduction to the Special Issue on Artificial Intelligence and Cyber-Physical Systems - Part 2 Authors: Jingtong Hu University of Pittsburgh, Pittsburgh, PA, USA University of Pittsburgh, Pittsburgh, PA, USAView Profile , Qi Zhu Northwestern University, Evanston, IL, USA Northwestern University, Evanston, IL, USAView Profile , Susmit Jha SRI International, Menlo Park, CA, USA SRI International, Menlo Park, CA, USAView Profile Authors Info & Claims ACM Transactions on Cyber-Physical SystemsVolume 6Issue 2April 2022 Article No.: 10pp 1–3https://doi.org/10.1145/3517045Published:19 July 2022Publication History 0citation57DownloadsMetricsTotal Citations0Total Downloads57Last 12 Months57Last 6 weeks8 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Conventionally, DNN models are trained once in the cloud and deployed in edge devices such as cars, robots, or unmanned aerial vehicles (UAVs) for real-time inference. However, there are many cases that require the models to adapt to new environments, domains, or users. In order to realize such domain adaption or personalization, the models on devices need to be continuously trained on the device. In this work, we design EF-Train, an efficient DNN training accelerator with a unified channel-level parallelism-based convolution kernel that can achieve end-to-end training on resource-limited low-power edge-level FPGAs. It is challenging to implement on-device training on resource-limited FPGAs due to the low efficiency caused by different memory access patterns among forward and backward propagation and weight update. Therefore, we developed a data reshaping approach with intra-tile continuous memory allocation and weight reuse. An analytical model is established to automatically schedule computation and memory resources to achieve high energy efficiency on edge FPGAs. The experimental results show that our design achieves 46.99 GFLOPS and 6.09 GFLOPS/W in terms of throughput and energy efficiency, respectively.

Implementing a neural network (NN) inference in a millimeter-scale system is challenging due to limited energy and storage size. This article proposes an energy-aware adaptive NN inference implementation that utilizes one of two exits with different accuracies and computation options. The early-exit path provides a shorter processing time but less accuracy than the main-exit path. To compensate for the reduced accuracy, it additionally applies the main-exit path if the entropy of the early-exit inference is higher than a predetermined value. The NN is implemented with a custom low-power 180-nm CMOS processor chip and a 90-nm embedded flash memory chip and tested by the CIFAR-10 dataset. The measurement results show that the implemented convolutional NN (CNN) reduces processing time and thus energy consumption by 43.9% compared with a main-exit-only method while sacrificing its accuracy from 69.9% to 66.2%. Also, we explore the required minimum battery capacity at each optimal configuration for accuracy and/or energy consumption to achieve energy-autonomous operation under measured exemplary light profiles. It requires a minimum battery capacity of 855 mJ, acceptable for the target miniature system with two millimeter-scale batteries (684 mJ each). Compared with the state-of-the-art CNN technique (BranchyNet) allowing early stopping, the proposed design improves the accuracy by 0.7% and 3.3% to maintain energy-autonomous operation with two and one millimeter-scale batteries, respectively. Compared with the state-of-the-art lightweight CNN technique (MobileNet), this work provides flexibility with a tradeoff between accuracy and processing time for different application requirements.

This work aims to achieve intelligence on embedded devices by deploying deep neural networks (DNNs) onto resource-constrained microcontroller units (MCUs). Apart from the low frequency (e.g., 1-16 MHz) and limited storage (e.g., 16KB to 256KB ROM), one of the largest challenges is the limited RAM (e.g., 2KB to 64KB), which is needed to save the intermediate feature maps of a DNN. Most existing neural network compression algorithms aim to reduce the model size of DNNs so that they can fit into limited storage. However, they do not reduce the size of intermediate feature maps significantly, which is referred to as working memory and might exceed the capacity of RAM. Therefore, it is possible that DNNs cannot run in MCUs even after compression. To address this problem, this work proposes a technique to dynamically prune the activation values of the intermediate output feature maps in the runtime to ensure that they can fit into limited RAM. The results of our experiments show that this method could significantly reduce the working memory of DNNs to satisfy the hard constraint of RAM size, while maintaining satisfactory accuracy with relatively low overhead on memory and run-time latency.

Implantable Cardioverter Defibrillator (ICD) is an ultra-low-power device which monitors heart rate and delivers in-time defibrillation on detected ventricular arrhythmias (VAs). The parameters of VAs detection mechanism on each recipient’s ICD are supposed to be fine-tuned to obtain accurate detection due to the individual’s unique rhythm features. However, the process extremely relies on clinical expertise and thus must be conducted manually and routinely by cardiologists diagnosing massive amount of rhythm data. In this paper, we introduce a novel self-supervised on-device personalization of convolutional neural network (CNNs) for VAs detection. We first propose a computing framework consisting of an edge device and an ICD to enable efficient on-device CNNs personalization and real-time inference respectively. Then, we propose a generative model that learns to synthesize patient-specific intracardiac EGMs signals, which can then be used as personalized training data to improve patient-specific VAs detection performance on ICDs. Evaluations on three detection models show that the self-supervised on-device personalization significantly improve VAs detection performance under a patient-specific setting.

After a model is deployed on edge devices, it is desirable for these devices to learn from unlabeled data to continuously improve accuracy. Contrastive learning has demonstrated its great potential in learning from unlabeled data. However, the online input data are usually none independent and identically distributed (non-iid) and edge devices’ storages are usually too limited to store enough representative data from different data classes. We propose a framework to automatically select the most representative data from the unlabeled input stream, which only requires a small data buffer for dynamic learning. Experiments show that accuracy and learning speed are greatly improved.

Automatic myocardial segmentation of contrast echocardio-graphy has shown great potential in the quantification of myocardial perfusion parameters. Segmentation quality control is an important step to ensure the accuracy of segmentation results for quality research as well as its clinical application. Usually, the segmentation quality control happens after the data acquisition. At the data acquisition time, the operator could not know the quality of the segmentation results. On-the-fly segmentation quality control could help the operator to adjust the ultrasound probe or retake data if the quality is unsatisfied, which can greatly reduce the effort of time-consuming manual correction. However, it is infeasible to deploy state-of-the-art DNN-based models because the segmentation module and quality control module must fit in the limited hardware resource on the ultrasound machine while satisfying strict latency constraints. In this paper, we propose a hardware-aware neural architecture search framework for automatic myocardial segmentation and quality control of contrast echocardiography. We explicitly incorporate the hardware latency as a regularization term into the loss function during training. The proposed method searches the best neural network architecture for the segmentation module and quality prediction module with strict latency.

Deep learning models have been deployed in an increasing number of edge and mobile devices to provide healthcare. These models rely on training with a tremendous amount of labeled data to achieve high accuracy. However, for medical applications such as dermatological disease diagnosis, the private data collected by mobile dermatology assistants exist on distributed mobile devices of patients, and each device only has a limited amount of data. Directly learning from limited data greatly deteriorates the performance of learned models. Federated learning (FL) can train models by using data distributed on devices while keeping the data local for privacy. Existing works on FL assume all the data have ground-truth labels. However, medical data often comes without any accompanying labels since labeling requires expertise and results in prohibitively high labor costs. The recently developed self-supervised learning approach, contrastive learning (CL), can leverage the unlabeled data to pre-train a model for learning data representations, after which the learned model can be fine-tuned on limited labeled data to perform dermatological disease diagnosis. However, simply combining CL with FL as federated contrastive learning (FCL) will result in ineffective learning since CL requires diverse data for accurate learning but each device in FL only has limited data diversity. In this work, we propose an on-device FCL framework for dermatological disease diagnosis with limited labels. Features are shared among devices in the FCL pre-training process to provide diverse and accurate contrastive information without sharing raw data for privacy. After that, the pre-trained model is fine-tuned with local labeled data independently on each device or collaboratively with supervised federated learning on all devices. Experiments on dermatological disease datasets show that the proposed framework effectively improves the recall and precision of dermatological disease diagnosis compared with state-of-the-art methods.

Life-threatening ventricular arrhythmias (VAs) detection on intracardiac electrograms (IEGMs) is essential to Implantable Cardioverter Defibrillators (ICDs). However, current VAs detection methods count on a variety of heuristic detection criteria, and require frequent manual interventions to personalize criteria parameters for each patient to achieve accurate detection. In this work, we propose a one-dimensional convolutional neural network (1D-CNN) based life-threatening VAs detection on IEGMs. The network architecture is elaborately designed to satisfy the extreme resource constraints of the ICD while maintaining high detection accuracy. We further propose a meta-learning algorithm with a novel patient-wise training tasks formatting strategy to personalize the 1D-CNN. The algorithm generates a well-generalized model initialization containing across-patient knowledge, and performs a quick adaptation of the model to the specific patient's IEGMs. In this way, a new patient could be immediately assigned with personalized 1D-CNN model parameters using limited input data. Compared with the conventional VAs detection method, the proposed method achieves 2.2% increased sensitivity for detecting VAs rhythm and 8.6% increased specificity for non-VAs rhythm.

This paper describes a miniature edge device that performs neural network inference with different exit options depending on available energy. In addition to the main-exit path, it provides an alternative, early-exit path that requires less computation and thus increase the number of inference operations for given energy. To compensate its degraded accuracy, the proposed device provides entropy as a confidence level for the early exit. The network is implemented with a custom low-power 180 nm CMOS processor chip and a 90 nm embedded flash memory chip and tested by images from CIFAR-10 dataset. The measurement results show the proposed neural network reduces processing time and thus energy consumption by 41.3% compared with the main-exit only method while sacrificing its accuracy from 69.5% to 66.0%.

The 55th Design Automation Conference (DAC) held its first System Design Contest (SDC) in 2018. SDC'18 features a lower power object detection challenge (LPODC) on designing and implementing novel algorithms based object detection in images taken from unmanned aerial vehicles (UAV). The dataset includes 95 categories and 150k images, and the hardware platforms include Nvidia's TX2 and Xilinx's PYNQ Z1. DAC-SDC'18 attracted more than 110 entries from 12 countries. This paper presents in detail the dataset and evaluation procedure. It further discusses the methods developed by some of the entries as well as representative results. The paper concludes with directions for future improvements.

Co-exploration of neural architectures and hardware design is promising due to its capability to simultaneously optimize network accuracy and hardware efficiency. However, state-of-the-art neural architecture search algorithms for the co-exploration are dedicated for the conventional von-Neumann computing architecture, whose performance is heavily limited by the well-known memory wall. In this article, we are the first to bring the computing-in-memory architecture, which can easily transcend the memory wall, to interplay with the neural architecture search, aiming to find the most efficient neural architectures with high network accuracy and maximized hardware efficiency. Such a novel combination makes opportunities to boost performance, but also brings a bunch of challenges: The optimization space spans across multiple design layers from device type and circuit topology to neural architecture; and the presence of device variation may drastically degrade the neural network performance. To address these challenges, we propose a cross-layer exploration framework, namely NACIM, which jointly explores device, circuit and architecture design space and takes device variation into consideration to find the most robust neural architectures, coupled with the most efficient hardware design. Experimental results demonstrate that NACIM can find the robust neural network with 0.45 percent accuracy loss in the presence of device variation, compared with a 76.44 percent loss from the state-of-the-art NAS without consideration of variation; in addition, NACIM achieves an energy efficiency up to 16.3 TOPs/W, 3.17x higher than the state-of-the-art NAS.

introduction Share on Introduction to the Special Issue on Artificial Intelligence and Cyber-Physical Systems: Part 1 Authors: Jingtong Hu University of Pittsburgh, Pittsburgh, PA, USA University of Pittsburgh, Pittsburgh, PA, USAView Profile , Qi Zhu Northwestern University, Evanston, IL, USA Northwestern University, Evanston, IL, USAView Profile , Susmit Jha SRI International, Menlo Park, CA, USA SRI International, Menlo Park, CA, USAView Profile Authors Info & Claims ACM Transactions on Cyber-Physical SystemsVolume 5Issue 4October 2021 Article No.: 33pp 1–3https://doi.org/10.1145/3471164Online:22 September 2021Publication History 1citation77DownloadsMetricsTotal Citations1Total Downloads77Last 12 Months77Last 6 weeks8 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Atrial Fibrillation (AF), one of the most prevalent arrhythmias, is an irregular heart-rate rhythm causing serious health problems such as stroke and heart failure. Deep learning based methods have been exploited to provide an end-to-end AF detection by automatically extracting features from Electrocardiogram (ECG) signal and achieve state-of-the-art results. However, the pre-trained models cannot adapt to each patient’s rhythm due to the high variability of rhythm characteristics among different patients. Furthermore, the deep models are prone to overfitting when fine-tuned on the limited ECG of the specific patient for personalization. In this work, we propose a prior knowledge incorporated learning method to effectively personalize the model for patient-specific AF detection and alleviate the overfitting problems. To be more specific, a prior-incorporated portion importance mechanism is proposed to enforce the network to learn to focus on the targeted portion of the ECG, following the cardiologists’ domain knowledge in recognizing AF. A prior-incorporated regularization mechanism is further devised to alleviate model overfitting during personalization by regularizing the fine-tuning process with feature priors on typical AF rhythms of the general population. The proposed personalization method embeds the well-defined prior knowledge in diagnosing AF rhythm into the personalization procedure, which improves the personalized deep model and eliminates the workload of manually adjusting parameters in conventional AF detection method. The prior knowledge incorporated personalization is feasibly and semi-automatically conducted on the edge, device of the cardiac monitoring system. We report an average AF detection accuracy of 95.3% of three deep models over patients, surpassing the pre-trained model by a large margin of 11.5% and the fine-tuning strategy by 8.6%.

Multi-head self-attention (attention mechanism) has been employed in a variety of fields such as machine translation, language modeling, and image processing due to its superiority in feature extraction and sequential data analysis. This is benefited from a large number of parameters and sophisticated model architecture behind the attention mechanism. To efficiently deploy attention mechanism on resource-constrained devices, existing works propose to reduce the model size by building a customized smaller model or compressing a big standard model. A customized smaller model is usually optimized for the specific task and needs effort in model parameters exploration. Model compression reduces model size without hurting the model architecture robustness, which can be efficiently applied to different tasks. The compressed weights in the model are usually regularly shaped (e.g. rectangle) but the dimension sizes vary (e.g. differs in rectangle height and width). Such compressed attention mechanism can be efficiently deployed on CPU/GPU platforms as their memory and computing resources can be flexibly assigned with demand. However, for Field Programmable Gate Arrays (FPGAs), the data buffer allocation and computing kernel are fixed at run time to achieve maximum energy efficiency. After compression, weights are much smaller and different in size, which leads to inefficient utilization of FPGA on-chip buffer. Moreover, the different weight heights and widths may lead to inefficient FPGA computing kernel execution. Due to the large number of weights in the attention mechanism, building a unique buffer and computing kernel for each compressed weight on FPGA is not feasible. In this work, we jointly consider the compression impact on buffer allocation and the required computing kernel during the attention mechanism compressing. A novel structural pruning method with memory footprint awareness is proposed and the associated accelerator on FPGA is designed. The experimental results show that our work can compress Transformer (an attention mechanism based model) by 95x. The developed accelerator can fully utilize the FPGA resource, processing the sparse attention mechanism with the run-time throughput performance of 1.87 Tops in ZCU102 FPGA.

Hardware-aware Neural Architecture Search (NAS), which automatically finds an architecture that works best on a given hardware design, has prevailed in response to the ever-growing demand for real-time Artificial Intelligence (AI). However, in many situations, the underlying hardware is not pre-determined. We argue that simply assuming an arbitrary yet fixed hardware design will lead to inferior solutions, and it is best to co-explore neural architecture space and hardware design space for the best pair of neural architecture and hardware design. To demonstrate this, we employ Network-on-Chip (NoC) as the infrastructure and propose a novel framework, namely NANDS, to co-explore NAS space and NoC Design Search (NDS) space with the objective to maximize accuracy and throughput. Since two metrics are tightly coupled, we develop a multi-phase manager to guide NANDS to gradually converge to solutions with the best accuracy-throughput tradeoff. On top of it, we propose techniques to detect and alleviate timing performance bottleneck, which allows better and more efficient exploration of NDS space. Experimental results on common datasets, CIFAR10, CIFAR-100 and STL-10, show that compared with state-of-the-art hardware-aware NAS, NANDS can achieve 42.99% higher throughput along with 1.58% accuracy improvement. There are cases where hardware-aware NAS cannot find any feasible solutions while NANDS can.

This work aims to enable persistent, event-driven sensing and decision capabilities for energy-harvesting (EH)-powered devices by deploying lightweight DNNs onto EH-powered devices. However, harvested energy is usually weak and unpredictable and even lightweight DNNs take multiple power cycles to finish one inference. To eliminate the indefinite long wait to accumulate energy for one inference and to optimize the accuracy, we developed a power trace-aware and exit-guided network compression algorithm to compress and deploy multi-exit neural networks to EH-powered microcontrollers (MCUs) and select exits during execution according to available energy. The experimental results show superior accuracy and latency compared with state-of-the-art techniques.

There is growing interest in deploying energy harvesting processors and accelerators in Internet of Things (IoT). Energy harvesting harnesses the energy scavenged from the environment to power a system. Although it has many advantages over battery-operated systems such as lightweight, compact size, and no necessity of recharging and maintenance, it may suffer frequently power-down and a fluctuating power supply even with power on. Non-volatile processor (NVP) is a promising architecture for effective computing in energy harvesting scenarios. Recently, non-volatile accelerators (NVA) have been proposed to perform computations of deep learning algorithms. In this paper, we overview the recent studies of NVP and NVA across the layers of hardware, architecture, software and their co-design. Especially, we present the design insights of how the state-of-the-art works adapt their specific designs to the intermittent and fluctuating power conditions with the energy harvesting technology. Finally, we discuss recent trends using NVP and NVA in energy harvesting scenarios.

Life-threatening ventricular arrhythmias (VA) are the leading cause of sudden cardiac death (SCD), which is the most significant cause of natural death in the US [6]. The implantable cardioverter defibrillator (ICD) is a small device implanted to patients under high risk of SCD as a preventive treatment. The ICD continuously monitors the intracardiac rhythm and delivers shock when detecting the life-threatening VA. Traditional methods detect VA by setting criteria on the detected rhythm. However, those methods suffer from a high inappropriate shock rate and require a regular follow-up to optimize criteria parameters for each ICD recipient. To ameliorate the challenges, we propose the personalized computing framework for deep learning based VA detection on medical IoT systems. The system consists of intracardiac and surface rhythm monitors, and the cloud platform for data uploading, diagnosis, and CNN model personalization. We equip the system with real-time inference on both intracardiac and surface rhythm monitors. To improve the detection accuracy, we enable the monitors to detect VA collaboratively by proposing the cooperative inference. We also introduce the CNN personalization for each patient based on the computing framework to tackle the unlabeled and limited rhythm data problem. When compared with the traditional detection algorithm, the proposed method achieves comparable accuracy on VA rhythm detection and 6.6% reduction in inappropriate shock rate, while the average inference latency is kept at 71ms.

Real-time cardiac magnetic resonance imaging (MRI) plays an increasingly important role in guiding various cardiac interventions. In order to provide better visual assistance, the cine MRI frames need to be segmented on-the-fly to avoid noticeable visual lag. In addition, considering reliability and patient data privacy, the computation is preferably done on local hardware. State-of-the-art MRI segmentation methods mostly focus on accuracy only, and can hardly be adopted for real-time application or on local hardware. In this work, we present the first hardware-aware multi-scale neural architecture search (NAS) framework for real-time 3D cardiac cine MRI segmentation. The proposed framework incorporates a latency regularization term into the loss function to handle realtime constraints, with the consideration of underlying hardware. In addition, the formulation is fully differentiable with respect to the architecture parameters, so that stochastic gradient descent (SGD) can be used for optimization to reduce the computation cost while maintaining optimization quality. Experimental results on ACDC MICCAI 2017 dataset demonstrate that our hardware-aware multi-scale NAS framework can reduce the latency by up to 3.5× and satisfy the real-time constraints, while still achieving competitive segmentation accuracy, compared with the state-of-the-art NAS segmentation framework.

Recently, Long Short-Term Memory (LSTM), a type of recurrent neural network, has been widely employed in realtime applications, such as speech recognition, word segmentation, machine translation, etc. While existing works demonstrate that LSTM can be efficiently deployed in cloud platforms, the high communication latency between cloud and edge will drastically reduce its efficiency. Therefore, efficient LSTM accelerators at the edge are highly demanded. The limited resource in edge devices and the heterogeneous operations in LSTM (e.g., LSTM gates) bring challenges for the LSTM accelerator design. It seems straightforward to implement each operation as a specific hardware kernel. However, the data dependency among gates leads to significant running stalls in the existing heterogeneous-kernel accelerator, resulting in low parallelism and low resource utilization. To overcome the above challenges, this work proposes a novel generic LSTM accelerator design for Field-programmable Gate Array (FPGA) and Application-specific Integrated Circuit (ASIC) platforms, where two fundamental computing patterns (i.e., element-wise multiplication and addition) are incorporated in a unified computing kernel to execute operations in all LSTM gates simultaneously. Thus, the running stalls caused by heterogeneous kernels can be eliminated, achieving full parallelism in LSTM. The proposed technique and architecture are validated on Xilinx PYNQ-Z1 FPGA which can fully utilize the available resource, achieving 10x faster in inference time and 15.2x improvement in computing power efficiency compared with the state-of-the-art LSTM accelerator.

Energy harvesting is an attractive way to power future Internet of Things (IoT) devices since it can eliminate the need for battery or power cables. However, harvested energy is intrinsically unstable. While Field-programmable Gate Array (FPGAs) have been widely adopted in various embedded systems, it is hard to survive unstable power since all the memory components in FPGA are based on volatile Static Random-access Memory (SRAMs). The emerging non-volatile memory-based FPGAs provide promising potentials to keep configuration data on the chip during power outages. Few works have considered implementing efficient runtime intermediate data checkpoint on non-volatile FPGAs. To realize accumulative computation under intermittent power on FPGA, this article proposes a low-cost design framework, Data-Flow-Tracking FPGA (DFT-FPGA), which utilizes binary counters to track intermediate data flow. Instead of keeping all on-chip intermediate data, DFT-FPGA only targets on necessary data that is labeled by off-line analysis and identified by an online tracking system. The evaluation shows that compared with state-of-the-art techniques, DFT-FPGA can realize accumulative computing with less off-line workload and significantly reduce online roll-back time and resource utilization.

Hardware and neural architecture co-search that automatically generates artificial intelligence (AI) solutions from a given dataset are promising to promote AI democratization; however, the amount of time that is required by current co-search frameworks is in the order of hundreds of GPU hours for one target hardware. This inhibits the use of such frameworks on commodity hardware. The root cause of the low efficiency in existing co-search frameworks is the fact that they start from a “cold” state (i.e., search from scratch). In this article, we propose a novel framework, namely, HotNAS, that starts from a “hot” state based on a set of existing pretrained models (also known as model zoo) to avoid lengthy training time. As such, the search time can be reduced from 200 GPU hours to less than 3 GPU hours. In HotNAS, in addition to hardware design space and neural architecture search space, we further integrate a compression space to conduct model compressing during the co-search, which creates new opportunities to reduce latency, but also brings challenges. One of the key challenges is that all of the above search spaces are coupled with each other, e.g., compression may not work without hardware design support. To tackle this issue, HotNAS builds a chain of tools to design hardware to support compression, based on which a global optimizer is developed to automatically co-search all the involved search spaces. Experiments on ImageNet dataset and Xilinx FPGA show that, within the timing constraint of 5 ms, neural architectures generated by HotNAS can achieve up to 5.79% Top-1 and 3.97% Top-5 accuracy gain, compared with the existing ones.

We propose a novel hardware and software co-exploration framework for efficient neural architecture search (NAS). Different from existing hardware-aware NAS which assumes a fixed hardware design and explores theNAS spaceonly, our framework simultaneously explores both the architecture search space and thehardware design spaceto identify the best neural architecture and hardware pairs that maximize both test accuracy and hardware efficiency. Such a practice greatly opens up the design freedom and pushes forward the Pareto frontier between hardware efficiency and test accuracy for better design tradeoffs. The framework iteratively performs a two-level (fast and slow) exploration. Without lengthy training, the fast exploration can effectively fine-tune hyperparameters and prune inferior architectures in terms of hardware specifications, which significantly accelerates the NAS process. Then, the slow exploration trains candidates on a validation set and updates a controller using the reinforcement learning to maximize the expected accuracy together with the hardware efficiency. In this article, we demonstrate that the co-exploration framework can effectively expand the search space to incorporate models with high accuracy, and we theoretically show that the proposed two-level optimization can efficiently prune inferior solutions to better explore the search space. The experimental results on ImageNet show that the co-exploration NAS can find solutions with the same accuracy, 35.24% higher throughput, 54.05% higher energy efficiency, compared with the hardware-aware NAS.

This work aims to enable on-device training of convolutional neural networks (CNNs) by reducing the computation cost at training time. CNN models are usually trained on high-performance computers and only the trained models are deployed to edge devices. But the statically trained model cannot adapt dynamically in a real environment and may result in low accuracy for new inputs. On-device training by learning from the real-world data after deployment can greatly improve accuracy. However, the high computation cost makes training prohibitive for resource-constrained devices. To tackle this problem, we explore the computational redundancies in training and reduce the computation cost by two complementary approaches: 1) self-supervised early instance filtering on data level and 2) error map pruning (EMP) on the algorithm level. The early instance filter selects important instances from the input stream to train the network and drops trivial ones. The EMP further prunes out insignificant computations when training with the selected instances. Extensive experiments show that the computation and energy cost is substantially reduced without any or with marginal accuracy loss. For example, when training ResNet-110 on CIFAR-10, we achieve 67.8% computation saving while preserving full accuracy and 75.1% computation saving with a marginal accuracy loss of 1.3%. When training LeNet on MNIST, we save 79% computation while boosting accuracy by 0.2%. Besides, practical energy saving is measured on edge platforms. We achieve 67.6% energy saving when training ResNet-110 on mobile GPU and 74.1% energy saving when training LeNet on MCU without any accuracy loss.

Static random access memory– (SRAM) based field programmable gate arrays (FPGAs) are currently facing challenges of limited capacity and high leakage power. To solve this problem, non-volatile memory (NVM) is proposed as the alternative to build non-volatile FPGAs (NVFPGAs). Even though the feasibility of NVFPGA has been confirmed, the utilization of multiple level cells (MLCs) has not been fully exploited yet. In this article, we study architecture of MLC-based NVFPGAs, and propose five cluster structures. To give detailed comparisons and extensive discussions, we conduct experiments for area, performance and leakage power evaluation. Based on explorations of the characteristics of MLC-based NVFPGAs, we further present MLC-aware timing-driven packing method to improve delay. In critical paths, our proposed method reduces the overhead of the additional delay in slow MLC cells. Experiments show that, compared to SRAM-based FPGAs, the proposed architecture with the proposed CAD flow can reduce the area, critical path delay and leakage power by 31%, 10%, and 95%, respectively.

With the advent of the Internet of Things (IoT), energy-harvesting nonvolatile processors (NVPs) have become promising platforms due to their durability when running on an intermittent power supply and fast read/write operations. However, the penalties caused by an increasing amount of data to be processed and growing communication demands pose critical challenges in the scheduling of tasks on energy-harvesting NVP platforms with tight energy and latency budgets. Together with the high power switching overhead that is induced under unstable power conditions, an increase in the amount of data to be processed significantly degrades the system performance. To overcome the problems of high communication and switching overheads for energy-harvesting NVP platforms, this article proposes a novel communication-aware task-scheduling technique. The algorithm first selects one or more executable tasks to be performed based on the task benefits and then calls a task partitioning algorithm to dynamically divide the scheduled tasks. We evaluate the performance of our proposed algorithm in comparison with the performance-aware task scheduling (PATH) and greedy iterative (GI) algorithms. Experimental results show that the proposed algorithm can reduce the execution time by 17.08% and 13.72% on average compared with the PATH and GI algorithms, respectively.

A fundamental question lies in almost every application of deep neural networks: what is the optimal neural architecture given a specific data set? Recently, several Neural Architecture Search (NAS) frameworks have been developed that use reinforcement learning and evolutionary algorithm to search for the solution. However, most of them take a long time to find the optimal architecture due to the huge search space and the lengthy training process needed to evaluate each candidate. In addition, most of them aim at accuracy only and do not take into consideration the hardware that will be used to implement the architecture. This will potentially lead to excessive latencies beyond specifications, rendering the resulting architectures useless. To address both issues, in this paper we use Field Programmable Gate Arrays (FPGAs) as a vehicle to present a novel hardware-aware NAS framework, namely FNAS, which will provide an optimal neural architecture with latency guaranteed to meet the specification. In addition, with a performance abstraction model to analyze the latency of neural architectures without training, our framework can quickly prune architectures that do not satisfy the specification, leading to higher efficiency. Experimental results on common data set such as ImageNet show that in the cases where the state-of-the-art generates architectures with latencies 7.81× longer than the specification, those from FNAS can meet the specs with less than 1% accuracy loss. Moreover, FNAS also achieves up to 11.13× speedup for the search process. To the best of the authors' knowledge, this is the very first hardware aware NAS.

Real-time inference with low latency requirement has become increasingly important for numerous applications in both cloud computing and edge computing. The FPGA-based Deep Neural Network (DNN) accelerators have demonstrated the superior performance and energy efficiency over CPUs and GPUs; in addition, for real-time AI with low batch size, FPGA is expected to achieve further performance improvement over the general purpose computing platform. However, the performance gain of the single-FPGA design is hindered by the limited on-chip resource. In this paper, we leverage a cluster of FPGAs to fully exploit the parallelism in DNNs with the objective of obtaining super-linear performance. To achieve this goal, a novel design, "XFER", is proposed to deploy DNNs to FPGA cluster by splitting the DNN layer to multiple FPGAs and moving traffics from memory bus to inter-FPGA links. The resultant system can achieve both workload balance and traffic balance. As a case study, we implement Convolutional Neural Networks (CNNs) on ZCU102 FPGA boards. Evaluation results demonstrate that XFER on two FPGAs can achieve 3.48x speedup compared with state-of-the-art FPGA designs, achieving super-linear speedup.

The papers in this special section focus on emerging non-volatile memory technologies (NVM). Emerging NVM technologies have attracted significant interest in recent years because of the fast-growing performance and capacity demands on memory and storage in the big data era. Well known examples include the 3D XPoint memory and various NVDIMM hybrid memory technologies. They have shown potential towards larger memory and storage capacities with nearly zero leakage power, while extending memory/ system architecture design approaches. The unique characteristics of NVM technologies not only introduce new opportunities, but simultaneously create challenges to the designs at multiple levels of abstraction in computer systems, including those of device management, CPU cache management, memory/storage architecture, and system design. Furthermore, emerging NVM technologies also drive the development of techniques which perform computing operations in memory, i.e., processing-in-memory (PIM), by taking advantage of crossbar-based accelerators using NVMs. Thus, for the emerging NVM technologies, there is an urgent need for technology innovation, modeling, analysis, design, and application, ranging from the device-level to the system-level.

Pipelining is a powerful technique to achieve high performance in computing systems. However, as computing platforms become large-scale and integrate with heterogeneous processing elements (PEs) (CPUs, GPUs, field-programmable gate arrays, etc.), it is difficult to employ a global clock to achieve synchronous pipelines. Therefore, self-timed (or asynchronous) pipelines are usually adopted. Nevertheless, due to their complex running behavior, the performance modeling and systematic optimizations for self-timed pipeline (STP) systems are more complicated than those for synchronous ones. This paper employs marked graph theory to model STPs and presents algorithms to detect performance bottlenecks. Based on the proposed model, we observe that the system performance can be improved by inserting buffers. Due to the limited memory resources on the PEs, it is critical to minimize the number of buffers for STPs while satisfying the required timing constraints. In this paper, we propose integer linear programming formulations to obtain the optimal solutions and devise efficient algorithms to obtain the near-optimal solutions. Experimental results show that the proposed algorithms can achieve 53.10% improvement in the maximum performance and 54.04% reduction in the number of buffers, compared with the technique for the slack matching problem.

As power failures often occur in energy harvesting powered nonvolatile processors (NVPs), checkpointing is needed during program execution. It is observed that checkpointing is implemented with high overhead in applications with loops, because a large amount of data needs backup during loop execution. As such, we are motivated to reduce the amount of checkpointing data by analyzing data locality and shortening data lifetime in loops. This paper proposes a checkpointing-aware loop tiling technique which targets to reduce the checkpointing and recovering overheads for loops. Specifically, we first derive the optimal tile size for nested loops considering checkpointing distance and data dependencies. Then, the implementations of checkpointing and recovering for tiled loops are presented. Finally, the experiments are conducted to evaluate the effectiveness of the proposed method. The experimental results show that compared to the no-tiling method, the checkpointing-aware loop tiling method reduces the checkpointing and recovering data by 36.2% on average and reduces the total execution time and dynamic energy for checkpointing and recovering by 27.2% and 22.9% on average, respectively.

Energy harvesters are becoming increasingly popular as power sources for IoT edge devices. However, one of the intrinsic problems of energy harvester is that harvesting power is often weak and frequently interrupted. Therefore, energy harvesting powered edge devices have to work intermittently. To maintain execution progress, execution states need to be checkpointed into the non-volatile memory before each power failure. In this way, previous execution states can be resumed after power comes back again. Nevertheless, frequent checkpointing and low charging efficiency generate significant energy overhead. To alleviate these problems, this article conducts a thorough energy efficiency analysis and proposes three algorithms to maximize the energy efficiency of program execution. First, a non-volatile processor-aware task scheduling algorithm is proposed to reduce the size of checkpointing data. Second, a tentative checkpointing avoidance technique is proposed to avoid checkpointing for further reduction of checkpointing overhead. Finally, a dynamic wake-up strategy is proposed to wake up the edge device at proper voltages where the total hardware and software overhead is minimized for further energy efficiency maximization. The experiments on a real testbed demonstrate that, with the proposed algorithms, an edge device is resilient to the extremely weak and intermittent power supply and the energy efficiency can be achieved more than 2× higher than the fundamental baseline and 1.5× higher than the state-of-the-art technique.

Real-time Deep Neural Network (DNN) inference with low-latency requirement has become increasingly important for numerous applications in both cloud computing (e.g., Apple’s Siri) and edge computing (e.g., Google/Waymo’s driverless car). FPGA-based DNN accelerators have demonstrated both superior flexibility and performance; in addition, for real-time inference with low batch size, FPGA is expected to achieve further performance improvement. However, the performance gain from the single-FPGA design is obstructed by the limited on-chip resource. In this paper, we employ multiple FPGAs to cooperatively run DNNs with the objective of achieving super-linear speed-up against single-FPGA design. In implementing such systems, we found two barriers that hinder us from achieving the design goal: (1) the lack of a clear partition scheme for each DNN layer to fully exploit parallelism, and (2) the insufficient bandwidth between the off-chip memory and the accelerator due to the growing size of DNNs. To tackle these issues, we propose a general framework, “Super-LIP”, which can support different kinds of DNNs. In this paper, we take Convolutional Neural Network (CNN) as a vehicle to illustrate Super-LIP. We first formulate an accurate system-level model to support the exploration of best partition schemes. Then, we develop a novel design methodology to effectively alleviate the heavy loads on memory bandwidth by moving traffic from memory bus to inter-FPGA links. We implement Super-LIP based on ZCU102 FPGA boards. Results demonstrate that Super-LIP with 2 FPGAs can achieve 3.48× speedup, compared to the state-of-the-art single-FPGA design. What is more, as the number of FPGAs scales up, the system latency can be further reduced while maintaining high energy efficiency.

Maintaining the temporal validity of real-time data in cyber-physical systems is of critical importance to ensure the correct decision making and appropriate system operation. Most existing work on real-time data retrieval assume that the real-time data under study are always available for retrieval, and the developed scheduling algorithms mainly focus on making real-time decisions while meeting the temporal validity constraints. This assumption, however does not hold in many real-time applications with intermittent data availability. In this paper, we study the Availability-constrained Fresh Data Retrieval (AFDR) problem, which aims to retrieve all required real-time data for a given set of decision tasks on time while taking both the temporal validity and data availability constraints into consideration. We formulate the AFDR problem as an ILP problem and study its complexity under different settings. Given the general case of the AFDR problem is proved to be NP-hard, we focus on the cases that data items have unit-size retrieval time. For the single decision task scenario, we propose a polynomial-time optimal data retrieval algorithm, which consists of a task finish time selection phase and an optimal retrieval schedule construction phase, to solve the AFDR problem. For the multiple decision task scenario, we propose an efficient heuristic algorithm by transforming the temporal validity constraint of a real-time data item to the availability constraint. The effectiveness of the proposed algorithms has been validated through extensive experiments. Our results show that the heuristic algorithm outputs around $1.5\times$1.5× feasible cases compared to that of the state-of-the-art scheme.

Non-volatile main memory-based systems pose an opportunity for an attacker to readily access sensitive information on the memory because of its long retention time. While real-time memory encryption with dedicated AES engine can address this vulnerability, it incurs extra performance and energy overheads. As an alternative, we propose an AES in-memory implementation, AIM, to encrypt the whole/part of the memory only when it is necessary. We leverage the benefits offered by the inmemory computing architecture to address the challenges of the bandwidth intensive encryption application. We take advantage of NVM's intrinsic logic operation capability to implement the AES task. Embracing the massive parallelism inside the memory, AIM outperforms existing mechanisms with higher throughput yet lower energy consumption. Compared with state-of-the-art AES engine running at 2.1GHz, AIM can speed up the encryption process by 80 χ for a 1GB NVM.

Energy harvesting is a promising solution to power ubiquitous Internet-of-Things (IoT) devices. But the frequent and inevitable power failure incurs significant backup overhead, greatly degrading performance and energy efficiency. Nonvolatile processor (NVP), which can checkpoint processor states, is designed to tackle this problem. The conventional system-level design method involves repeated system modification and verification on hardware, in which measurement on hardware consumes the majority time. To expedite the NVP-based system design process, we propose a rapid system prototyping flow to eliminate repeated hardware measurement in the design flow. This method involves an NVP system-level simulator, which takes the harvester power trace, system characteristics extracted from hardware, and user design as the input, and analyzes system energy and time profile under this power trace. Iterative system optimization and verification are conducted on the simulator, with only the final verification on hardware. We demonstrate the advantages of this method by two design cases, in which time, energy efficiency and the impact of different capacitor size are optimized.

Field programmable gate array (FPGA) has been one of the most popular platforms to implement convolutional neural networks (CNNs) due to its high performance and cost efficiency; however, limited by the on-chip resources, the existing single-FPGA architectures cannot fully exploit the parallelism in CNNs. In this paper, we explore heterogeneous FPGA-based designs to effectively leverage both task and data parallelism, such that the resultant system can achieve the minimum cost while satisfying timing constraints. In order to maximize the task parallelism, we investigate two critical problems: 1) buffer placement, where to place buffers to partition CNNs into pipeline stages and 2) task assignment, what type of FPGA to implement different CNN layers. We first formulate the system-level optimization problem with a mixed integer linear programming model. Then, we propose an efficient dynamic programming algorithm to obtain the optimal solutions. On top of that, we devise an efficient algorithm that exploits data parallelism within CNN layers to further improve cost efficiency. Evaluations on well-known CNNs demonstrate that the proposed techniques can obtain an average of 30.82% reduction in system cost under the same timing constraint, and an average of 1.5 times speedup in performance under the same cost budget, compared with the state-of-the-art techniques.

The capacity of SRAM-based FPGA block RAM (BRAM) is restrained by the low density and high leakage power of the current CMOS technology. In this paper, we propose a nonvolatile memory (NVM)-based BRAM architecture which enables flexible conversions between single-level cell (SLC) and multilevel cell (MLC) states. We show that despite the high per-access latency and power consumption, MLC-based BRAM blocks reduce the routing cost between logic units and on-chip data storages, which potentially leads to a smaller critical path delay and power consumption. Therefore, we propose an NVM BRAM architecture and an EDA framework which adaptively packs data into SLC- or MLC-state BRAMs during FPGA design flow in order to achieve better system performance. This paper illustrates that a simple memory device replacement from SRAM to NVM leads to nonoptimal system performance. On the other hand, compared with operating all NVM BRAM blocks in the SLC state with better per-access latency and power consumption, the proposed hybrid SLC-MLC architecture and design flow improves the critical path delay by 18.51%, with a system power reduction of 25.83% at the same time. Moreover, compared with the traditional “fast” SRAM-based BRAM blocks under the same BRAM area constraint, our hybrid NVM BRAM architecture improves the critical path delay by 8.55% on average, with an average system power reduction of 54.34% at the same time.

Internet of Things (IoT) edge devices usually work under power-constrained scenarios like outdoor environmental monitoring. Considering the cost and sustainability in a long run, energy harvesting technology is preferable for edge devices. Nevertheless, the harvesting power is generally weak and unstable, making it difficult for edge devices to maintain the normal functionality. Hence, it is crucial to improve the energy efficiency of edge devices. This paper proposes a software paradigm, ENZYME, to improve the energy efficiency of edge devices for transient computing with ultralow energy harvesting power supplies. ENZYME consists of two lightweight yet highly efficient software modules including Routine Handler and frequency modulator (FM). Routine Handler assists power regulator to maximize power extraction from energy harvesters with proper operation routines. Further, FM maximizes the utility of the extracted energy for program execution via efficient runtime clock frequency modulation. The lightweight and highly efficient natures enable ENZYME to be integrated into low-power IoT edge devices easily and efficiently. Experimental results demonstrate that ENZYME achieves more than 8.8% energy efficiency over state-of-the-art techniques with Routine Handler, and 35.71% extra energy utility with FM upon applying Routine Handler.

Energy harvesting is becoming a favorable alternative to power future generation embedded systems, as it is more environmentally and user friendly. However, energy harvesting powered embedded systems suffer from frequent execution interruption due to unstable energy supply. To tackle this problem, nonvolatile memory has been deployed to save the whole volatile state for computation. When power resumes, the processor can restore the state back to volatile memories and continue execution. However, without careful consideration, the process of checkpointing and resuming could cause inconsistency between volatile and nonvolatile memories, which leads to irreversible errors. In this article, we propose a consistency-aware adaptive checkpointing scheme that ensures correctness for all checkpoints. The proposed technique efficiently identifies all possible inconsistency positions in programs and inserts auxiliary code to ensure correctness by offline analysis. In addition, adaptive checkpointing assisted register file profiling and online tracking techniques further reduce the overhead of each checkpoint. Evaluation results show that the proposed checkpointing strategy can successfully eliminate inconsistency errors and greatly reduce the checkpointing overhead.

The emerging Phase Change Memory (PCM) is considered to be a promising candidate to replace DRAM as the next generation main memory due to its higher scalability and lower leakage power. However, the high write power consumption has become a major challenge in adopting PCM as main memory. In addition to the fact that writing to PCM cells requires high write current and voltage, current loss in the charge pumps also contributes a large percentage of high power consumption. The pumping efficiency of a PCM chip is a concave function of the write current. Leveraging the characteristics of the concave function, the overall pumping efficiency can be improved if the write current is uniform. In this article, we propose a peak-to-average (PTA) write scheme, which smooths the write current fluctuation by regrouping write units. In particular, we calculate the current requirements for each write unit by their values when they are evicted from the last level cache (LLC). When the write units are waiting in the memory controller, we regroup the write units by LLC-assisted PTA to reach the current-uniform goal. Experimental results show that LLC-assisted PTA achieved 13.4% of overall energy saving compared to the baseline.

Resistive RAM (RRAM) is a promising non-volatile memory (NVM) device which can replace traditional SRAM as on-chip storage for logic and data in FPGAs. While RRAM outperforms SRAM by offering high scalability, low leakage power, and near-zero power-on delay, RRAM-FPGAs have limited programming cycles, and different writes frequencies of memory and logic blocks make the challenge more severe. To overcome this endurance challenge, we propose an age-aware placement framework for RRAM-FPGAs with uniform reconfigurable logic/memory units. The framework, consisting of a dynamic reconfiguration region allocation algorithm and a logic/memory co-placement algorithm, balances write distributions across the entire FPGA according to logic and memory write frequency differences. The proposed algorithms have been integrated into the VTR synthesis flow. Experiments show that the framework achieves 94.9% write reduction, thus effectively extending RRAM-FPGA programming cycles.

Intelligent computing and maintenance-free powering are two desirable characteristics of wearable IoT devices. Energy harvesting nonvolatile intelligent processor (NIP) with neural network computation capability has the potential to advance these goals. Individual ultraviolet (UV) exposure monitoring progressively becomes one conspicuous application of wearable devices. In resource constrained wearable sensor nodes, we can alleviate the data transmission burden via convolutional neural networks (CNNs) based pattern recognition. Nevertheless, in spite of the substantially improved computing capability of NIP, typically computational and memory intensive CNNs are still too bulky for on-node implementation. We develop an CNN-based pattern recognition system for nonvolatile IoT platform for smart UV monitoring, and propose a optimization method to achieve extremely tiny and efficient CNNs. Experimental results show that the offline-trained CNN can recognize individual UV exposure patterns with accuracy of 85%, and the simplified on-node CNN can achieve 93.2% parameters reduction with only 5% accuracy loss.

Static random access memory (SRAM) based field programmable gate arrays (FPGAs) are currently facing challenges of limited capacity and high leakage power. To solve this problem, non-volatile memory (NVM) is proposed as the alternative to build non-volatile FPGAs (NVFPGAs). Even though the feasibility of NVFPGA has been confirmed, the utilization of multiple level cells (MLC) has not been fully exploited yet. In this paper, we study architecture of MLC based NVFPGAs, and propose five cluster structures, as well as the corresponding working mode supported by MLC based clusters. To give detailed comparisons and extensive discussions, we conduct experiments for area, performance and leakage power evaluation. Experiments show that, compared to SRAM based FPGAs, the proposed architecture can reduce the area, latency and leakage power by 32.66% and 7.45%, and 96.13%, respectively.

Energy harvesting techniques become increasingly popular as power supplies for embedded systems. However, the harvested energy is intrinsically unstable. Thus, the program execution may be interrupted frequently. Although the development of non-volatile processors (NVP) can save and restore execution states, both hardware and software challenges exist for energy harvesting powered embedded systems. On the hardware side, existing power detector only signals the ``poor'' quality of the harvested power based on a preset threshold voltage. The inappropriate setting of this threshold will make the NVP based embedded system suffer from either unnecessary checkpointing or checkpointing failures. On the software side, not all tasks can be checkpointed. Once the power is off, these tasks will have to restart from the beginning. In this paper, a task scheduler is proposed to maximize task progress by prioritizing tasks which cannot be checkpointed when power is weak so that they can finish before the power outage. To assist task scheduling, three additional modules including voltage monitor, checkpointing handler, and routine handler, are proposed. Experimental results show increased overall task progress and reduced energy consumption.

Energy harvesting systems become increasingly popular as power sources for many embedded systems. However, the harvesting power is often weak and the execution is frequently interrupted. Therefore, embedded systems have to work intermittently. To maintain the execution progress for better energy utilization, embedded systems need to save all execution states and program stacks into the non-volatile memory before each power failure, which is known as checkpointing. Then, embedded system waits until power comes back on again. By then the system can resume previous execution state. Nevertheless, frequent checkpointing incurs extra energy overhead. Besides, the charging efficiency of the storage capacitor reduces as the capacitor is charging up. These problems reduce the energy efficiency, resulting in less program execution progress. To alleviate these problems, this paper proposes three algorithms. First, a priority-based task scheduling (PTS) is proposed to prioritize the execution of tasks which have less checkpointing contents for a lower software overhead. Then, a tentative checkpointing avoidance (TCA) technique is proposed to avoid unnecessary checkpointing for further reduction of software overhead. Finally, a dynamic wake-up strategy (DWS) is proposed to wake up the system at proper voltages where the total hardware and software overheads are minimized. In this way, the system can achieve further energy efficiency improvements. The experiments on a real testbed show that the proposed prioritized algorithms enable the targeted embedded system to be resilient to extremely weak and intermittent power while achieving better energy utilization.

I/O is becoming one of major performance bottlenecks in NAND-flash-based mobile devices. Novel nonvolatile memories (NVMs), such as phase change memory and spin-transfer torque random access memory, can provide fast read/write operations. In this paper, we propose a unified NVM/flash architecture to improve the I/O performance. A transparent scheme, virtualized flash (vFlash), is also proposed to manage the unified architecture. Within vFlash, interapp and intra-app techniques are proposed to optimize the application performance by exploiting the historical locality and I/O access patterns of applications. Since vFlash is on the bottom of the I/O stack, the application features will be lost. Therefore, we also propose a cross-layer technique to transfer the application information from the application layer to the vFlash layer. The proposed scheme is evaluated based on an Android platform, and the experimental results show that the proposed scheme can effectively improve the I/O performance of mobile devices.

Aggressive technology scaling and adoption of multilevel-cell technique lead to progressive increase of bit error rate (BER) of NAND flash memory. Consequently, conventional error correction code is not adequate to guarantee system reliability. As an alternative, low density parity check (LDPC) code is introduced to provide more powerful error correction capability. However, to achieve better performance, LDPC code demands extra memory sensing operations and more data transfer cycles, directly leading to longer read latency. To achieve both system reliability and read efficiency, we propose the FlexLevel NAND flash storage system design in this paper. FlexLevel consists of two levels of optimization: 1) LevelAdjust and 2) AccessEval. At device level, the LevelAdjust technique is proposed to reduce BER by broadening noise margin via threshold voltage level reduction. With LevelAdjust, BER is greatly reduced and no extra sensing levels are required to protect data integrity. Hence, read performance is improved. However, while LevelAdjust can improve system reliability and read performance, it causes density loss. To balance read performance improvement and density loss, we propose the AccessEval technique at system level. AccessEval identifies data with high LDPC overhead and only applies LevelAdjust technique to these data. The experimental results show that compared with the best existing works, the proposed design can achieve up to 11% read speedup with negligible density loss.

Wearable devices gain increasing popularity since they can collect important information for healthcare and well-being purposes. Compared with battery, energy harvesting is a better power source for these wearable devices due to many advantages. However, harvested energy is naturally unstable and program execution will be interrupted frequently. Nonvolatile processors demonstrate promising advantages to back up volatile state before the system energy is depleted. However, it also introduces non-negligible energy and area overhead. In this paper, we aim to reduce the amount of data that need to be backed up during a power failure. Based on the observation that stack size varies along program execution, we propose to analyze the application program and identify efficient backup positions, by which the stack content to back up can be significantly reduced. The evaluation results show an average of 45.7% reduction on nonvolatile stack size for stack backup, with 0.58% storage overhead. In the mean time, with the proposed schemes, the energy utilization and program forward progress can be greatly improved compared with instant backup.

Phase change memory (PCM) is one of the most promising candidates to replace DRAM as main memory in deep submicron regime. Regardless of single-level or multiple-level cells, the programming costs to each state exhibit significant asymmetries in latency, energy and endurance. In this paper, we exploit the potential of reducing programming costs in terms of latency, energy, and endurance for PCM through state remapping. First, quantitative programming models are constructed for cost assessments. Then, both dynamic and static remapping schemes are analyzed and compared. The observation that the efficacy of dynamic state remappings is instable motivates us to propose a static remapping technique, which outperforms previous work in cost reduction within much lower implementation overhead. The optimality of the proposed static state remapping is also proved. The evaluation results confirm the efficacy of the proposed state remapping technique in delivering a stable and promising cost reduction in latency, energy, and wear.

Existing Nonvolatile Memories (NVMs) have many attractive features to be the main memory of embedded systems. These features include low power, high density, and better scalability. Recently, Multilevel Cell (MLC) NVM has gained more and more popularity as it can provide a higher density than the traditional Single-Level Cell (SLC) NVM. However, there are also drawbacks in MLC NVM, namely, limited write endurance and expensive write operation. These two drawbacks have to be overcome before MLC NVM can be practically adopted as the main memory. In MLC Nonvolatile Main Memory (NVMM), two different types of write operations with very diverse data retention times are allowed. The first type maintains data for years but takes a longer time to write and is detrimental to the endurance. The second type maintains data for a short period but takes a shorter time to write. By observing that much of the data written to main memory is temporary and does not need to last long during the execution of a program, in this article, we propose novel task scheduling and write operation selection algorithms to improve MLC NVMM endurance and program efficiency. An Integer Linear Programming (ILP) formulation is first proposed to obtain optimal results. Since ILP takes exponential time to solve, we also propose the Multiwrite Mode-Aware Scheduling (MMAS) algorithm to achieve a near-optimal solution in polynomial time. Additionally, the Dynamical Memory Block Screening (DMS) algorithm is proposed to achieve wear leveling. The experimental results demonstrate that the proposed techniques can greatly improve the lifetime of the MLC NVMM as well as the efficiency of the program.

Field-programmable gate arrays (FPGAs) have drawn lots of attentions due to their programmability and high performance. Recently, ultralow-power FPGAs for Internet of Things, together with energy-harvesting technique, have become an emerging self-powered computing platform. However, volatile memory in FPGA will lose their states under unstable power supplies and cannot work efficiently. Nonvolatile FPGA becomes a promising alternative. This paper proposes a hardware/software codesign nonvolatile FPGA with efficient offline/online checkpointing strategy (CP-FPGA). Backup energy is reduced by offline selecting proper checkpointing locations to minimize backup data. An online scheduler is further proposed to balance computation rollback overhead against backup energy. Experimental results show that the proposed CP-FPGA reduces 39.5% energy consumption on average compared with the state-of-the-art techniques.

The emerging Phase Change Memory (PCM) is considered as a promising candidate to replace DRAM as the next generation main memory since it has better scalability and lower leakage power. However, the high write power consumption has become a main challenge in adopting PCM as main memory. In addition to the fact that writing to PCM cells requires high write current and voltage, current loss in the charge pumps (CPs) also contributes a large percentage of the high power consumption. The pumping efficiency of a PCM chip is a concave function of the write current. Based on the characteristics of the concave function, the overall pumping efficiency can be improved if the write current is uniform. In this paper, we propose the peak-to-average (PTA) write scheme, which smooths the write current fluctuation by regrouping write units. An off-line optimal Integer Programming (IP) formulation and an efficient online algorithm are proposed to achieve this goal. Experimental results show that PTA can improve the charge pump efficiency to ∼40% with little overhead. Meanwhile, PTA can achieve 17.0% energy reduction on average.

Non-Volatile memory-based FPGAs (NV-FPGAs) are expected to replace traditional SRAM-based FPGAs to achieve higher scalability and lower power consumption. Yet the slow write performance of NVMs not only challenges FPGA reconfiguration speed and overhead but also constrains the programming cycles of FPGAs. To efficiently configure switch boxes, the majority component of an FPGA, this paper proposes a routing path reuse technique. Technical contributions include a mathematical reconfiguration cost model of routing resources, a reuse-aware routing algorithm, as well as the incorporation of the proposed algorithm into the standard VTR CAD tool. Experiments on standard MCNC benchmarks show that the proposed scheme is able to achieve as much as 40% path reuse rate and reduce as much as 34.0% configuration cost for routing resources.

Nonvolatile processors have manifested strong vitality in battery-less energy harvesting sensor nodes due to their characteristics of zero standby power, resilience to power failures and fast read/write operations. However, I/O and sensing operations cannot store their system states after power off, hence they are sensitive to power failures and high power switching overhead is induced during power oscillation, which significantly degrades the system performance. In this paper, we propose a novel performance-aware task scheduling technique considering power switching overhead for energy harvesting nonvolatile processors. We first give the analysis of the power switching overhead on energy harvesting sensor nodes. Then, the scheduling problem is formulated by MILP (Mixed Integer Linear Programming). Furthermore, a task splitting strategy is adopted to improve the performance and an heuristic scheduling algorithm is proposed to reduce the problem complexity. Experimental results show that the proposed scheduling approach can improve the performance by 14% on average compared to the state-of-the-art scheduling strategy. With the employment of the task splitting approach, the execution time can be further reduced by 10.6%.

Spin-transfer torque random access memory (STT-RAM) is considered as a promising candidate to replace SRAM as the next generation cache memory since it has better scalability and lower leakage power. Recently, 2-bit multi-level cell (MLC) STT-RAM has been proposed to further increase data density. However, a key drawback for MLC STT-RAM is that the magnetization directions of its hard and soft domains cannot be flipped to two opposite directions simultaneously, which leads to the two-step problem in state transitions. Two-step state transitions would significantly impact the lifetime of MLC STT-RAM due to the wasted flips in the soft domains. To solve the problem, this paper proposes a novel two-step state transition minimization (TSTM) scheme, to improve the lifetime of MLC STT-RAM when it is employed in cache design. The basic idea is by sacrificing certain cells as auxiliary flags, the two-step state transitions in STT-RAM can be well eliminated. Experimental results show that the proposed scheme can improve the lifetime of MLC STT-RAM to 318.5%.

As massive multi-threading in GPU imposes tremendous pressure on memory subsystems, efficient bandwidth utilization becomes a key factor affecting the GPU throughput. In this work, we propose thread batch enabled memory partitioning (TEMP), to improve GPU performance through the improvement of memory bandwidth utilization. In particular, TEMP clusters multiple thread blocks sharing the same set of pages into a thread batch and dispatches the entire thread batch to a stream multiprocessor. TEMP separates the memory access streams of different thread batches by OS memory management, preserving the intrinsic locality of thread batches and increasing the memory access parallelism. Experimental results show that TEMP can obtain up to 10.3% performance improvement and 14.6% DRAM energy reduction compared to a state-of-the-art scheduler without any memory-side optimizations.

Due to size, longevity, safety, and recharging concerns, energy harvesting is becoming a better choice for many wearable embedded systems than batteries. However, harvested energy is intrinsically unstable. In order to overcome this drawback, non-volatile processors (NVPs) have been proposed to bridge intermittent program execution. However, even with NVPs, frequent power interruptions will severely degrade system performance. Hence, in this paper we adopt a multi-source in-door energy harvesting architecture to compensate the shortcoming of single energy source. We further investigate power harvesting prediction techniques, which are critical for NVP systems since they can coordinate with task scheduler in the NVP system to compensate the intermittent ambient energy harvesting. We investigate prediction methods both for single energy harvesting source and for multiple energy harvesting sources, the total output power of which is more stable compared with the single source case. A comprehensive evaluation framework has been developed using actually measured harvesting traces on the proposed neural network-based power harvesting prediction methods. It turns out that the most favorable prediction methods are directly predicting the total output power of DC-DC converters (connecting between energy sources and NVP), or predicting the total input power of DC-DC converters first and then inferring the total output power using a learned mapping function, for multi-source power harvesting predictions.

Energy harvesting is becoming a preferred choice for future wearable embedded systems compared to batteries because of size, longevity, and maintenance convenience. However, harvested energy is intrinsically unstable. In order to overcome this drawback, non-volatile processors (NVPs) have been proposed to bridge intermittent program execution. However, the harvested power is limited even with multiple energy harvesters when they are in-door. Therefore, a near-threshold processor is ideal to maintain low power consumption. One of the biggest challenges in realizing near-threshold non-volatile processor is to provide a required high write voltage to non-volatile memories when there is a power failure and checkpoint is needed. In order to address this challenge, in this paper, we propose a dynamic converter reconfiguration for ambient energy harvesting-based NVPs to support near-threshold computing. We further investigate thorough optimization techniques to achieve high robustness in reconfiguration and checkpointing, high conversion efficiency, and low ripple magnitude. Experimental results demonstrate that the proposed techniques can significantly reduce the power consumption and improve the performance of energy harvesters and NVPs.

Due to size, longevity, safety, and recharging concerns, energy harvesting is becoming a better choice for many wearable embedded systems. However, harvested energy is intrinsically unstable. In order to overcome this drawback, nonvolatile processors (NVPs) was proposed to bridge intermittent program execution. However, even with NVPs, frequent power interruption will severely degrade system performance. In this paper, we will propose a multi-source energy harvesting system to combine multiple harvesting sources to provide a more stable power supply using real-life ambient in-door irradiation level and thermal energy power trace. Maximum power extraction and converter parameter optimization techniques will be discussed. Experimental results show that by optimizing the MOSFET switch size in DC-DC converters between the harvesting system and NVPs, the average output power of the multi-source harvesting system can be increased. The proposed architecture is very promising in providing a stable energy source for NVPs.

Wearable devices gain increasing popularity since they can collect important information for healthcare and well-being purposes. Compared with battery, energy harvesting is a better power source for these wearable devices due to many advantages. However, harvested energy is naturally unstable and program execution will be interrupted frequently. Nonvolatile processor (NVP) demonstrates promising advantages to back up volatile state before the system energy is depleted. Due to the backup and resumption procedures resulted from frequent power failures, non-volatile processor exhibits different characteristics from traditional processors, necessitating a set of adaptive design and optimization strategies. Recently, there have been both hardware and software researches aiming to develop correct and efficient non-volatile processors. In this paper, we summarize the software-level techniques for NVP, covering error-correctness schemes, backup timing determination, backup content optimization, adaptive software modifications and NVP simulators and tools, to provide an overview of state-of-the-art NVP research from the software and system level.

Architecting PCM, especially MLC PCM, as main memory for MCUs is a promising technique to replace conventional DRAM deployment. However, PCM/MLC PCM suffers from long write latency and large write energy. Recent work has proposed a compiler directed dual-write (CDDW) scheme to combat the drawbacks of PCM by adopting fast or slow mode for different write operations. For large-scale loops, we observe that write instances' lifetime is very long and can only be written by the expensive slow mode. This paper proposes a write mode aware loop tiling approach to effectively reduce the lifetime of write instances and maximize the number of efficient fast writes in loops. The experimental results show that the proposed approach improves performance by 50.8 percent and reduces dynamic energy by 32.0 percent across a set of benchmarks compared to the CDDW approach on average.

Applications that run in the embedded systems normally should be finished within a timing constraint in energy-efficient fashion. Due to these two requirements, the embedded systems often employ software-controlled scratch pad memory (SPM) instead of hardware-controlled cache as their on-chip memory. The data accesses in SPMs are controlled purely by the software, which provides better time-predictability and precise time-control. In this paper, we propose a time, energy, and area efficient domain wall memory (DWM)-based SPM for embedded systems. To efficiently manage this type of novel SPM, an integer nonlinear programming formulation and the instructions group schedule algorithm are proposed to generate memory access instruction scheduling and data placement. In addition, the longest move reduce algorithm is also proposed to configure different types of DWM memory cells to achieve minimal area size. Experimental results show that the proposed techniques can generate a configuration of DWM-based SPM with minimal area size while satisfying time constraint.

I/O is becoming one of major performance bottlenecks in NAND-flash-based smartphones. Novel NVMs (nonvolatile memories), such as PCM (Phase Change Memory) and STT-RAM (Spin-Transfer Torque Random Access Memory), can provide fast read/write operations. In this paper, we propose an unified NVM/flash architecture to improve the I/O performance. A transparent scheme, vFlash (Virtualized Flash), is also proposed to manage the unified architecture. Within vFlash, inter-app technique is proposed to optimize the application performance by exploiting the historic locality of applications. Since vFlash is on the bottom of the I/O stack, the application features will be lost. Therefore, we also propose a cross-layer technique to transfer the application information from the application layer to the vFlash layer. The proposed scheme is evaluated based on a real Android platform, and the experimental results show that the read and write performance for the proposed scheme is 2.45 times and 3.37 times better than that of the stock Android 4.2 system, respectively.

Phase change memory (PCM), given its non-volatility and low static energy consumption, is a promising candidate to be used as main memory. However, due to its limited endurance and slow write performance, state-of-the-art solutions tend to construct a DRAM-PCM hybrid memory instead of using PCM exclusively. While existing optimizations to this hybrid architecture focus on tuning DRAM configurations to further reduce writes to PCM, we aim at developing a proactive solution. Specifically, we exploit the flexibility of mapping virtual pages to physical pages, and propose a page allocation algorithm that considers both segment information and conflict misses in DRAM to distribute heavily written pages across different DRAM sets. Trace-driven experiments confirm the effectiveness of proposed technique in reducing both DRAM misses and PCM writes, thus simultaneously improving performance and lifetime of DRAM-PCM hybrid main memory.

Embedded systems powered with harvested energy experience frequent execution interruption due to unstable energy source. Nonvolatile (NV) register based processor is proposed to realize fast resume after power failure. The states in the volatile registers are checkpointed to NV registers. However, frequent checkpointing causes performance degradation and consumes excessive power. In this paper, we propose the checkpoint aware instruction scheduling (CAIS) algorithm to reduce the writes to NV registers. Experiments show that CAIS can improve performance and reduce power consumption.

Domain Wall Memory (DWM), a recently developed spin-based non-volatile memory technology, inherently offers unprecedented benefits in density by storing multiple bits in the domains of a ferromagnetic nanowire, which logically resembles a bit-serial tape. However, this structure also leads to a unique challenge that the bits must be sequentially accessed by performing \shift" operations, resulting in variable and potential higher access latencies. In this paper, we propose a hardware and software co-optimize approach to improve area efficiency and performance for DWM in application-specific embedded systems. For an application-specific embedded system, this technique can obtain a DWM which consists of both micro-cell DWM and macro-cell DWM with minimal area size. Meanwhile, instruction schedule and data allocation with minimal memory access overhead are generated. Experimental results show that the proposed method can minimize the DWM area size while satisfying a system performance constraint.

LDPC code is introduced in NAND flash memory to handle high BER (bit error rate) incurred by technology scaling. Despite strong error correction capability, LDPC decoding induces long NAND flash read latency. In this work, we propose FlexLevel -- a robust NAND flash storage system design to improve data reliability and read efficiency affected by the LDPC operations. FlexLevel first reduces BER by enlarging noise margins via Vth (threshold voltage) level reduction. It reduces the sensing levels of LDPC but also causes loss of storage capacity. To compensate this capacity loss with minimum impact on read performance, FlexLevel identifies the data with high LDPC overhead and only applies the Vth level reduction technique to those data. Experimental results show that compared with state-of-the-art, FlexLevel can achieve up to 33% read speedup with very moderate capacity loss.

Wearable devices are becoming increasingly important in our daily lives. Energy harvesting instead of battery is a better power source for these wearable devices due to many advantages. However, harvested energy is often unstable and program execution will be frequently interrupted. Non-volatile processors demonstrate promising advantages to back up volatile state before the system energy is depleted. But Non-volatile processors require additional memory for backing up, thus introducing non-negligible overhead in terms of energy, runtime as well as chip area. In this work, we target at non-volatile register reduction for energy harvesting based wearable devices. This paper proposes to stack trimming the memory footprint via a novel compiler directed method. The evaluation results deliver on average 28.6% reduction of non-volatile register files for backing up stack area, with ultra low runtime overhead.

Massive multi-threading of GPGPU demands for efficient usage of caches with limited capacity. In this work, we propose a versatile warp scheduler (VWS) to reduce the cache miss rate in GPGPU. VWS retains the intra-warp cache locality using an efficient per-warp working set estimator and enhances intra-/inter-cooperative thread array (CTA) cache locality through imposing a CTA-aware scheduling policy and a new CTA dispatching mechanism. The significantly improved hit rate of cache hierarchy enables VWS to achieve on average 38.4% and 9.3% IPC improvement across diverse GPGPU applications compared to a widely-used and a state-of-the-art warp schedulers, respectively.

Energy harvesting has become a favorable alternative to batteries for wearable embedded systems since it is more environmental and user friendly. However, harvested energy is intrinsically unstable, which could frequently interrupt a processor's execution. To tackle this problem, non-volatile processors have been proposed to checkpoint the whole volatile processor state into attached non-volatile memories periodically. When power resumes, the processor can copy the checkpointed state back to volatile memories and continue execution. However, without careful consideration, the process of checkpointing and resuming could cause inconsistency among different memory addresses and lead to irreversible errors. In this paper, we present a consistency aware checkpointing scheme that ensures correctness for all checkpoints. The proposed technique efficiently identifies all possible inconsistency positions in programs and inserts auxiliary code to ensure correctness. Evaluation results show that the proposed checkpointing technique can successfully eliminate inconsistency errors and greatly reduce the checkpointing overhead.

Non-volatile memories (NVMs) such as Phase Change Memory (PCM) have been considered as promising candidates of next generation main memory for embedded systems due to their attractive features. These features include low power, high density, and better scalability. However, most existing NVMs suffer from two drawbacks, namely, limited write endurance and expensive write operation in terms of both time and energy. These problems are worsen when modern high-level languages employ virtual machine with garbage collector that generates a large amount of extra writes on non-volatile main memory. To tackle this challenge, this paper proposes three techniques: Living Objects Remapping (LORE), Dead Object Stamping (DOS), and Smart Wiping with Maximum Likelihood Estimation (SMILE) to reduce the unnecessary writes when garbage collector handles objects. The experimental results show that the proposed techniques not only significantly reduce the writes during each garbage collection cycle but also greatly improve the performance of virtual machine.

Non-volatile memories (NVMs) outperform traditional SRAMs in terms of low power consumption, high capacity, near-zero power-on delay, and high error-resistance. Researchers have demonstrated the possibilities of implementing FPGA building blocks with various types of NVMs. However, NVMs also bring several new design challenges to FPGAs: the slow write performance of NVM may degrade FPGA (re)configuration speed, while the limited write endurance of NVM constrains the number of times that the FPGA can be (re)configured. Unfortunately, none of these NVM features are taken into consideration in current FPGA synthesis tools, which have been optimized solely for SRAM-based FPGAs. To tackle this limitation, we propose to make the FPGA placement process aware of the slow and costly NVM writes. Our contributions are three-fold: We first construct mathematical models to characterize reconfiguration costs in NVM-based FPGAs. Second, we identify three types of flexibilities that can be exploited to reduce the reconfiguration cost. Finally, we present three approaches for designers to fine-tune the placement process to balance the reconfiguration cost and traditional timing and routability constraints according to their needs. The proposed algorithms are incorporated in Verilog-to-Routing (VTR) CAD tool. Experiments on standard MCNC benchmark circuits show that our approach eliminates up to 67% NVM writes during the reconfiguration process, thus effectively improving the performance and endurance of NVM-based FPGAs.

Non-volatile memory (NVM) technologies have been known for their advantages of large capacity, low energy consumption, high error-resistance, and near-zero power-on delay. It is expected that they will replace traditional SRAM as FPGA reconfigurable blocks. While NVMs promise FPGAs with more reconfigurable resources, lower power consumption, and higher resilience to power interruptions, they also impose two new design challenges: the slow write performance of NVMs may degrade FPGA reconfiguration speed, while their limited write endurance constrains FPGA programming cycles. To overcome these challenges, we propose a similarity driven approach to reduce reconfiguration cost in NVM-based FPGAs. When synthesizing a new design, its similarity to the design currently on the FPGA is characterized by taking both LUT contents and CLB-level topology into consideration. The reconfiguration cost minimization problem is formulated as a bipartite graph matching problem and solved optimally. Experiments on standard circuit benchmarks show that the proposed algorithms eliminate more than 57.4% of NVM writes during the reconfiguration process, thus effectively improving performance and endurance of NVM-based FPGAs.

Multi-core processors have been adopted in modern embedded systems to meet the ever increasing performance requirements. Scratchpad memory (SPM), a software-controlled on-chip memory, has been used in embedded systems as an alternative to hardware-controlled cache due to its advantage in die area, power consumption, and timing predictability. SPMs in multi-core systems can be accessed by both local core and remote cores. In order to alleviate data contention on a SPM unit, multi-port SPMs are employed in multi-core systems. In such systems, proper task scheduling and data assignment can significantly improve the overall performance by exploring the parallelism of computation tasks and concurrent data accesses on SPMs. Since scheduling for multi-core systems is NP-Complete in general. In this paper, we propose an ILP formulation to optimally determine the task scheduling and data assignment on multi-core systems with multi-port SPMs. Since ILP takes exponential time to finish, we also propose a heuristic method, including the task assignment with remote access reduced (TARAR) algorithm and the minimum memory access cost (MMAC) algorithm, to obtain near optimal solutions within polynomial time. According to the experimental results, the ILP formulation can improve the system performance by 23.02 percent over the HAFF algorithm on average, while the heuristic algorithm can improve the system performance by 16.48 percent over HAFF on average.

Phase change memory (PCM) is a promising DRAM replacement in embedded systems due to its attractive characteristics, such as low-cost, shock-resistivity, nonvolatility, high density, and low leakage power. However, relatively low endurance has limited its practical applications. In this paper, in addition to existing hardware level optimizations, we propose software enabled wear-leveling techniques to further extend PCMs lifetime when it is adopted in embedded systems. Most existing software optimization techniques focus on reducing the total number of writes to PCM, but none of them consider wear leveling, in which the writes are distributed more evenly over the PCM. An integer linear programming formulation and a polynomial-time algorithm, the software wear-leveling algorithm, are proposed in this paper to achieve wear leveling without hardware overhead. According to the experimental results, the proposed techniques can reduce the number of writes on the most-written addresses by more than 80% when compared with a greedy algorithm, and by more than 60% when compared with the existing optimal data allocation algorithm with under 6% memory access overhead.

As the advance of memory technologies, multiple types of memory such as different kinds of non-volatile memory (NVM), SRAM, DRAM, etc. provide a flexible configuration considering performance, energy and cost. For improving the performance of systems with multiple types of memory, data allocation is one of the most important tasks. The previous studies on data allocation problem assume the worst (fixed) case of data-access frequencies. However, the data allocation produced by employing worst case usually leads to an inferior performance for most of time. In this paper, we model this problem by probabilities and design efficient algorithms that can give optimal-cost data allocation with a guaranteed probability. The proposed DAGP algorithm produces a set of feasible data allocation solutions which generates the minimum access time or cost guaranteed by a given probability. The experiments show that our technique can significantly reduce the access time or cost compared with the technique considering worst case scenario. For example, comparing with the optimal result generated by employing the worst cases, our technique can reduce memory access time by 10.35% on average when guaranteed probability is set to be 0.8. Moreover, for 80 percents of cases, memory access time is reduced by 23.98% on average.

Phase Change Memory (PCM) has been considered as a leading candidate to replace the traditional DRAM in embedded systems due to its promising characteristics such as low leakage power, low cost, non-volatility, and high scalability. One of the constraints that undermine the credential of PCM as main memory is its limited write endurance. In this paper, we develop wear-leveling techniques purely on the Operating System (OS) level to extend lifetime of PCM. Without extra hardware support, OS management is more flexible to be integrated into existing embedded systems. To achieve wear-leveling, the Periodical Page Swapping (PPS), Rearrangement Inequality Based Page Allocation (RIPA), and Write Intensity Based Process Scheduling (WIPS) algorithms are proposed in this paper on OS level. The experimental results show that the proposed techniques can significantly extend the lifetime of PCM main memory.

It is common that embedded systems are powered by limited and unstable power supply. In order to improve the reliability of embedded systems against unstable power supply, non-volatile memory (e.g. FRAM) based registers are proposed for embedded processors. FRAM-based registers have many advantages over traditional CMOS-based volatile registers such as non-volatility and power-economy. However, similar to other non-volatile memories (NVM), write operations to FRAM consume more time and power compared with read operations and limit the lifetime of the registers. Existing compiler optimization techniques never take the writes to registers into consideration. Therefore, code generated by a traditional compiler has an adverse effect on processors with non-volatile registers. This paper aims at improving the lifetime and efficiency of non-volatile registers based embedded processors by generating NV register friendly code. To achieve the goal, in this paper, we investigate the usage of memory access instructions and propose the NV Register Aware Instruction Selection (NAIS) algorithm to reduce the write operations on non-volatile registers. According to the experimental results, the proposed algorithm can reduce the writes on NV registers by 66.89% on average when compared with GCC [1]. Thus the lifetime of NV registers is extended to 2 times as long as before on average. The time cost is reduced by 56.68% and the energy consumption is reduced by 59.76% on average.

In power and size sensitive embedded systems, non-volatile memories (NVMs) are replacing DRAM as the main memory since they have higher density, lower static power consumption, and lower costs. Unfortunately, these technologies are limited by their endurance and long write latencies. To minimize the main memory access time and extend the lifetime of the NVM, we optimally schedule tasks by an ILP formulation. We also present a heuristic, Concatenation Scheduling, to solve large problems in a reasonable amount of time. Our experimental results show that when compared with list scheduling, concatenation scheduling can reduce the total memory access time by an average of 9.99% and increase the lifetime of the NVM by 26.66%. When compared with list scheduling, ILP can reduce the total memory access time by an average of 12.39% and increase the lifetime of the NVM by 38.74%.

The recent emergence of various Non-Volatile Memories (NVMs), with many attractive characteristics such as low leakage power and high-density, provides us with a new way of addressing the memory power consumption problem. In this article, we target embedded CMPs, and propose a novel Hybrid Scratch Pad Memory (HSPM) architecture which consists of SRAM and NVM to take advantage of the ultra-low leakage power, high density of NVM, and fast access of SRAM. A novel data allocation algorithm as well as an algorithm to determine the NVM/SRAM ratio for the novel HSPM architecture are proposed. The experimental results show that the data allocation algorithm can reduce the memory access time by 33.51% and the dynamic energy consumption by 16.81% on average for the HSPM architecture when compared with a greedy algorithm. The NVM/SRAM size determination algorithm can further reduce the memory access time by 14.7% and energy consumption by 20.1% on average.

High-performance computing systems typically employ heterogeneous multicore design to improve both execution performance and efficiency. Task assignment is critical in exploiting the diversity of computation capability, energy consumption, as well as communication cost on heterogeneous multicore processors. In this paper, we explore the opportunity of task assignment on heterogeneous multicore processors to minimize execution and communication costs considering time constraint. The general heterogeneous task assignment problem is NP-Complete. However, we find that optimal task assignment can be achieved for widely used, tree-shaped task graphs using dynamic programming. We first propose a dynamic programming algorithm, the Optimal Tree Assign (OTA) algorithm, to generate optimal assignments for trees. Then, we develop the Integer Linear Programming model of the general task assignment problem for Directed Acyclic Graphs. A polynomial-time heuristic, the Extended Tree Assignment algorithm, is also proposed to produce near-optimal solutions for the general heterogeneous task assignment problem efficiently. The experimental results show that the proposed algorithms outperform both homogeneous task assignment method and greedy strategy for all the benchmarks. The OTA algorithm reduces the total system time by 42.5 percent and 23.5 percent on average compared with the homogeneous task assignment method and greedy algorithm, respectively.

Phase Change Memory (PCM) is a promising DRAM replacement in embedded systems due to its attractive characteristics. However, relatively low endurance has limited its practical applications. In this paper, in additional to existing hardware level optimizations, we propose software enabled wear-leveling techniques to further extend PCM's lifetime when it is adopted in embedded systems. A polynomial-time algorithm, the Software Wear-Leveling (SWL) algorithm, is proposed in this paper to achieve wear-leveling without hardware overhead. According to the experimental results, the proposed technique can reduce the number of writes on the most-written bits by more than 80% when compared with a greedy algorithm, and by around 60% when compared with the existing Optimal Data Allocation (ODA) algorithm with under 6% memory access overhead.

Virtually Shared Scratch-Pad Memory (VS-SPM) with multiple memory banks can be used as on-chip memory on multiprocessor systems-on-chips (MPSoCs) to close the speed gap between fast processors and slow memories. By exploring the parallelism of computation tasks on processors and concurrent data accesses on each SPM, the results of task assignment and data allocation can significantly affect the overall performance of a schedule. In this paper, we propose ILP formulations for solving the problem of task assignment and scheduling on MPSoCs with multi-bank VS-SPM.We also propose a polynomial-time algorithm, the Potential Remote Access Prediction (PRAP) algorithm, to generate near-optimal results efficiently. The experimental results demonstrate the effectiveness of our technique.

Phase change memory (PCM) has emerged as a promising candidate to replace DRAM in embedded systems. However, it can only sustain a limited number of write operations. To solve this issue, this paper proposes a novel and effective wear-leveling technique in software level to prolong the lifetime of PCM-based embedded systems. A polynomial-time algorithm, Multi-Space Wear Leveling Algorithm (MWL), is proposed to achieve effective wear-leveling. The experimental results show our technique can greatly extend the lifetime of PCM-based embedded systems compared with the previous work. Compared with the method without adopting wear-leveling, it introduces no more than 0.7% extra writes and 0.6% running overhead.

Effective task assignment, which is essential for achieving high performance in a heterogeneous multiprocessor system, remains a challenging problem despite extensive studies. This paper addresses the task assignment problem with guaranteed reliability and timing constraint for heterogeneous multiprocessor system. Inherently, heterogeneous systems are more complex than homogeneous systems. The added complexity could increase the potential for system failures. In this paper, we describe a method to determine an assignment which satisfies the timing constraint and the reliability requirement. We develop an Integer Linear Programming (ILP) formulation to find the optimal solutions. For the general problem, the task assignment problem is NP-Complete. Therefore, we propose a polynomial-time heuristic algorithm, DAG Heu algorithm, to solve the general problem. Experimental results on benchmark task graphs of several well-known parallel applications show that the proposed algorithm and the ILP formulation significantly outperform existing algorithms.

Scratch pad memories (SPM) are attractive alternatives for caches on multicore systems since caches are relatively expensive in terms of area and energy consumption. The key to effectively utilizing SPMs on multicore systems is the data placement algorithm. In this paper, two polynomial time algorithms, regional data placement for multicore (RDPM) and regional data placement for multicore with duplication (RDPM-DUP), have been proposed to generate near-optimal data placement with minimum total cost. There is only one copy for each data in RDPM, while RDPM-DUP allows data duplication. Experimental results show that the proposed RDPM algorithm alone can reduce the time cost of memory accesses by 32.68% on average compared with existing algorithms. With data duplication, the RDPM-DUP algorithm further reduces the time cost by 40.87%. In terms of energy consumption, the proposed RDPM algorithm with exclusive copy can reduce the total cost by 33.47% on average. When RDPM-DUP is applied, the improvement increases up to 38.15% on average.

Recent advances in circuit and semiconductor technologies have pushed Non-Volatile Memory (NVM) technologies into a new era. These technologies exhibit appealing properties such as low power consumption, non-volatility, shock-resistivity, and high density. However, there are challenges to which we need answers in the road of applying non-volatile memories as main memory in embedded computer systems. First, when compared with DRAM, NVMs have a limited number of write/erase cycles. Second, write activities on NVM are more expensive than DRAM memory in terms of energy consumption and access latency. Both challenges will benefit from the reduction of the write activities on the NVMs. In this paper, we target embedded Chip Multiprocessors (CMPs) with Scratch Pad Memory (SPM) and non-volatile main memory. We introduce scheduling, data migration, and recomputation techniques to reduce the number of write activities on NVMs. Experimental results show that the proposed methods can reduce the number of writes by 58.46% on average, which means that the NVM can last 2.8 times as long as before. For Phase Change Memory (PCM), the lifetime is extended from 2.5 years to about 7 years on average and 15 years at the most. Also, the finish time of the tested programs is reduced by an average of 38.07%, and the energy consumption is reduced by an average of 51.23%.

Embedded systems normally have a tight energy budget. Since the on-chip cache typically consumes 25%-50% of the processor's area and energy consumption, scratch pad memory (SPM), which is a software-controlled on-chip memory, has been widely adopted in many embedded systems due to its smaller area and lower power consumption. However, as the speed of the CMOS transistors increases along with density, leakage power consumption is becoming a critical issue for memory components with a large number of transistors. In this paper, we propose a novel hybrid SPM which consists of static random-access memory (SRAM) and nonvolatile memory (NVM) to take advantage of the ultralow leakage power and high density of latter. A novel dynamic data management algorithm is also proposed to make use of the full potential of NVM. According to the experimental results, with the help of the proposed algorithm, the novel hybrid SPM architecture can reduce the memory access time by 18.17%, the dynamic energy by 24.29%, and the leakage power by 37.34% compared with a baseline pure SRAM SPM with the same area.

In embedded systems power consumption and area tightly constrain the cache capacity and management logic. Many good cache replacement policies have been proposed in the past, but none approach the performance of the least recently used (LRU) algorithm without incurring high overheads. In fact, many embedded designers consider even pseudo-LRU too complex for their embedded systems processors. In this paper, we propose a new level 1 (L1) data cache replacement algorithm, Protected Round-Robin (PRR) that is simple enough to be incorporated into embedded processors while providing miss rates that are very similar to the miss rates of LRU. Our experiments showed that on average the miss rates of PRR are only 0.22% higher than the miss rates of LRU on a 32KB, 4-way L1 data cache with 32 byte long cache lines. PRR has miss rates that are on average 4.72% and 4.66% lower than random and round-robin replacement algorithms, respectively.

Non-Volatile Memories (NVMs) have many advantages over traditional DRAM. It is desirable to apply NVM as main memory in embedded Chip Multi-Processor (CMP) systems. However, NVMs have drawbacks that need to be overcome. That is, a write to the NVMs is expensive. Loops are the most critical and time-consuming part in digital signal processing (DSP) applications. However, loops are difficult to parallelize on multi-processor systems due to the inter-iteration dependencies. This paper targets on embedded CMP systems and proposes techniques to improve loop parallelism while considering reducing the write activities to the NVMs when they are used as main memory. The experimental results show that the proposed algorithm can reduce the number of write activities on NVM by 21.1% on average. In other words, the average lifetime of NVM can be extended to at least 2 times longer than before and the total schedule length is reduced by 19.6% on average.

Phase Change Memory (PCM) has emerged as a promising alternative of DRAM to provide energy-efficient and high-capacity memory for high performance servers. A new DRAM + PCM hybrid memory architecture has been proposed to leverage PCM's high density and DRAM's robustness and performance. One of the big challenges of PCM is its limited write endurance (107~ 108times per cell). By knowing the association between DRAM and PCM, malicious software can easily force DRAM cache to be flushed continuously, which produces writes to certain PCM cells repeatedly (known as selective attack) and wears out PCM. Although existing wear-leveling approaches could evenly distribute writes under selective attack, the overall endurance of PCM is still severely impacted, and therefore it is suboptimal. In this paper, we propose Cache Address Remapping (CAR), that can adaptively remap DRAM cache address, to hide the association between DRAM and PCM. Moreover, CAR can minimize the write-back traffic to PCM under selective attack by uniformly distributing the writes to a single cache set into different cache sets. We propose a practical and low overhead implementation of CAR, called RanCAR. Experimental results show that CAR could reduce DRAM cache miss rate by ~4600x under selective attack, and prolong PCM lifetime from several minutes to 13.8 years on average.

Scratch Pad Memory (SPM), a software-controlled on-chip memory, has been widely adopted in many embedded systems due to its small area and low power consumption. As technology scaling reaches the sub-micron level, leakage energy consumption is surpassing dynamic energy consumption and becoming a critical issue. In this paper, we propose a novel hybrid SPM which consists of non-volatile memory (NVM) and SRAM to take advantage of the ultra-low leakage power consumption and high density of NVM as well as the efficient writes of SRAM. A novel dynamic data allocation algorithm is proposed to make use of the full potential of both NVM and SRAM. According to the experimental results, with the help of the proposed algorithm, the novel hybrid SPM architecture can reduce memory access time by 18.17%, dynamic energy by 24.29%, and leakage power by 37.34% on average compared with a pure SRAM based SPM with the same size area.

Multi-core systems have been a popular design for high-performance embedded systems. Scratch Pad Memory (SPM), a software-controlled on-chip memory, has been widely adopted in many embedded systems due to its small area and low energy consumption. Existing data allocation algorithms either cannot achieve optimal results or take exponential time to complete. In this paper, we propose one polynomial-time algorithms to solve the data allocation problem on multi-core system with exclusive data copy. According to the experimental results, the proposed optimal data allocation method alone reduces time cost of memory accesses by 16.45% on average compared with greedy algorithm. The proposed data allocation algorithm also can reduce the energy cost significantly.

Scratch-Pad Memory (SPM) has been widely adopted in many embedded systems as well as digital signal processor systems. This paper proposes a polynomial time optimal data placement algorithm to minimize the memory access cost of one program region for memory architectures with multiple types of memory units including SPM in order to achieve high performance with low cost. The experimental results show our algorithms can reduce time cost of memory access by 18.19% and the energy cost by 16.97% compared with random data placement, which is better than the existing greedy algorithms.

Nonvolatile memories such as Flash memory, phase change memory (PCM), and magnetic random access memory (MRAM) have many desirable characteristics for embedded systems to employ them as main memory. However, there are two common challenges we need to answer before we can apply nonvolatile memory as main memory practically. First, nonvolatile memory has limited write/erase cycles compared to DRAM. Second, a write operation is slower than a read operation on nonvolatile memory. These two challenges can be answered by reducing the number of write activities on nonvolatile main memory. In this paper, we proposed two optimization techniques, write-aware scheduling and recomputation, to minimize write activities on nonvolatile memory. With the proposed techniques, we can both speed up the completion time of programs and extend nonvolatile memory's lifetime. The experimental results show that the proposed techniques can reduce the number of write activities on nonvolatile memory by 55.71% on average. Thus, the lifetime of nonvolatile memory is extended to 2.5 times as long as before on average. The completion time of programs can be reduced by 56.67% on systems with NOR Flash memory and by 47.63% on systems with NAND Flash memory on average.

Recent advances in circuit and process technologies have pushed non-volatile memory technologies into a new era. These technologies exhibit appealing properties such as low power consumption, non-volatility, shock-resistivity, and high density. However, there are challenges to which we need answers in the road of applying non-volatile memories as main memory in computer systems. First, non-volatile memories have limited number of write/erase cycles compared with DRAM memory. Second, write activities on non-volatile memory are more expensive than DRAM memory in terms of energy consumption and access latency. Both challenges will benefit from reduction of the write activities on the nonvolatile memory.

Flash Memory is a desirable candidate for main memory replacement in embedded systems due to its low leakage power consumption, higher density and non-volatility characteristics. There are two challenges in applying flash memory as main memory. First, the write operations are much slower than read operations. Second, the lifetime of flash memory depends on the number of the write/erase operations. In this paper, we introduce a smart victim cache architecture to reduce the write activities by exploring the coarse grain accessing character of NAND flash memory. Experimental results show that the proposed approaches can reduce write activities on flash main memory by 65.38% on average compared to traditional architecture.

In power and size sensitive embedded systems, flash memory and phase change memory are replacing DRAM as the main memory. Unfortunately, these technologies are limited by their endurance and long write latencies. To minimize the main memory access time, we optimally schedule tasks by an ILP formulation that can be generally applied to other main memory technologies, including DRAM. We also present a heuristic, Wander Scheduling, to solve larger instances in a reasonable amount of time. Our experimental results show that when compared with list scheduling, Wander Scheduling can reduce memory access times by an average of 40.73% and increase the lifetime of flash and phase change memory by 82.56%.

The widening gap between processor and memory performance is the main bottleneck for modern computer systems to achieve high processor utilization. To hide memory latency, a variety of techniques have been proposed—from intermediate fast memories (caches) to various prefetching and memory management techniques. In this article, we propose a new loop scheduling with memory management technique, Iterational Retiming with Partitioning (IRP), that can completely hide memory latencies for applications with multidimensional loops on architectures like CELL processor. In IRP, the iteration space is first partitioned carefully. Then a two-part schedule, consisting of processor and memory parts, is produced such that the execution time of the memory part never exceeds the execution time of the processor part. These two parts are executed simultaneously and complete memory latency hiding is reached. In this article, we prove that such optimal two-part schedule can always be achieved given the right partition size and shape. Experiments on DSP benchmarks show that IRP consistently produces optimal solutions as well as significant improvement over previous techniques.

According to the characteristics of the "3-D" structure of contemporary DRAM chips, the row first column ordered (RFCO) algorithm is proposed in this paper to minimize memory access schedule length. In memory systems with a single memory controller, assuming that the memory access trace is known before scheduling, the RFCO algorithm can generate schedules which are 7.89% shorter than burst scheduling on average. If memory accesses are coming to the single memory controller in real time, the RFCO algorithm can generate schedules which are 8.03% shorter than burst scheduling on average.

Many high-performance DSP processors employ multi-module on-chip memory to improve performance and power consumption. This paper studies the scheduling and assignment problem that minimizes the total energy while satisfying performance for applications with loops. An algorithm, LSAMEM (Loop Scheduling and Assignment to Minimize Energy for Memory), is proposed. The algorithm attempts to maximum energy saving while satisfying timing constraint with guaranteed probability. The experimental results show that the average improvement on energy-saving is significant by using LSAMEM.

In wireless sensor networks, the preloaded program code and data on sensor nodes often need to be updated due to changes in user requirements or environmental conditions. Sensor nodes are severely restricted by energy constraints. It is especially energy consuming for sensor nodes to update code through radio packages. To efficiently update code through wireless radio, we propose an algorithm, reprogramming with minimal transferred data (RMTD), to find the optimum combination of copying from the old code image and downloading from the host machine to minimize the number of bytes needed to be transferred from the host machine to a sensor node. Our experiments show that, for small code modifications, RMTD reduces the number of bytes transferred by 93.25% over the existing Rsync-based algorithm. For normal code changes, RMTD shows an improvement of 59.82% in average.

This paper studies the scheduling and assignment problem that minimizes the total energy including both dynamic and leakage energy for applications with loops on multi-voltage, multi-processor DSP. An algorithm, LSAMP (Loop Scheduling and Assignment to Minimize Power), is proposed. The algorithm attempts to minimize the total energy while satisfying timing constraint with guaranteed probability. We will perform scheduling and assignment simultaneously. Our approach shows better performance than the approach that considers scheduling and assignment at separate phases. Compared with previous work, our algorithm shows a significant improvement in total energy reduction.

Many high-performanceDSP processors employ multi-bank on-chip memory to improve performance and energy consumption. This architectural feature supports higher memory bandwidth by allowing multiple data memory accesses to be executed in parallel. This paper studies the scheduling and assignment problem on minimizing the total energy consumption while satisfying timing constraint with heterogeneous multi-bank memory for applications with loop. An algorithm, TASL (Type Assignment and Scheduling for Loops), is proposed. The algorithm uses loop scheduling and assignment with the consideration of variable partition to find the best configuration for both memory and ALU.

Multiple memory banks design is employed in many high performance DSP processors. This architectural feature supports higher memory bandwidth by allowing multiple data memory access to be executed in parallel. Dedicated address generation units (AGUs) are commonly presented in DSPs to perform address arithmetic in parallel to the main datapath. Address assignment, optimization of memory layout of program variables to reduce address arithmetic instruction, has been studied extensively on single memory architecture. Make effective use of AGUs on multiple memory banks is a great challenge to compiler design and has not been studied previously. In this paper, we exploit address assignment with variable partitioning for scheduling on DSP architectures with multiple memory banks and AGUs. Our approach is built on novel graph models which capture both parallelism and serialism demands. An efficient scheduling algorithm, Address Assignment Sensitive Variable Partitioning (AASVP), is proposed to best leverage both multiple memory banks and AGUs. Experimental results show significant improvement compare to existing methods.