On-device machine learning (ML) moves computation from the cloud to personal devices, protecting user privacy and enabling intelligent user experiences. However, fitting models on devices with limited resources presents a major technical challenge: practitioners need to optimize models and balance hardware metrics such as model size, latency, and power. To help practitioners create efficient ML models, we designed and developed Talaria : a model visualization and optimization system. Talaria enables practitioners to compile models to hardware, interactively visualize model statistics, and simulate optimizations to test the impact on inference metrics. Since its internal deployment two years ago, we have evaluated Talaria using three methodologies: (1) a log analysis highlighting its growth of 800+ practitioners submitting 3,600+ models; (2) a usability survey with 26 users assessing the utility of 20 Talaria features; and (3) a qualitative interview with the 7 most active users about their experience using Talaria.

A low-power online deep neural network (DNN) training processor is proposed for a real-time object tracking in mobile devices. For a real-time object tracking, a homogeneous core architecture is proposed to achieve 1.33× higher throughput than previous DNN training processor. To reduce the external memory access (EMA), a binary feedback alignment (BFA) algorithm and an integral run-length compression (iRLC) decoder are proposed. While the BFA reduces the EMA by 11.4% compared to the conventional back-propagation approach, the iRLC decoder achieves 29.7% EMA reduction without throughput degradation. Finally, a dropout controller is proposed and achieves 43.9% power reduction through clock-gating. Implemented with 65 nm CMOS technology, the 4.4 mm2DNN training processor achieves 141.1 mW power consumption at 30.4 frames-per-second (fps) real-time object tracking in mobile devices.

An ultra-low-power face frontalization processor (FFP) is proposed for accurate face recognition in wearable devices. 3D face frontalization is essential in face recognition to guarantee human-level accuracy even with rotated or tilted faces. To reduce external memory access (EMA), which causes large power consumption, regression weight quantization with K-means clustering is proposed with the result of 81.25% EMA reduction. In addition, pipelined memory-level zero-skipping regression reduces the EMA by additional 98.43% without latency overhead. Moreover, for low-power consumption of accelerating heterogeneous workload, energy-efficient shared PE array architecture is proposed. While accelerating computation intensive process by allocating large number of PEs for utilizing data-level parallelism, unused PEs are clock-gated for preventing needless power consumption during computationally light process. Proposed workload adaptation with clock-gating showed 37.14% power reduction. The proposed FFP was implemented in 65nm CMOS process, and showed 0.53mW power consumption with 4.73fps throughput, both of which satisfy condition for always-on face recognition in wearable devices.

An energy-efficient speech extraction (SE) processor is proposed for the robust speech recognition in the head-mounted display (HMD) systems. Speech extraction is essential for robust speech recognition in noisy environment. For the low-latency speech extraction, FastSE is proposed to overcome 50x larger complex cICA-based selection process which results in <;2ms SE latency. Moreover, a reinforced-FastSE (RFSE) scheme is proposed to achieve 97.2% accuracy with small on-chip memory size of only 33kB for the low-power HMD applications. Also, Reconfigurable matrix operation accelerator (RMAT) is implemented for energy-efficient acceleration of dominant matrix operation on SE. As a result, the proposed SE processor achieves 1.3x lower latency with 4.24x smaller memory compared to the state-of-the-art work, so that speech recognition in noisy environment becomes possible for mobile HMD applications.