Successful integration of deep neural networks (DNNs) or deep learning (DL) has resulted in breakthroughs in many areas. However, deploying these highly accurate models for data-driven, learned, automatic, and practical machine learning (ML) solutions to end-user applications remains challenging. DL algorithms are often computationally expensive, power-hungry, and require large memory to process complex and iterative operations of millions of parameters. Hence, training and inference of DL models are typically performed on high-performance computing (HPC) clusters in the cloud. Data transmission to the cloud results in high latency, round-trip delay, security and privacy concerns, and the inability of real-time decisions. Thus, processing on edge devices can significantly reduce cloud transmission cost. Edge devices are end devices closest to the user, such as mobile phones, cyber–physical systems (CPSs), wearables, the Internet of Things (IoT), embedded and autonomous systems, and intelligent sensors. These devices have limited memory, computing resources, and power-handling capability. Therefore, optimization techniques at both the hardware and software levels have been developed to handle the DL deployment efficiently on the edge. Understanding the existing research, challenges, and opportunities is fundamental to leveraging the next generation of edge devices with artificial intelligence (AI) capability. Mainly, four research directions have been pursued for efficient DL inference on edge devices: 1) novel DL architecture and algorithm design; 2) optimization of existing DL methods; 3) development of algorithm–hardware codesign; and 4) efficient accelerator design for DL deployment. This article focuses on surveying each of the four research directions, providing a comprehensive review of the state-of-the-art tools and techniques for efficient edge inference.

The adverse glycemic events triggered by the inaccurate insulin infusion in Type I diabetes (T1D) can lead to fatal complications. Predicting blood glucose concentration (BGC) based on clinical health records is critical for control algorithms in the artificial pancreas (AP) and aiding in medical decision support. This paper presents a novel deep learning (DL) model incorporating multitask learning (MTL) for personalized blood glucose prediction. The network architecture consists of shared and clustered hidden layers. Two layers of stacked long short-term memory (LSTM) form the shared hidden layers that learn generalized features from all subjects. The clustered hidden layers comprise two dense layers adapting to the gender-specific variability in the data. Finally, the subject-specific dense layers offer additional fine-tuning to personalized glucose dynamics resulting in an accurate BGC prediction at the output. OhioT1DM clinical dataset is used for the training and performance evaluation of the proposed model. A detailed analytical and clinical assessment have been performed using root mean square (RMSE), mean absolute error (MAE), and Clarke error grid analysis (EGA), respectively, which demonstrates the robustness and reliability of the proposed method. Consistently leading performance has been achieved for 30- (RMSE = 16.06 $\pm$ 2.74, MAE = 10.64 $\pm$ 1.35), 60- (RMSE = 30.89 $\pm$ 4.31, MAE = 22.07 $\pm$ 2.96), 90- (RMSE = 40.51 $\pm$ 5.16, MAE = 30.16 $\pm$ 4.10), and 120-minute (RMSE = 47.39 $\pm$ 5.62, MAE = 36.36 $\pm$ 4.54) prediction horizon (PH). In addition, the EGA analysis confirms the clinical feasibility by maintaining more than 94% BGC predictions in the clinically safe zone for up to 120-minute PH. Moreover, the improvement is established by benchmarking against the state-of-the-art statistical, machine learning (ML), and deep learning (DL) methods.

Characterizing the spatial relationship between blood vessel and lymphatic vascular structures, in the mice dura mater tissue, is useful for modeling fluid flows and changes in dynamics in various disease processes. We propose a new deep learning-based approach to fuse a set of multi-channel single-focus microscopy images within each volumetric z-stack into a single fused image that accurately captures as much of the vascular structures as possible. The red spectral channel captures small blood vessels and the green fluorescence channel images lymphatics structures in the intact dura mater attached to bone. The deep architecture Multi-Channel Fusion U-Net (MCFU-Net) combines multi-slice regression likelihood maps of thin linear structures using max pooling for each channel independently to estimate a slice-based focus selection map. We compare MCFU-Net with a widely used derivative-based multi-scale Hessian fusion method [8]. The multi-scale Hessian-based fusion produces dark-halos, non-homogeneous backgrounds and less detailed anatomical structures. Perception based no-reference image quality assessment metrics PIQUE, NIQE, and BRISQUE confirm the effectiveness of the proposed method.