Performing symbol detection for multiple-input and multiple-output orthogonal frequency division multiplexing (MIMO-OFDM) systems is challenging and resource-consuming. In this paper, we present a liquid state machine (LSM), a type of reservoir computing based on spiking neural networks (SNNs), to achieve energy-efficient and sustainable symbol detection on the Loihi chip for MIMO-OFDM systems. SNNs are more biological-plausible and energy-efficient than conventional deep neural networks (DNN) but have lower performance in terms of accuracy. To enhance the accuracy of SNNs, we propose a knowledge distillation training algorithm called DNN-SNN co-learning, which employs a bi-directional learning path between a DNN and an SNN. Specifically, the knowledge from the output and intermediate layer of the DNN is transferred to the SNN, and we exploit a decoder to convert the spikes in the intermediate layers of an SNN into real numbers to enable communication between the DNN and the SNN. Through the bi-directional learning path, the SNN can mimic the behavior of the DNN by learning the knowledge from the DNN. Conversely, the DNN can better adapt itself to the SNN by using the knowledge from the SNN. We introduce a new loss function to enable knowledge distillation on regression tasks. Our LSM is implemented on Intel's Loihi neuromorphic chip, a specialized hardware platform for SNN models. The experimental results on symbol detection in MIMO-OFDM systems demonstrate that our LSM on the Loihi chip is more precise than conventional symbol detection algorithms. Also, the model consumes approximately 6 times less energy per sample than other quantized DNN-based models with comparable accuracy.

3D face reconstruction from a single image still suffers from low accuracy and inability to recover textures in invisible regions. In this paper, we propose a method for generating a 3D portrait with complete texture. The coarse face-and-head model and texture parameters are obtained using 3D Morphable Model fitting. We design an image-geometric inverse renderer that acquires normal, albedo, and light to jointly reconstruct the facial details. Then, we use a texture fusion network to extract the valid texture from rendered faces employing different viewpoints. Specifically, this fused texture recovers the invisible region of the input face, which illustrates the realistic surface of our 3D geometric model. Our approach faithfully reconstructs the original face details, including the profiles and the head region. Extensive experiments are performed to demonstrate that our method outperforms state-of-the-art techniques in various challenging scenarios.

Face texture completion plays a significant role in virtual human research, and the quality of face texture needs to be improved urgently. One of the major obstacles to single-face texture generation is that the generated textures are always incomplete for self-occlusion of the input face, and the other is that pixel details are limited by the illumination. To address this, we propose a method for complete face texture generation based on generative adversarial networks. The face parameters obtained from 3D Morphable Model are processed as conditional vectors in the encoder, and the multivariate Gaussian distribution of the latent code is used in the networks to learn the complete texture features. We established a face texture dataset CFT for training the network. Meanwhile, we show the effectiveness of the proposed approach in qualitative and quantitative experiments. The visual results under different tasks show superior performances compared with the state-of-the-art approaches.

Most deep learning-based state-of-the-art stereo matching methods significantly depend on large-scale datasets. However, it is implausible to collect sufficient real-world samples with dense and clear ground-truth disparity maps in practice. Although synthetic datasets’ appearance has alleviated the demand for extensive real data, there is a domain shift between synthetic and real sets. To tackle this problem, we propose an individually trained synthetic-to-real domain adaptation (SDA) network that maps synthetic images into the real domain. Specifically, our approach translates the data style from synthetic domain to real domain while maintaining the content and the spatial information. First, edge cues are leveraged to guide domain adaptation in preserving the spatial consistency between input and the generated image. Second, we combine the spatial feature transform (SFT) layer to effectively fuse features from the edge map and the source image. Extensive experiments demonstrate that: 1) when only trained on synthetic data and generalized to real data, our model evidently outperforms many state-of-the-art domain adaptation methods; 2) our translated synthetic datasets (TSD) help to improve the generalization capability of any stereo matching CNNs. Codes and data will be available athttps://github.com/Archaic-Atom/SDA_network.

In this paper, we introduce a deep spiking delayed feedback reservoir (DFR) model to combine DFR with spiking neuros: DFRs are a new type of recurrent neural networks (RNNs) that are able to capture the temporal correlations in time series while spiking neurons are energy-efficient and biologically plausible neurons models. The introduced deep spiking DFR model is energy-efficient and has the capability of analyzing time series signals. The corresponding field programmable gate arrays (FPGA)-based hardware implementation of such deep spiking DFR model is introduced and the underlying energy-efficiency and recourse utilization are evaluated. Various spike encoding schemes are explored and the optimal spike encoding scheme to analyze the time series has been identified. To be specific, we evaluate the performance of the introduced model using the spectrum occupancy time series data in MIMO-OFDM based cognitive radio (CR) in dynamic spectrum sharing (DSS) networks. In a MIMO-OFDM DSS system, available spectrum is very scarce and efficient utilization of spectrum is very essential. To improve the spectrum efficiency, the first step is to identify the frequency bands that are not utilized by the existing users so that a secondary user (SU) can use them for transmission. Due to the channel correlation as well as users' activities, there is a significant temporal correlation in the spectrum occupancy behavior of the frequency bands in different time slots. The introduced deep spiking DFR model is used to capture the temporal correlation of the spectrum occupancy time series and predict the idle/busy subcarriers in future time slots for potential spectrum access. Evaluation results suggest that our introduced model achieves higher area under curve (AUC) in the receiver operating characteristic (ROC) curve compared with the traditional energy detection-based strategies and the learning-based support vector machines (SVMs).

With the advance of edge computing, a fast and efficient machine learning model running on edge devices is needed. In this paper, we propose a novel quantization approach that reduces the memory and compute demands on edge devices without losing much accuracy. Also, we explore its application in communication such as symbol detection in 5G systems, attack detection of smart grid, and dynamic spectrum access. Conventional neural networks such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs) could be exploited on these applications and achieve state-of-the-art performance. However, conventional neural networks consume a large amount of computation and storage resources, and thus do not fit well to edge devices. Reservoir computing (RC), which is a framework for computation derived from RNN, consists of a fixed reservoir layer and a trained readout layer. The advantages of RC compared to traditional RNNs are faster learning and lower training costs. Besides, RC has faster inference speed with fewer parameters and resistance to overfitting issues. These merits make the RC system more suitable for applications running on edge devices. We apply the proposed quantization approach to RC systems and demonstrate the proposed quantized RC system on Xilinx Zynq®-7000 FPGA board. On the sequential MNIST dataset, the quantized RC system utilizes 62%, 65%, and 64% less of DSP, FF, and LUT, respectively compared to the floating-point RNN. The inference speed is improved by 17 times with an 8% accuracy drop.

In Internet-of-Thing (IoT) networks, enormous low-power IoT devices execute latency-sensitive yet computation intensive machine learning tasks. However, the energy is usually scarce for IoT devices, especially for some without battery and relying on solar power or other renewables forms. In this paper, we introduce a cross-layer optimization framework for distributed computing among low-power IoT devices. Specifically, a programming layer design for distributed IoT networks is presented by addressing the problems of application partition, task scheduling, and communication overhead mitigation. Furthermore, the associated federated learning and local differential privacy schemes are developed in the communication layer to enable distributed machine learning with privacy preservation. In addition, we illustrate a three-dimensional network architecture with various network components to facilitate efficient and reliable information exchange among IoT devices. Moreover, a model quantization design for IoT devices is illustrated to reduce the cost of information exchange. Finally, a parallel and scalable neuromorphic computing system for IoT devices is established to achieve energy-efficient distributed computing platforms in the hardware layer. Based on the introduced cross-layer optimization framework, IoT devices can execute their machine learning tasks in an energy-efficient way while guaranteeing data privacy and reducing communication costs.