As an efficient model compression technique, knowledge distillation has become an important research topic in the field of deep learning. However, the requirement of pre-trained teacher networks makes the process cumbersome and inefficient, which prompted researchers to propose a more efficient mechanism. Therefore, self-knowledge distillation is proposed, which does not require assistance from additional teacher networks. In previous surveys, self-knowledge distillation has usually been considered a special case of knowledge distillation. In recent years, significant progress has been made in self-knowledge distillation, which has evolved beyond the functions or roles of traditional knowledge distillation. However, there is no individual and intensive survey of self-knowledge distillation methods up to now. Therefore, this paper reviews and investigates existing self-knowledge distillation methods from a comprehensive perspective. Specifically, first, according to different sources of knowledge, this paper categorizes self-knowledge distillation methods into three types, label knowledge-based, feature knowledge-based and data knowledge-based. Then, this paper introduces the evaluation protocol and performance of SKD. In particular, the commonly used experimental datasets and evaluation networks are summarized, aiming to encourage researchers to choose common network architectures and evaluation datasets for promoting the standardization of self-knowledge distillation's comparison. Finally, this paper introduces the applications of self-knowledge distillation in different task scenarios, enabling researchers to quickly locate the relevant task fields. Furthermore, this paper introduces the technical purposes of applying self-knowledge distillation and explores the core motivations for using SKD across different tasks, promoting researchers' extensive attempts at self-distillation technology in their own research fields that have not yet applied self-knowledge distillation.

Wide dynamic range compression (WDRC) provides level-dependent amplification, intended to make the output of a hearing aid fall between the hearing threshold and the highest comfortable level of the listener. Hearing aids often combine noise reduction with WDRC, applied sequentially. Unfortunately, this can result in across-source modulation, especially when fast-acting compression is used. However, fast-acting compression is theoretically preferable to compensate for the loss of compression in the cochlea. To apply fast-acting compression to speech while avoiding across-source modulation, we propose a deep-learning-based method that integrates noise reduction and source-independent WDRC in a two-stage low-complexity framework. The method applies fast-acting compression to speech and slow-acting compression to noise by an amount depending on the audiogram, with a controllable residual noise level. Objective measurements using simulated hearing-impaired listeners showed that the proposed method reduced negative interactions of speech and noise in highly non-stationary noise scenarios and generalized well across various hearing losses.

Self-knowledge distillation does not require a pre-trained teacher network like traditional knowledge distillation. Existing methods either require additional parameters or require additional memory consumption. To alleviate this problem, this paper proposes a more efficient self-knowledge distillation method, named LRMS (learning from role-model samples). In every mini-batch, LRMS selects out a role-model sample for each sampled category, and takes its prediction as the proxy semantic for the corresponding category. Then, predictions of the other samples are constrained to be consistent with the proxy semantics, which makes the distribution of predictions for samples within the same category more compact. Meanwhile, the regularization targets corresponding to proxy semantics are set with a higher distillation temperature to better utilize the classificatory information about the categories. Experimental results show that diverse architectures achieve improvements on four image classification datasets by using LRMS. Code is acaliable: https://github.com/KAI1179/LRMS

Development of interactive web applications to deposit, visualize and analyze biological datasets is a major subject of bioinformatics. R is a programming language for data science, which is also one of the most popular languages used in biological data analysis and bioinformatics. However, building interactive web applications was a great challenge for R users before the Shiny package was developed by the RStudio company in 2012. By compiling R code into HTML, CSS and JavaScript code, Shiny has made it incredibly easy to build web applications for the large R community in bioinformatics and for even non-programmers. Over 470 biological web applications have been developed with R/Shiny up to now. To further promote the utilization of R/Shiny, we reviewed the development of biological web applications with R/Shiny, including eminent biological web applications built with R/Shiny, basic steps to build an R/Shiny application, commonly used R packages to build the interface and server of R/Shiny applications, deployment of R/Shiny applications in the cloud and online resources for R/Shiny.

Internet technologies are making rapid progress, but achieving high performance of network application over the WAN remains a challenge. When branch office's workers access enterprise applications locating in data center, the performances drop dramatically. To solve this problem, many hardware accelerators are produced. In this paper, we introduce a Software-only IP Network Application Accelerator (Soft Accel), for improving network applications performance over the WAN. To overcome the performance-limiting factors associated with WAN, transport protocol and application protocol, Soft Accelcombines application proxy, delta compress and TCPoptimization techniques. We have implemented Soft Accel in the form of framework, so people can design and test their optimization technologies based on it. Experiments in WAN environment show that Soft Accel can improve the performance of IP network applications at least by 2x.

Virtualization provides an effective management in server consolidation. The transparence enables different kinds of servers running in the same platform, making full use of hardware resource. However, virtualization introduces two-level schedulers: one from Guest OS, where the tasks are scheduled to virtual CPUs (VCPUs), the other from the virtual machine monitor (VMM), where VCPUs are scheduled to CPUs. As a result, the lower level scheduler is ignorant of the task information so that it cannot allocate appropriate proportion of CPU resource for every Guest OS in some cases. This paper presents an affinity-aware Dynamic Pinning Scheduling scheduler (DP-Scheduling). We aim at two objects: Bridging the semantic gap between Guest OS and VMM, introducing an affinity-aware method and providing the tasks information about CPU affinity to VMM, Bringing up a novel scheduling, DP-Scheduling, so that VCPU can be pinned or unpinned on one CPU's running queue dynamically. For this purpose, we first get the Machine Address (MA) of process descriptor from the angle of VMM. The affinity information is also acquired before the task is enabled to run. To acknowledge the affinity information, DP-Scheduling calls an API provided by us. Depending on the affinity information, we put forward a series of measures to implement pinning dynamically as well as to keep workload balance. All implementation is confined to Xen VMM and Credit scheduler. Our experiments demonstrate that DP-Scheduling outperforms Credit scheduling by testing various indicators for CPU-bound tasks, without interfering the load balance.

Though with the rapid development, there remains a challenge on achieving high performance of I/O virtualization. The Para virtualized I/O driver domain model, used in Xen, provides several advantages including fault isolation, live migration, and hardware independence. However, the high CPU overhead of driver domain leads to low throughput for high bandwidth links. Direct I/O can achieve high performance but at the cost of removing the benefits of the driver domain model. This paper presents software techniques and optimizations to achieve high throughput network I/Ovirtualization by driver domain virtualization model on multicore systems. In our experiments on multi-core system with a quad-port 1GbE NIC, we observe the overall throughput of multiple guest VMs can only be 2.2Gb/s, while the link bandwidth is 4Gb/s in total. The low performance results from the disability of driver domain to concurrently serve multiple guest VMs running bandwidth-intensive applications. Consequently, two approaches are proposed. First, a multi task let net back is implemented to serve multiple net fronts on currently. Second, we implement a new event channel dispatch mechanism to balance event associated with networkI/O over VCPUs of driver domain. To reduce the CPU overhead of the driver domain model, we also propose two optimizations: lower down event frequency in netback and implement LRO in net front. By applying all the above techniques, our experiments show that the overall throughput can be improved from the original 2.2Gb/s to 3.7Gb/s and the multi-core CPU resources can be utilized efficiently. We believe that the approaches of our study can be valuable for high throughput I/O virtualization in the coming multi-core era.