Recent online knowledge distillation (OKD) methods focus on capturing rich and useful intermediate information by performing multi-layer feature learning. Existing works only consider intermediate layer feature maps between the same layers and ignore valuable information across layers, which results in the lack of appropriate cross-layer supervision in detail and the process of learning. Besides, this manner provides insufficient supervision information to supervise the learning of student, since it fails to construct a qualified teacher. In this work, we propose a Deep Cross-layer Collaborative Learning network (DCCL) for OKD, which efficiently exploits fruitful knowledge of peer student models by keeping appropriate intermediate cross-layer supervision. Specifically, each student gradually integrates its own features at different layers for feature matching, so as to effectively utilize features in low and high levels for learning more composite knowledge. Moreover, we assign a collaborative knowledge learning strategy, in which a qualified teacher is established via fusing the features of last convolution layers for enhancing high-level representation. In this way, all student models continuously transfer the rich teacher’s internal representation as well as capture its dynamic growth process, and in turn assist the learning of the fusion teacher to further supervise students. In the experiments, our proposed DCCL has shown great generalization ability with various backbone models on CIFAR-100, Tiny ImageNet and ImageNet, and also demonstrated superior performance against mainstream OKD works. Our code is available here:https://github.com/nanxiaotong/DCCL.

Knowledge distillation (KD) transfers discriminative knowledge from a large and complex model (known as teacher) to a smaller and faster one (known as student). Existing advanced KD methods, limited to fixed feature extraction paradigms that capture teacher's structure knowledge to guide the training of the student, often fail to obtain comprehensive knowledge to the student. Toward this end, in this article, we propose a new approach, synchronous teaching knowledge distillation (STKD), to integrate online teaching and offline teaching for transferring rich and comprehensive knowledge to the student. In the online learning stage, a blockwise unit is designed to distill the intermediate-level knowledge and high-level knowledge, which can achieve bidirectional guidance of the teacher and student networks. Intermediate-level information interaction provides more supervisory information to the student network and is useful to enhance the quality of final predictions. In the offline learning stage, the STKD approach applies a pretrained teacher to further improve the performance and accelerate the training process by providing prior knowledge. Trained simultaneously, the student learns multilevel and comprehensive knowledge by incorporating online teaching and offline teaching, which combines the advantages of different KD strategies through our STKD method. Experimental results on the SVHN, CIFAR-10, CIFAR-100, and ImageNet ILSVRC 2012 real-world datasets show that the proposed method achieves significant performance improvements compared with the state-of-the-art methods, especially with satisfying accuracy and model size. Code for STKD is provided at https://github.com/nanxiaotong/STKD.

General anomaly detection techniques have always received a lot of attention. Current detection methods usually focus solely on representation learning or anomaly judgment. This paper proposes a Multi-Subspace Deviation Network (MSDN) framework to build a model combining feature learning with anomaly score learning under the condition that a small number of labeled anomalies can be observed. Concretely, our framework combines a feature learner with two specific projectors: a self-supervised projector and an anomaly score learner. We utilize random affine transformations to map the raw data to multiple subspaces and train a classifier to predict the transformation label in the self-supervised module. Anomaly scores are then obtained directly from a deviation network, where the contrastive loss is used to amplify the gap in the anomaly scores between normal objects and anomalies. Extensive experiments on eight datasets show that our proposed method achieves higher detection accuracy than previous schemes with fewer observed anomalies.

Existing online knowledge distillation (KD) methods solve the dependency problem of the high-capacity teacher model via mutual learning and ensemble learning. But they focus on the utilization of logits information in the last few layers and fail to construct a strong teacher model to better supervise student networks, leading to the inefficiency of KD. In this work, we propose a simple but effective online knowledge distillation algorithm, called Attentive Feature Interaction Distillation (AFID). It applies interactive teaching in which the teacher and the student can send, receive, and give feedback on an equal footing, ultimately promoting the generality of both. Specifically, we set up a Feature Interaction Module for two sub-networks to conduct low-level and mid-level feature learning. They can alternately transfer attentive features maps to exchange interesting regions and fuse the other party’s map with the features of self-extraction for information enhancement. Besides, we assign a Feature Fusion Module, in which a Peer Fused Teacher is formed to fuse the output features of two sub-networks to guide sub-networks and a Peer Ensemble Teacher is established to accomplish mutual learning between the two teachers. Integrating Feature Interaction Module and Feature Fusion Module into a unified framework takes full advantage of the interactive teaching mechanism and makes the two sub-networks capture and transfer more fine-grained features to each other. Experimental results on CIFAR-100 and ImageNet ILSVRC 2012 real datasets show that AFID achieves significant performance improvements compared with existing online KD and classical teacher-guide methods.

The emergence of edge computing eases latency pressure in remote cloud and computing pressure of terminal devices, providing new solutions for real-time applications. Jobs of end devices are offloaded to a server in the cloud or an edge cluster for execution. Unreasonable job dispatching strategies will not only affect the completion time of tasks violating the users’ QoS but also reduce the resource utilization of servers increasing the operating costs of service providers. In this paper, we propose an online job dispatching model named Drag-JDEC based on deep reinforcement learning and graph neural network. For natural directed acyclic graph-type jobs, we use a graph attention network to aggregate the features of neighbor nodes and transform them into high-dimensional ones. Combining with the current status of edge servers, the deep reinforcement learning module makes the dispatching decision for each task in the job to keep load balancing and meet the users’ QoS. Experiments using real job data sets show that Drag-JDEC outperforms traditional and state-of-the-art algorithms for balancing the workload of edge servers and adapts to various edge server configurations well, reaching the maximum improvement of 34.43%.

In the era of cloud computing, the over-occupancy of data center resources (CPU, memory, disk) and subsequent machine failure have resulted in great loss to users and enterprises. So it makes sense to anticipate the server workload in advance. Previous research on server workloads has focused on trend analysis and time series fitting. We propose an approach to forecast the workloads of servers based on machine learning. And our data comes from a database-based data center that is real, large-scale, and enterprise-class. We use the servers' historical monitoring data for our models to predict future workloads and hence provide the ability to automatically warn overload and reallocate resources. We calculate the failure detection rate and false alarm rate of our overload detection models, as well as put forward an evaluation based on the overload processing cost. Experimental results show that machine learning methods especially Random Forest can better predict the server load than traditional time series analysis method. We use the forecast results to propose some scheduling strategies to prevent server overload, achieve intelligent operation and maintenance, and failure prediction. Compared with the traditional time series analysis method, our method uses less data and lower dimensions, and yields more accurate predictions.

Blockchain is a new decentralized and distributed network applying technologies such as P2P network, cryptography and so on, in which P2P lays the foundation. Various consensus mechanisms, which correspond to trust models in P2P, solve the trust problems caused due to the anonymity of nodes. In addition, different incentive models are established in the P2P and blockchain network to encourage nodes to share resources subjectively, thereby improving the contribution of nodes to the system. Based on the analysis of trust and incentive models in P2P, this paper summarizes the consensus and incentive mechanisms of blockchain network. What's more, we indicate the future prospects in blockchain including the improvement of blockchain derived from P2P, the idea of a coin-free blockchain, the problems faced by blockchain, issues about blockchain storage and blockchain application scenarios, which will provide guidance for the future study of blockchain network.