Accurately extracting buildings from high-resolution remote sensing images is crucial for urban planning, smart city construction, map updating, and other fields. However, the buildings extracted from remote sensing images still have deficiencies such as fuzzy boundaries and poor integrity due to the occlusion of trees and shadows, the interference of redundant information, and the characteristics of buildings with different shapes and scales. In this paper, a novel building extraction model EGAFNet is proposed for remote sensing images, which utilizes EfficientNetV2 as the encoder and multiscale feature enhancement module as the decoder to improve the accuracy and efficiency. To address the issue of missing boundary information, a branch for extract boundary features and an edge guidance module are constructed to enhance the network’s ability to express the boundary. In addition, a scale-aware adaptive fusion module is designed to adaptively aggregate the multi-scale features to enhance the network’s ability to capture the different scale features. We also adopt a multi-layer supervision strategy that predict the outputs for each layer of the decoding phase and impose multi-level loss constraints to achieve fast fitting of the model. Experiments are conducted on the WHU building dataset and a dataset of building instances of typical cities in China. The experimental results demonstrate that the EGAFNet achieves the best F1-score and IoU compared with other typical building extraction methods. It suggests that EGAFNet can effectively improve the learning ability of the boundary information and enhance the multi-scale building feature representation, thus realizing more accurate building extraction. The code will be available at https://github.com/Mwyang/EGAFNet.

The concepts of conditional mutual information (CMI) and normalized CMI (NCMI) are introduced to measure the concentration and separation performance of a classification deep neural network (DNN) in the output probability distribution space of the DNN, where CMI and the ratio between CMI and NCMI represent the intraclass concentration and interclass separation of the DNN, respectively. By using NCMI to evaluate popular DNNs pretrained over CIFAR-100 and ImageNet in the literature, it is shown that their validation accuracies are more or less inversely proportional to their NCMI values. Based on this observation, the standard deep learning (DL) framework is further modified to minimize the standard cross entropy (CE) function subject to an NCMI constraint, yielding CMI constrained DL (CMIC-DL). A novel alternating learning algorithm is proposed to solve such a constrained optimization problem. Extensive experimental results show that DNNs trained within CMIC-DL outperform the state-of-the-art models trained within the standard DL and other loss functions in the literature in terms of both accuracy and robustness against adversarial attacks. In addition, visualizing the evolution of the learning process through the lens of CMI and NCMI is also advocated.

Deep neural networks (DNNs) could be deceived by generating human-imperceptible perturbations of clean samples. Therefore, enhancing the robustness of DNNs against adversarial attacks is a crucial task. In this paper, we aim to train robust DNNs by limiting the set of outputs reachable via a norm-bounded perturbation added to a clean sample. We refer to this set as adversarial polytope, and each clean sample has a respective adversarial polytope. Indeed, if the respective polytopes for all the samples are compact such that they do not intersect the decision boundaries of the DNN, then the DNN is robust against adversarial samples. Hence, the inner-working of our algorithm is based on learning confined adversarial polytopes (CAP). By conducting a thorough set of experiments, we demonstrate the effectiveness of CAP over existing adversarial robustness methods in improving the robustness of models against state-of-the-art attacks including AutoAttack.

It is believed that in knowledge distillation (KD), the role of the teacher is to provide an estimate for the unknown Bayes conditional probability distribution (BCPD) to be used in the student training process. Conventionally, this estimate is obtained by training the teacher using maximum log-likelihood (MLL) method. To improve this estimate for KD, in this paper we introduce the concept of conditional mutual information (CMI) into the estimation of BCPD and propose a novel estimator called the maximum CMI (MCMI) method. Specifically, in MCMI estimation, both the log-likelihood and CMI of the teacher are simultaneously maximized when the teacher is trained. In fact, maximizing the teacher's CMI value ensures that the teacher can effectively capture the contextual information within the images, and for visualizing this information, we deploy Eigen-CAM. Via conducting a thorough set of experiments, we show that by employing a teacher trained via MCMI estimation rather than one trained via MLL estimation in various state-of-the-art KD frameworks, the student's classification accuracy consistently increases, with the gain of up to 3.32\%. This suggests that the teacher's BCPD estimate provided by MCMI method is more accurate than that provided by MLL method. In addition, we show that such improvements in the student's accuracy are more drastic in zero-shot and few-shot settings. Notably, the student's accuracy increases with the gain of up to 5.72\% when 5\% of the training samples are available to student (few-shot), and increases from 0\% to as high as 84\% for an omitted class (zero-shot).

Federated learning (FL) is a promising technology wherein edge devices/clients collaboratively train a machine learning model under the orchestration of a central server. However, due to the inherent data heterogeneity among clients, local datasets on individual clients often exhibit class imbalance, i.e., samples from majority classes vastly outnumber those from minority classes. This imbalance significantly diminishes the performance of the trained model. To understand why, we first closely examine the output probability distribution clusters of the local deep neural networks (DNNs) in the probability space over the label set, and observe that for class imbalanced datasets, FL has two interesting phenomena: (1) dispersion problem-clusters corresponding to minority classes tend to disperse; and (2) gravity problem-clusters corresponding to minority classes are drawn toward those of majority classes. To overcome these two problems, we then introduce information quantities into FL, propose a new information theoretic loss function for FL, and develop a new FL framework called Fed-IT. It is shown that Fed-IT significantly outperforms previous counterparts, while maintaining client privacy.

In this paper, we introduce the notions of conditional mutual information (CMI) and normalized conditional mutual information (NCMI) for classification deep neural networks (DNNs). In particular, CMI and the ratio between CMI and NCMI quantify the intra-class concentration and inter-class separation of a DNN in its output probability distribution space, respectively. Utilizing NCMI to assess widely recognized DNNs pre-trained on ImageNet reveals a notable inverse relationship between their validation accuracies and NCMI values on the ImageNet validation dataset. Building upon this insight, the conventional deep learning (DL) framework is modified by minimizing the standard cross-entropy function while imposing an NCMI constraint. This refinement results in a novel approach known as CMI-constrained deep learning (CMIC-DL). Comprehensive experimental findings demonstrate that DNNs trained using CMIC-DL outperform state-of-the-art models trained within standard DL and other loss functions in the literature. In addition, some semantic meaning of CMI is also discovered.

Conventional image compression techniques are mostly developed for the human visual system. However, with the extensive use of deep neural networks (DNNs), more and more images will be consumed by DNN-based intelligent machines, which makes it crucial to develop image compression techniques customized for DNN vision while being JPEG compliant. In this paper, we first propose a new distortion measure, dubbed the sensitivity weighted error (SWE). Then, we develop OptS, a DNN-oriented compression algorithm with full JPEG compatibility, which designs optimal quantization tables for DNN models based on SWE. To test the performance of our algorithm, experiments of image classification are conducted on the ImageNet dataset for two prevailing DNN models. Results demonstrate that our algorithm achieves better rate-accuracy (R-A) performance than the default JPEG. For some DNN model, the compression ratio of our algorithm can reach 8.3×1, reducing the compression rate (bits per pixel, bpp) of the default JPEG by 57.4% with no accuracy loss. Our source code is available at https://github.com/zkxufo/OptS.git.

Considering the ability to perform multi-processor architecture systems on FPGA, partial reconfiguration is an opportunity to improve weak soft-core performances by specializing coprocessors according to context-dependent application needs. But at the application level, there is a need for straightforward programming models that allow applications to be easily mapped on an ad hoc architecture without tedious rewriting, while at the same time ensuring efficient production code. In this paper we describe two programming libraries XTask and XFunc, which are written in C and rely on a reconfigurable MPSoC architecture model (XPSoC) and on HW/SW libraries of standard functions that can be easily used by means of HW independent API. Finally, we demonstrate the XPSoC methodology, with the design of a self-adaptive image encoding system including runtime configuration decisions.