Contextual semantic information plays a pivotal role in the brain's visual interpretation of the surrounding environment. When processing visual information, electrical signals within synapses facilitate the dynamic activation and deactivation of synaptic connections, guided by the contextual semantic information associated with different objects. In the realm of Artificial Intelligence (AI), neural networks have emerged as powerful tools to emulate complex signaling systems, enabling tasks such as classification and segmentation by understanding visual information. However, conventional neural networks have limitations in simulating the conditional activation and deactivation of synapses, collectively known as the connectome, a comprehensive map of neural connections in the brain. Additionally, the pixel-wise inference mechanism of conventional neural networks failed to account for the explicit utilization of contextual semantic information in the prediction process. To overcome these limitations, we developed a novel neural network, dubbed the Shape Memory Network (SMN), which excels in two key areas: (1) faithfully emulating the intricate mechanism of the brain's connectome, and (2) explicitly incorporating contextual semantic information during the inference process. The SMN memorizes the structure suitable for contextual semantic information and leverages this structure at the inference phase. The structural transformation emulates the conditional activation and deactivation of synaptic connections within the connectome. Rigorous experimentation carried out across a range of semantic segmentation benchmarks demonstrated the outstanding performance of the SMN, highlighting its superiority and effectiveness. Furthermore, our pioneering network on connectome emulation reveals the immense potential of the SMN for next-generation neural networks.

Deep learning-based scene recognition algorithms have been developed for real-time application in indoor localization systems. However, owing to the slow calculation time resulting from the deep structure of convolutional neural networks , deep learning-based algorithms have limitations in the usage of real-time applications, despite their high accuracy in classification tasks . To significantly reduce the computation time of these algorithms and slightly improve their accuracy, we thus propose a path-selective deep learning network, denoted as Selective Optimal Network (SoN). The SoN selectively uses the depth-variable networks depending on a new indicator, denoted as the classification-complexity of a source image. The SoN reduces the prediction time by selecting optimal depth for the baseline networks corresponding to the input samples. The network was evaluated using two public datasets and two custom datasets for indoor localization and scene classification, respectively. The experimental results indicated that, compared to other deep learning models, the SoN exhibited improved accuracy and enhanced the processing speed by up to 78.59%. Additionally, the SoN was applied to a smartphone-based indoor positioning system in real-time. The results indicated that the SoN shows excellent performance for rapid and accurate classification in real-time applications of indoor localization systems.

For several decades, the significance of geospatial object segmentation in remote sensing (RS) images has been emphasized for both scientific and industrial purposes. Object segmentation plays a pivotal role in the analysis of urban and rural area expansion, as well as in advancing sustainable development within the realm of RS. Deep learning (DL)-based segmentation methodologies, overcoming the limitations of the conventional vision-based analysis, have yielded precise predictions by utilizing convolutional neural networks (CNNs). However, CNNs classify images at the pixel level and generate outputs based on probability distributions derived from the SoftMax function. This approach precludes the reflection of morphological properties, such as shape and object density, during predictions in RS imagery, leading to imprecise results. In addition, due to the intrinsic attributes of probability-based segmentation, fine-grained segmentation may not be achieved, leading to coarse predictions in the boundaries of geospatial objects. To address this issue, this article introduces a novel DL framework, the density-based guide network (DG-Net), which incorporates the density of segmentation targets into pixel-wise classification through a test-time adaptation learning methodology. DG-Net first discerns the density of segmentation targets in the input images, then fine-tunes the baseline network to reflect this density, thereby generating precise segmentation outputs. The effectiveness of DG-Net is demonstrated through various multitarget segmentation benchmarks in RS imagery. Experimental results demonstrate the superior performance of the DG-Net in object segmentation when compared to state-of-the-art (SotA) models across numerous aerial image and satellite image datasets.

Deep learning (DL) techniques for precise semantic segmentation have remained a challenge because of the vague boundaries of target objects caused by the low resolution of images. Despite the improved segmentation performance using up/downsampling operations in early DL models, conventional operators cannot fully preserve spatial information and thus generate vague boundaries of target objects. Therefore, for the precise segmentation of target objects in many domains, this paper presents two novel operators: (1) upsampling interpolation method (USIM), an operator that upsamples input feature maps and combines feature maps into one while preserving the spatial information of both inputs, and (2) USIM gate (UG), an advanced USIM operator with boundary-attention mechanisms. We designed our experiments using aerial images where the boundaries critically influence the results. Furthermore, we verified the feasibility that our approach effectively segments target objects using the cityscapes dataset. The experimental results demonstrate that using the USIM and UG with state-of-the-art DL models can improve the segmentation performance with clear boundaries of target objects (Intersection over Union: +6.9$%$; BJ: +10.1$%$). Furthermore, mathematical proofs verify that the USIM and UG contribute to the handling of spatial information.

Fluorescence imaging-based diagnostic systems have been widely used to diagnose skin diseases due to their ability to provide detailed information related to the molecular composition of the skin compared to conventional RGB imaging. In addition, recent advances in smartphones have made them suitable for application in biomedical imaging, and therefore various smartphone-based optical imaging systems have been developed for mobile healthcare. However, an advanced analysis algorithm is required to improve the diagnosis of skin diseases. Various deep learning-based algorithms have recently been developed for this purpose. However, deep learning-based algorithms using only white-light reflectance RGB images have exhibited limited diagnostic performance. In this study, we developed an auxiliary deep learning network called fluorescence-aided amplifying network (FAA-Net) to diagnose skin diseases using a developed multi-modal smartphone imaging system that offers RGB and fluorescence images. FAA-Net is equipped with a meta-learning-based algorithm to solve problems that may occur due to the insufficient number of images acquired by the developed system. In addition, we devised a new attention-based module that can learn the location of skin diseases by itself and emphasize potential disease regions, and incorporated it into FAA-Net. We conducted a clinical trial in a hospital to evaluate the performance of FAA-Net and to compare various evaluation metrics of our developed model and other state-of-the-art models for the diagnosis of skin diseases using our multi-modal system. Experimental results demonstrated that our developed model exhibited an 8.61% and 9.83% improvement in mean accuracy and area under the curve in classifying skin diseases, respectively, compared with other advanced models.

The simulation of human neurons and neurotransmission mechanisms has been realized in deep neural networks based on the theoretical implementations of activation functions. However, recent studies have reported that the threshold potential of neurons exhibits different values according to the locations and types of individual neurons, and that the activation functions have limitations in terms of representing this variability. Therefore, this study proposes a simple yet effective activation function that facilitates different thresholds and adaptive activations according to the positions of units and the contexts of inputs. Furthermore, the proposed activation function mathematically exhibits a more generalized form of Swish activation function, and thus we denoted it as Adaptive SwisH (ASH). ASH highlights informative features that exhibit large values in the top percentiles in an input, whereas it rectifies low values. Most importantly, ASH exhibits trainable, adaptive, and context-aware properties compared to other activation functions. Furthermore, ASH represents general formula of the previously studied activation function and provides a reasonable mathematical background for the superior performance. To validate the effectiveness and robustness of ASH, we implemented ASH into many deep learning models for various tasks, including classification, detection, segmentation, and image generation. Experimental analysis demonstrates that our activation function can provide the benefits of more accurate prediction and earlier convergence in many deep learning applications.

Various deep learning-based segmentation models have been developed to segment buildings in aerial images. However, the segmentation maps predicted by the conventional convolutional neural network-based methods cannot accurately determine the shapes and boundaries of segmented buildings. In this article, to improve the prediction accuracy for the boundaries and shapes of segmented buildings in aerial images, we propose the boundary-oriented binary building segmentation model (B3SM). To construct the B3SM for boundary-enhanced semantic segmentation, we present two-scheme learning (Schemes I and II), which uses the upsampling interpolation method (USIM) as a new operator and a boundary-oriented loss function (B-Loss). In Scheme I, a raw input image is processed and transformed into a presegmented map. In Scheme II, the presegmented map from Scheme I is transformed into a more fine-grained representation. To connect these two schemes, we use the USIM operator. In addition, the novel B-Loss function is implemented in B3SM to extract the features of the boundaries of buildings effectively. To perform quantitative evaluation of the shapes and boundaries of segmented buildings generated by B3SM, we develop a new metric called the boundary-oriented intersection over union (B-IoU). After evaluating the effectiveness of two-scheme learning, USIM, and B-Loss for building segmentation, we compare the performance of B3SM to those of other state-of-the-art methods using public and custom datasets. The experimental results demonstrate that the B3SM outperforms other state-of-the-art models, resulting in more accurate shapes and boundaries for segmented buildings in aerial images.

The domain-adaptive semantic segmentation of aerial images using a deep-learning technique is still challenging owing to the domain gaps between aerial images obtained in different areas. Currently, various convolutional neural network (CNN)-based domain adaptation methods have been developed to decrease the domain gaps. However, they still show poor performance for object segmentation when they are applied to images from other domains. In this paper, we propose a novel CNN-based self-mutating network (SMN), which can adaptively adjust the parameter values of convolutional filters as a response to the domain of an input image for better domain-adaptive segmentation. For the SMN, the parameter mutation technique was devised for adaptively changing parameters, and a parameterfluctuationtechniquewasdevelopedtorandomlyconvulsetheparameters. By adopting the parameter mutation and fluctuation, adaptive self-changing and fine-tuning of parameters can be realized for images from different domains, resulting in better prediction in domain-adaptive segmentation. Meanwhile, the results of the ablation study indicate that the SMN provided 11.19% higher Intersection over Union values than other state-of-the-art methods, demonstrating its potential for the domain-adaptive segmentation of aerial images.

Semantic segmentation models based on convolutional neural networks (CNNs) have gained much attention in relation to remote sensing and have achieved remarkable performance for the extraction of buildings from high-resolution aerial images. However, the issue of limited generalization for unseen images remains. When there is a domain gap between the training and test data sets, the CNN-based segmentation models trained by a training data set fail to segment buildings for the test data set. In this article, we propose segmentation networks based on a domain adaptive transfer attack (DATA) scheme for building extraction from aerial images. The proposed system combines the domain transfer and the adversarial attack concepts. Based on the DATA scheme, the distribution of the input images can be shifted to that of the target images while turning images into adversarial examples against a target network. Defending adversarial examples adapted to the target domain can overcome the performance degradation due to the domain gap and increase the robustness of the segmentation model. Cross-data set experiments and ablation study are conducted for three different data sets: the Inria aerial image labeling data set, the Massachusetts building data set, and the WHU East Asia data set. Compared with the performance of the segmentation network without the DATA scheme, the proposed method shows improvements in the overall intersection over union (IoU). Moreover, it is verified that the proposed method outperforms even when compared with feature adaptation (FA) and output space adaptation (OSA).

We developed a forward-looking (FL) multimodal endoscopic system that offers color, spectral classified, high-frequency ultrasound (HFUS) B-mode, and integrated backscattering coefficient (IBC) images for tumor detection in situ. Examination of tumor distributions from the surface of the colon to deeper inside is essential for determining a treatment plan of cancer. For example, the submucosal invasion depth of tumors in addition to the tumor distributions on the colon surface is used as an indicator of whether the endoscopic dissection would be operated. Thus, we devised the FL multimodal endoscopic system to offer information on the tumor distribution from the surface to deep tissue with high accuracy. This system was evaluated with bilayer gelatin phantoms which have different properties at each layer of the phantom in a lateral direction. After evaluating the system with phantoms, it was employed to characterize forty human colon tissues excised from cancer patients. The proposed system could allow us to obtain highly resolved chemical, anatomical, and macro-molecular information on excised colon tissues including tumors, thus enhancing the detection of tumor distributions from the surface to deep tissue. These results suggest that the FL multimodal endoscopic system could be an innovative screening instrument for quantitative tumor characterization.

Extracting manufactured features such as buildings, roads, and water from aerial images is critical for urban planning, traffic management, and industrial development. Recently, convolutional neural networks (CNNs) have become a popular strategy to capture contextual features automatically. In order to train CNNs, a large training data are required, but it is not straightforward to use free-accessible data sets due to imperfect labeling. To address this issue, we make a large scale of data sets using RGB aerial images and convert them to digital maps with location information such as roads, buildings, and water from the metropolitan area of Seoul in South Korea. The numbers of training and test data are 72 400 and 9600, respectively. Based on our self-made data sets, we design a multiobject segmentation system and propose an algorithm that utilizes pyramid pooling layers (PPLs) to improve U-Net. Test results indicate that U-Net with PPLs, called UNetPPL, learn fine-grained classification maps and outperforms other algorithms of fully convolutional network and U-Net, achieving the mean intersection of union (mIOU) of 79.52 and the pixel accuracy of 87.61% for four types of objects (i.e., building, road, water, and background).