Multispectral homogeneous bands capture distinct and complementary spectral characteristics, therefore, fusing multiple bands has the potential to increase semantic segmentation performance. However, the fusion of highly correlated homogeneous bands (i.e. RGB, NIR, and SWIR) remains underexplored. We hypothesized that using correlation representations between highly correlated homogeneous spectral bands at higher‑level feature stages may improve segmentation accuracy. Therefore, we propose a novel semantic segmentation architecture that combines homogeneous modalities with a shared latent representation that exploits their intrinsic correlations. We also introduce interactive feature fusion blocks at early encoder stages to extract better cross-band correlations. Our experiments on two different remote sensing image sets, both UAV-based and satellite-based, show that our correlation‑driven fusion among homogeneous bands can enhance segmentation accuracy over state-of-the-art unimodal and multimodal models. Our code is available at: https://github.com/iremulku/CorrIFNet.

Plant health can be monitored dynamically using multispectral sensors that measure near-infrared (NIR). Despite this potential, obtaining and annotating high-resolution NIR images pose significant challenges for training deep neural networks. Typically, large networks pretrained on the RGB domain are utilized to fine-tune infrared images. This practice introduces a domain shift issue because of the differing visual traits between RGB and NIR images. As an alternative to fine-tuning, a method called low-rank adaptation (LoRA) enables more efficient training by optimizing rank-decomposition matrices while keeping the original network weights frozen. However, the existing parameter-efficient adaptation strategies for remote sensing images focus on RGB images and overlook domain shift issues in the NIR domain. Therefore, this study investigates the potential benefits of using vision transformer (ViT) backbones pretrained in the RGB domain, with LoRA for downstream tasks in the NIR domain. Extensive experiments demonstrate that employing LoRA with pretrained ViT backbones yields the best performance for downstream tasks applied to NIR images.

Drone detection has become an essential task in object detection as drone costs have decreased and drone technology has improved. It is, however, difficult to detect distant drones when there is weak contrast, long range, and low visibility. In this work, we propose several sequence classification architectures to reduce the detected false-positive ratio of drone tracks. Moreover, we propose a new drone vs. bird sequence classification dataset to train and evaluate the proposed architectures. 3D CNN, LSTM, and Transformer based sequence classification architectures have been trained on the proposed dataset to show the effectiveness of the proposed idea. As experiments show, using sequence information, bird classification and overall F1 scores can be increased by up to 73% and 35%, respectively. Among all sequence classification models, R(2+1)D-based fully convolutional model yields the best transfer learning and fine-tuning results.

Tracking objects can be a difficult task in computer vision, especially when faced with challenges such as occlusion, changes in lighting, and motion blur. Recent advances in deep learning have shown promise in challenging these conditions. However, most deep learning-based object trackers only use visible band (RGB) images. Thermal infrared electromagnetic waves (TIR) can provide additional information about an object, including its temperature, when faced with challenging conditions. We propose a deep learning-based image tracking approach that fuses RGB and thermal images (RGBT). The proposed model consists of two main components: a feature extractor and a tracker. The feature extractor encodes deep features from both the RGB and the TIR images. The tracker then uses these features to track the object using an enhanced attribute-based architecture. We propose a fusion of attribute-specific feature selection with an aggregation module. The proposed methods are evaluated on the RGBT234 [1] and LasHeR [2] datasets, which are the most widely used RGBT object-tracking datasets in the literature. The results show that the proposed system outperforms state-of-the-art RGBT object trackers on these datasets, with a relatively smaller number of parameters.

This study focuses on pixel-wise semantic segmentation of crop production regions by using satellite remote sensing multispectral imagery. One of the principal aims of the study is to find out whether the raw multiple channel inputs are more effective in the training process of the semantic segmentation models or if the formularized counterparts as the spectral indices are more effective. For this purpose, the vegetation indices NDVI, ARVI and SAVI and the water indices NDWI, NDMI, and WRI are employed as inputs. Additionally, using 8, 10 and 16 channels, multiple channel inputs are utilized. Moreover, all spectral indices are taken as separate channels to form a multiple channel input. We conduct deep learning experiments using two semantic segmentation architectures, namely U-Net and DeepLabV3+. Our results show that, in general, feeding raw multiple channel inputs to semantic segmentation models performs much better than feeding the spectral indices. Hence, regarding crop production region segmentation, deep learning models are capable of encoding multispectral information. When the spectral indices are compared among themselves, ARVI, which reduces the atmospheric scattering effects, achieves better accuracy for both architectures. The results also reveal that spatial resolution of multispectral data has a significant effect on the semantic segmentation performance, and therefore the RGB band, which has the lowest ground sample distance (0.31 m) outperforms multispectral bands and shortwave infrared bands.

In this study, we aim at designing a smart glove, which consists of different inertial sensors and an EMG sensor and developing a human-machine interaction application by pre-processing and fusing these different sensory data. We also aim at providing solutions in cases where image processing-based approaches are inefficient. In the proposed smart glove, the quaternion-based orientation data to be produced by the magnetometer and gyroscope together, the acceleration data to be generated by the accelerometer, and the analog data generated by the EMG sensor are collected and then prepared for use by different applications.

Memorability of an image is a characteristic determined by the human observers' ability to remember images they have seen. Yet recent work on image memorability defines it as an intrinsic property that can be obtained independent of the observer. The current study aims to enhance our understanding and prediction of image memorability, improving upon existing approaches by incorporating the properties of cumulative human annotations. We propose a new concept called the Visual Memory Schema (VMS) referring to an organization of image components human observers share when encoding and recognizing images. The concept of VMS is operationalised by asking human observers to define memorable regions of images they were asked to remember during an episodic memory test. We then statistically assess the consistency of VMSs across observers for either correctly or incorrectly recognised images. The associations of the VMSs with eye fixations and saliency are analysed separately as well. Lastly, we adapt various deep learning architectures for the reconstruction and prediction of memorable regions in images and analyse the results when using transfer learning at the outputs of different convolutional network layers.

Hyper-spectral satellite imagery, consisting of multiple visible or infrared bands, is extremely dense and weighty for deep operations. Regarding problems related to vegetation as, more specifically, tree segmentation, it is difficult to train deep architectures due to lack of large-scale satellite imagery. In this paper, we compare the success of different single channel indices, which are constructed from multiple bands, for the purpose of tree segmentation in a deep convolutional neural network (CNN) architecture. The utilized indices are either hand-crafted such as excess green index (ExG) and normalized difference vegetation index (NDVI) or reconstructed from the visible bands using feature space transformation methods such as principle component analysis (PCA). For comparison, these features are fed to an identical CNN architecture, which is a standard U-Net-based symmetric encoder-decoder design with hierarchical skip connections and the segmentation success for each single index is recorded. Experimental results show that single bands, which are constructed from the vegetation indices and space transformations, can achieve similar segmentation performances as compared to that of the original multi-channel case.

A novel method to detect human falls in depth videos is presented in this paper. A fast and robust shape sequence descriptor, namely the Silhouette Orientation Volume (SOV), is used to represent actions and classify falls. The SOV descriptor provides high classification accuracy even with a combination of simple associated models, such as Bag-of-Words and the Naïve Bayes classifier. Experiments on the public SDU-Fall dataset show that this new approach achieves up to 91.89% fall detection accuracy with a single-view depth camera. The classification rate is about 5% higher than the results reported in the literature. An overall accuracy of 89.63% was obtained for the six-class action recognition, which is about 25% higher than the state of the art. Moreover, a perfect silhouette-based action recognition rate of 100% is achieved on the Weizmann action dataset.

In this study, a scale‐invariant representation for closed planar curves (silhouettes) is proposed. The orientations of all points within the Gaussian scale‐space of the curve are extracted. This orientation scale‐space is used to create the silhouette orientation image in which the positions of each pixel indicate the curve's pixel positions and scales, whereas the colour represents orientation. The representation is extracted for multiple levels of the morphological scale‐space of the silhouette. The proposed representation is invariant to scale and transformable under planar rotation. Using linear and non‐linear distance learning methods, experiments on the MPEG7, ETH80 and Kimia shape datasets were conducted, with results indicating an advanced recognition capability.

ABSTRACT Biomechanical modeling of soft tissue is a complex problem for achieving realistic surgical simulations, surgical planning, and scientific analysis. In the literature, three categories of biomechanical models: spline based models, spring models, and finite element models (FEMs) are mainly used for dealing with this problem. Among these, spline based models offer relatively fast and realistic soft tissue simulations by utilizing both the spring and FEMs. In this paper, a new dynamic volume spline model for human face skin is proposed and the performance of our model is discussed by estimating the results of facial surgery of three different patients. Face models of the patients are obtained from 3D CT/MR scans by segmenting the skull, muscle, and skin layers. In these face models, the skull and the muscle layers are considered as the rigid boundary for the skin layer and the skin layer is modeled by our dynamic volume spline. The control points of the dynamic volume spline are localized masses with viscoelastic material properties (stiffness, damping, and mass). These parameters are computed from the skin material properties that were published in the literature. Once the face models are generated, facial surgery plannings are simulated. Infact, the pre‐surgery face models are modified according to the surgical plans and the estimated post‐surgery face models are compared with the actual post‐surgery face models. Moreover, in order to discuss the performance of our dynamic volume spline model, the same analyses are performed on the post‐surgery estimations of a conventional tool. Copyright © 2011 John Wiley & Sons, Ltd.

Using mean curvature (H) and Gaussian curvature (K) values or shape index (S) and curvedness (C) values, HK and SC curvature spaces are constructed in order to classify surface patches into types such as pits, peaks, saddles etc. Since both HK and SC curvature spaces classify surface patches in to similar types, their classification capabilities are comparable. Previously, HK and SC curvature spaces were compared in terms of their classification ability only at the given data resolution [2]. When calculating H, K, C and S values, the scale/resolution ratio is highly effective. However, due to its scale invariant nature, shape index (S) values are independent of the resolution or the scale. Thus it is no wonder that SC method gives better results than HK method when the comparison is carried out at an uncontrolled scale/resolution level. In this study, the scale/resolution ratio is set to a constant value for the whole database and scale spaces based on both HK and SC methods are built. Scale and orientation invariant features are extracted using scale spaces and these features are used in object recognition tasks. The methods are compared both mathematically and experimentally in terms of their surface classification and object recognition performances.

Finding slope units for a given watershed is an important task before analyzing landslide susceptibility. Usually, slope unit generation requires integration of several digital elevation model (DEM)-based outputs obtained from GIS and related hydrological software. Therefore, it is time consuming due to involved steps and compilation of various software outputs. In this paper a DEM-based, scale and resolution invariant, fully automatic method is proposed for creating slope units to be used in landslide susceptibility mapping.

An algorithm is proposed to extract transformation and scale invariant 3D fundamental elements from the surface structure of 3D range scan data. The surface is described by mean and Gaussian curvature values at every data point at various scales and a scale-space search is performed in order to extract the fundamental structures and to estimate the location and the scale of each fundamental structure. The extracted fundamental structures can later be used as nodes in a topological graph where the links between the nodes can be defined as the spatial and geometric relations between the fundamental elements.

An algorithm is proposed for 3D object representation using generic 3D features which are transformation and scale invariant. Descriptive 3D features and their relations are used to construct a graphical model for the object which is later trained and then used for detection purposes. Descriptive 3D features are the fundamental structures which are extracted from the surface of the 3D scanner output. This surface is described by mean and Gaussian curvature values at every data point at various scales and a scale-space search is performed in order to extract the fundamental structures and to estimate the location and the scale of each fundamental structure.