Fine-grained leaf image retrieval (FGLIR) is an emerging content-based image retrieval (CBIR) task that distinguishes cultivars of the same plant species. It remains highly challenging due to subtle inter-cultivar variations and the absence of supervision. Existing training-free methods generally treat convolutional features as local descriptors for aggregation, but they often overlook spatial correlations and structural cues. To address this gap, we extend Local Binary Patterns (LBP) from the raw image domain to the deep feature domain for the first time, enabling explicit modeling of spatial relationships. Specifically, each leaf image is partitioned into R-Angle patches to preserve invariant traits, which are processed by pre-trained CNNs to produce deep feature maps. We then propose the Local Deep-Binary Pattern Transform (LDbPT) and Local Deep Contrast-Pattern Transform (LDcPT) to enhance spatial and contrast encoding, together with anR-Angle Grouped Average Pooling strategy to embed shape information into feature aggregation. Furthermore, we hybridize CNNs with Vision Transformers (ViTs) within this framework, effectively combining local structural extraction with global contextual reasoning. Extensive experiments on two public FGLIR benchmarks with comprehensive evaluations show that our method achieves state-of-the-art retrieval accuracy while demonstrating robustness, efficiency, and strong generalization, confirming the effectiveness of the proposed training-free hybrid representation.

Crack segmentation is crucial in monitoring infrastructure degradation. However, the strong contextual dependencies of long-spanned morphology and numerous fine-grained branches hinder high-precision segmentation performance. To tackle the above obstacles, we provide a pure Mamba-based model termed CrackMamba for accurate crack segmentation. CrackMamba utilizes the Mamba-based Feature Extractor (MFE) to effectively model global dependencies of long-spanned cracks with linear computational complexity for powerful representation. To capture fine-grained crack branches, we design a novel texture refinement module, which employs a multi-scale aggregation strategy and utilizes the MFE and a Normalized Soft-Frangi-Filter (NSFF) module to integrate hierarchical features from the decoder. The CrackMamba and NSFF exhibit strong complementarity. The NSFF module enhances the capability of CrackMamba to segment fine-grained crack textures, while CrackMamba effectively eliminating crack-unrelated curves extracted by NSFF module. Extensive experiments are conducted on two benchmark crack datasets, and the results demonstrate that the proposed CrackMamba achieves the state-of-the-art (SOTA) performance with fewer parameters and higher computational efficiency.

Segmenting vein from leaf images is challenging because of the strong expansibility, high connectivity and irregular distribution of vein patterns. Although several recent attempts have been made to apply CNNs to leaf vein segmentation, they are weak to capture global contexts of lower-order veins and fine-grained features about higher-order veins due to the continuous stacking of convolution and pooling operations. Inspired by the great successes of transformers on semantic image segmentation, we for the first time develop a novel transformer based model, named LV-SegFormer for leaf vein segmentation. The LV-SegFormer leverages the powerful global modeling capability of transformer to generate advanced representations. The multi-stage alignment module and the hybrid decoder are designed to incorporate contexts from different stages and recover fine-grained features for performance boosting. Extensive experiments are conducted on two challenging benchmark datasets and the results demonstrate that the proposed LV-SegFormer achieves higher performance over the state-of-the-arts.

In the research community of content-based image retrieval, great success has been achieved for leaf image retrieval in species. However, little progress has been made on the more challenging fine-grained leaf image retrieval (FGLIR) which focuses on subspecies/cultivars recognition. To address it, a novel co-occurrence local binary pattern (CoLBP), named Multi-scale Binary-Pattern Dependency (MBPD), is proposed in this study. Despite the potential of CoLBP in encoding contextual information among texture patterns, how to correlate LBPs to yield discriminative co-occurrence features remains an open issue. We introduce two new concepts, Axisymmetric Co-occurrence (ACO) and Cross-thresholding (CRT), into the design of CoLBP. The ACO produces CoLBP by sliding a pair of axisymmetric lines over an adaptive local patch to capture spatial axisymmetric relationship among LBPs. While the CRT correlates LBPs in intensity domain through exchanging their respective thresholds. Their combination is used to yield ACO-CRT local descriptors to encode the dependency information in both spatial and intensity domains. The ACO-CRT local descriptors of multi-scale and multi-position are aggregated into MBPD representation for efficient dissimilarity measure between two leaf images. Extensive experiments validate the superior performance of our method over the state-of-the-arts on FGLIR.

Fine-grained leaf image retrieval (FGLIR) is a new unsupervised pattern recognition task in content-based image retrieval (CBIR). It aims to distinguish varieties/cultivars of leaf images within a certain plant species and is more challenging than general leaf image retrieval task due to the inherently subtle differences across different cultivars. In this study, we for the first time investigate the possible way to mine the spatial structure and contextual information from the activation of the convolutional layers of CNN networks for FGLIR. For achieving this goal, we design a novel geometrical structure, named Triplet Patch-Pairs Composite Structure (TPCS), consisting of three symmetric patch pairs segmented from the leaf images in different orientations. We extract CNN feature map for each patch in TPCS and measure the difference between the feature maps of the patch pair for constructing local deep self-similarity descriptor. By varying the size of the TPCS, we can yield multi-scale deep self-similarity descriptors. The final aggregated local deep self-similarity descriptors, named Structural Deep Patch Representation (SDePR), not only encode the spatial structure and contextual information of leaf images in deep feature domain, but also are invariant to the geometrical transformations. The extensive experiments of applying our SDePR method to the public challenging FGLIR tasks show that our method outperforms the state-of-the-art handcrafted visual features and deep retrieval models.

Fine-grained leaf image retrieval (FGLIR) aims to search similar leaf images in subspecies level which involves very high interclass visual similarity and accordingly poses great challenges to leaf image description. In this study, we introduce a new concept, named fan-beam binarization difference projection (FB-BDP) to address this challenging issue. It is designed based on the theory of fan-beam projection (FBP) which is a mathematical tool originally used for computed tomographic reconstruction of objects and has the merits of capturing the inner structure information of objects in multiple directions and excellent ability to suppress image noise. However, few studies have been made to apply FBP to the description of texture patterns. Rather than calculating ray integrals over the whole object area, FB-BDP restricts its ray integrals calculated over local patches to guarantee the locality of the extracted features. By binarizing the intensity-differences between the off-center and center rays, FB-BDP enable its ray integrals insensitive to illumination change and more discriminative in the characterization of texture patterns. In additional, due to inheriting the merits of FBP, the proposed FB-BDP is superior over the existing local image descriptors by its invariance to scaling transformation, robustness to noise, and strong ability to capture direction and structure texture patterns. The results of extensive experiments on FGLIR show its higher retrieval accuracy over the benchmark methods, promising generalization power and strong complementarity to deep features.

Accurate and effective segmentation of breast masses plays an important role in the early stages of breast cancer treatment. However, the irregular shape of mass, the blurred mass boundary and speckle noise in breast ultrasound (BUS) images make automatic segmentation still challenging. In this paper, we propose a classification task assist (CTA) module for boosting the performance of the commonly used deep segmentation models on BUS images. This module can be easily inserted into representative segmentation models to focus on a learning task of BUS image-level classification. The incorporation of this classification learning task enable the segmentation model achieve additional supervision information to enhance the extraction of semantic context information and the localization of segmentation object. We added the CTA module to four state-of-the-art segmentation models, including codec and non-codec structures, and the experimental results show that the insertion of the CTA module into the original model results in the lowest improvement of 1.48% and the highest improvement of 4.29% in the intersection over union (IoU) metric.

In this study, we address the challenging issue of fine-grained leaf image retrieval which focuses on distinguishing different cultivars within the same species. We propose a novel local binary pattern, named pairwise rotation-difference LBP (PRDLBP), for the characterization of leaf image patterns. Different from the conventional LBP which measure the local grayscale contrast between the center pixel and its circular neighboring pixels, we consider the grayscale contrast between the circular neighboring pixels that are rotational symmetric about the center pixel. The proposed PRDLBP is a co-occurrence LBP feature representation which can not only encode spatially symmetric co-occurrence information, but also be inherently invariant to rotation. Its stronger discriminative power over the state-of-the-arts has been validated on two challenging fine-grained leaf image retrieval tasks, soybean cultivar identification and peanut cultivar identification. This work may attract considerable attention to fine-grained leaf image retrieval and advance the research of leaf image pattern identification from species to cultivars.

This paper addresses the issue of fine-grained leaf image retrieval (FGLIR) which focuses on differentiating between different leaf cultivars within the same species. We investigate a novel bag-of-visual-words approaches (BoVW) to FGLIR. Firstly, we treat each leaf boundary point as the key-point from which to spread a chord pair for measuring the local characteristics including shape, gray-level and gradient co-occurrence texture features of the leaf image. By varying the length of the chord, we obtain multiscale local features which are then used to form two local shape and texture feature vectors. Secondly, we separately collect all the local shape and texture vectors from the database images to learn a leaf shape vocabulary and a leaf texture vocabulary by k-means clustering algorithm. By mapping the two kinds of local feature vectors to visual words in their corresponding leaf vocabularies, we can represent each leaf image as two bags of visual words (one for shape, another for texture). Finally, we convert them into two visual-word vectors by counting the occurrence of each leaf visual word in the image and concatenate them as the final image representation. The proposed leaf vocabulary representation is applied to two challenging FGLIR tasks, soybean cultivar identification and peanut cultivar identification. The experimental results indicate its superior performance over the state-of-the-art leaf descriptors and show its potential to address the issue of FGLIR.

Sketch-based image retrieval (SBIR) is a growing research field due to its potential application in both commercial value and academic value. Using sketches to retrieve images is an ideal substitute to text retrieve in capturing the fine-grained visual information. However, allowing to retrieve natural images from ambiguous and abstract sketch remains a challenging problem in computer vision and image processing. The main challenge of this task is not only the domain difference between free-hand sketch and natural image but also the abstraction of sketch. In this paper, in order to tackle this challenging task, we introduce a novel strategy named stroke harmonization, which is aim at effectively narrow the sketch-image domain gap by explicitly aligning strokes of sketches and natural images to the same stroke style domain. Based on the stroke harmonization, we propose a novel network, named StrokeNet, which consists of generative model and feature extraction model, to harmonizing free-hand sketch and natural image to the same intermediate domain, i.e., stroke domain, and then perform sketch-based image retrieval in this domain. The proposed method is comprehensively evaluated on two representative fine-grained SBIR datasets, QMUL-ShoeV2 and QMUL-ChairV2. The experiments consistently indicate that our method gains the superior accuracies over recent state-of-the-art methods without relying on complex architectures.

Leaf image recognition techniques have been actively researched for plant species identification. However it remains unclear whether analysing leaf patterns can provide sufficient information for further differentiating cultivars. This paper reports our attempt on cultivar recognition from leaves as a general very fine-grained pattern recognition problem, which is not only a challenging research problem but also important for cultivar evaluation, selection and production in agriculture. We propose a novel local R-symmetry co-occurrence method for characterising discriminative local symmetry patterns to distinguish subtle differences among cultivars. Through scalable and moving R-relation radius pairs, we generate a set of radius symmetry co-occurrence matrices (RsCoM)and their measures for describing the local symmetry properties of interior regions. By varying the size of the radius pair, the RsCoM measures local R-symmetry co-occurrence from global/coarse to fine scales. A new two-phase strategy of analysing the distribution of local RsCoM measures is designed to match the multiple scale appearance symmetry pattern distributions of similar cultivar leaf images. We constructed three leaf image databases, SoyCultivar, CottCultivar, and PeanCultivar, for an extensive experimental evaluation on recognition across soybean, cotton and peanut cultivars. Encouraging experimental results of the proposed method in comparison with the state-of-the-art leaf species recognition methods demonstrate the effectiveness of the proposed method for cultivar identification, which may advance the research in leaf recognition from species to cultivar.

Leaf image patterns have been actively researched for plant species recognition. However, as a very challenging fine-grained pattern identification issue, cultivar recognition in which the leaf image patterns usually have very subtle difference among cultivars has not yet received considerable attention in computer vision community. In this paper, a novel leaf image descriptor, named local angle co-occurrence histograms, is proposed for addressing this issue. It is a kind of co-occurrence descriptors that encoding both shape and texture features which make them more informative than the existing individual descriptors and co-occurrence features. A feature fusion scheme is proposed to integrate the handcrafted descriptors with deep learning features for further boosting the retrieval performance. The experimental results on the challenging soybean cultivar recognition and peanut cultivar recognition both indicate the superiority of the proposed method over the state-of-the-art methods on leaf image pattern characterization and validate the effectiveness of the proposed method for fine-grained leaf image retrieval.

Identifying butterfly species by image patterns is a challenging task in computer vision and pattern recognition community due to many butterfly species having similar shape patterns with complex interior structures and considerable pose variation. In additional, geometrical transformation and illumination variation also make this task more difficult. In this paper, a novel image descriptor, named Gaussian convolution angle (GCA) is proposed for butterfly species classification. The proposed GCA projects the butterfly vein image function and intensity image function along a group of vectors that start from a common contour points and ends at the remaining contour points which results a group of vectors that capture the complex vein patterns and texture patterns of butterfly images. The Gaussian convolutions of different scales are conducted to the resulting vector functions to generate a multiscale GCA descriptors. The proposed GCA is not only invariant to geometrical transformation including rotation, scaling and translation, but also invariant to lighting change. The proposed method has been tested on a publicly available butterfly image dataset that has 832 samples of 10 species. It achieves a classification accuracy of 92.03% which is higher than the benchmark methods.

Automated tongue recognition is a growing research field due to global demand for personal health care. Using mobile devices to take tongue pictures is convenient and of low cost for tongue recognition. It is particularly suitable for self-health evaluation of the public. However, images taken by mobile devices are easily affected by various imaging environment, which makes fine segmentation a more challenging task compared with those taken by specialized acquisition devices. Deep learning approaches are promising for tongue image segmentation because they have powerful feature learning and representation capability. However, the successive pooling operations in these methods lead to loss of information on image details, making them fail when segmenting low-quality images captured by mobile devices. To address this issue, we propose a dilated encoder network (DE-Net) to capture more high-level features and get high-resolution output for automated tongue image segmentation. In addition, we construct two tongue image datasets which contain images taken by specialized devices and mobile devices, respectively, to verify the effectiveness of the proposed method. Experimental results on both datasets demonstrate that the proposed method outperforms the state-of-the-art methods in tongue image segmentation.

Prediction of protein subcellular location has currently become a hot topic because it has been proven to be useful for understanding both the disease mechanisms and novel drug design. With the rapid development of automated microscopic imaging technology in recent years, classification methods of bioimage-based protein subcellular location have attracted considerable attention for images can describe the protein distribution intuitively and in detail. In the current study, a prediction method of protein subcellular location was proposed based on multi-view image features that are extracted from three different views, including the four texture features of the original image, the global and local features of the protein extracted from the protein channel images after color segmentation, and the global features of DNA extracted from the DNA channel image. Finally, the extracted features were combined together to improve the performance of subcellular localization prediction. From the performance comparison of different combination features under the same classifier, the best ensemble features could be obtained. In this work, a classifier based on Stacked Auto-encoders and the random forest was also put forward. To improve the prediction results, the deep network was combined with the traditional statistical classification methods. Stringent cross-validation and independent validation tests on the benchmark dataset demonstrated the efficacy of the proposed method.

Effectively describing and recognizing leaf shapes under arbitrary variations, particularly from a large database, remains an unsolved problem. In this research, we attempted a new strategy of describing leaf shapes by walking and measuring along a bunch of chords that pass through the shape. A novel chord bunch walks (CBW) descriptor is developed through the chord walking behavior that effectively integrates the shape image function over the walked chord to reflect both the contour features and the inner properties of the shape. For each contour point, the chord bunch groups multiple pairs of chords to build a hierarchical framework for a coarse-to-fine description that can effectively characterize not only the subtle differences among leaf margin patterns but also the interior part of the shape contour formed inside a self-overlapped or compound leaf. Instead of using optimal correspondence based matching, a Log-Min distance that encourages one-to-one correspondences is proposed for efficient and effective CBW matching. The proposed CBW shape analysis method is invariant to rotation, scaling, translation, and mirror transforms. Five experiments, including image retrieval of compound leaves, image retrieval of naturally self-overlapped leaves, and retrieval of mixed leaves on three large scale datasets, are conducted. The proposed method achieved large accuracy increases with low computational costs over the state-of-the-art benchmarks, which indicates the research potential along this direction.

Effectively describing and recognizing leaf shapes under arbitrary deformations, particularly from a large database, remains an unsolved problem. In this research, we attempted a new strategy of describing shape by walking along a bunch of chords that pass through the shape to measure the regions trespassed. A novel chord bunch walks (CBW) descriptor is developed through the chord walking that effectively integrates the shape image function over the walked chord to reflect the contour features and the inner properties of the shape. For each contour point, the chord bunch groups multiple pairs of chord walks to build a hierarchical framework for a coarse-to-fine description. The proposed CBW descriptor is invariant to rotation, scaling, translation, and mirror transforms. Instead of using the expensive optimal correspondence based matching, an improved Hausdorff distance encoded correspondence information is proposed for efficient yet effective shape matching. In experimental studies, the proposed method obtained substantially higher accuracies with low computational cost over the benchmarks, which indicates the research potential along this direction.

In this paper, we present a novel mathematical tool, Structure Integral Transform (SIT), for invariant shape description and recognition. Different from the Radon Transform (RT), which integrates the shape image function over a 1D line in the image plane, the proposed SIT builds upon two orthogonal integrals over a 2D K -cross dissecting structure spanning across all rotation angles by which the shape regions are bisected in each integral. The proposed SIT brings the following advantages over the RT: 1) it has the extra function of describing the interior structural relationship within the shape which provides a more powerful discriminative ability for shape recognition; 2) the shape regions are dissected by the K -cross in a coarse to fine hierarchical order that can characterize the shape in a better spatial organization scanning from the center to the periphery; and 3) it is easier to build a completely invariant shape descriptor. The experimental results of applying SIT to shape recognition demonstrate its superior performance over the well-known Radon transform, and the well-known shape contexts and the polar harmonic transforms.

A novel shape descriptor, termed multi-scale bisector integrals, is proposed in this paper. Different from the existing Radon transform based descriptors which integrate the shape image function over all the possible lines in its domain, the proposed method restrains the integrals only over a special class of lines, termed shape bisectors, for characterizing the essence of the shape. Integrating the shape image over the multi-orders of shape bisectors yields a multi-scale descriptor. The proposed descriptor is completely invariant to translation, scaling and rotation. The experimental results on the standard MEPG-7 CE-2 shape database demonstrate its superiority over the state-of-the-art approaches.

This paper presents a novel approach for both fast and accurately retrieving similar shapes. A hierarchical string cuts (HSC) method is proposed to partition a shape into multiple level curve segments of different lengths from a point moving around the contour to describe the shape gradually and completely from the global information to the finest details. At each hierarchical level, the curve segments are cut by strings to extract features that characterize the geometric and distribution properties in that particular level of details. The translation, rotation, scale and mirror invariant HSC descriptor enables a fast metric based matching to achieve the desired high accuracy. Encouraging experimental results on four databases demonstrated that the proposed method can consistently achieve higher (or similar) retrieval accuracies than the state-of-the-art benchmarks with a more than 120 times faster speed. This may suggest a new way of developing shape retrieval techniques in which a high accuracy can be achieved by a fast metric matching algorithm without using the time-consuming correspondence optimisation strategy.

A novel shape description method is proposed for mobile retrieval of leaf images to aid in plant recognition. In this method, traveling the shape contour, the convexity and concavity properties of the arches of various levels are measured, respectively, to generate a multiscale shape descriptor. Its performance has been tested on two leaf datasets and the experimental results indicated higher recognition accuracies than the state-of-the-art approaches with a speed improvement of more than 170 times. The proposed method has been successfully applied to develop a prototype system of online plant leaf identification working on a consumer mobile platform.