Fine-grained leaf image retrieval (FGLIR) is an emerging content-based image retrieval (CBIR) task that distinguishes cultivars of the same plant species. It remains highly challenging due to subtle inter-cultivar variations and the absence of supervision. Existing training-free methods generally treat convolutional features as local descriptors for aggregation, but they often overlook spatial correlations and structural cues. To address this gap, we extend Local Binary Patterns (LBP) from the raw image domain to the deep feature domain for the first time, enabling explicit modeling of spatial relationships. Specifically, each leaf image is partitioned into R-Angle patches to preserve invariant traits, which are processed by pre-trained CNNs to produce deep feature maps. We then propose the Local Deep-Binary Pattern Transform (LDbPT) and Local Deep Contrast-Pattern Transform (LDcPT) to enhance spatial and contrast encoding, together with anR-Angle Grouped Average Pooling strategy to embed shape information into feature aggregation. Furthermore, we hybridize CNNs with Vision Transformers (ViTs) within this framework, effectively combining local structural extraction with global contextual reasoning. Extensive experiments on two public FGLIR benchmarks with comprehensive evaluations show that our method achieves state-of-the-art retrieval accuracy while demonstrating robustness, efficiency, and strong generalization, confirming the effectiveness of the proposed training-free hybrid representation.

In the research community of content-based image retrieval, great success has been achieved for leaf image retrieval in species. However, little progress has been made on the more challenging fine-grained leaf image retrieval (FGLIR) which focuses on subspecies/cultivars recognition. To address it, a novel co-occurrence local binary pattern (CoLBP), named Multi-scale Binary-Pattern Dependency (MBPD), is proposed in this study. Despite the potential of CoLBP in encoding contextual information among texture patterns, how to correlate LBPs to yield discriminative co-occurrence features remains an open issue. We introduce two new concepts, Axisymmetric Co-occurrence (ACO) and Cross-thresholding (CRT), into the design of CoLBP. The ACO produces CoLBP by sliding a pair of axisymmetric lines over an adaptive local patch to capture spatial axisymmetric relationship among LBPs. While the CRT correlates LBPs in intensity domain through exchanging their respective thresholds. Their combination is used to yield ACO-CRT local descriptors to encode the dependency information in both spatial and intensity domains. The ACO-CRT local descriptors of multi-scale and multi-position are aggregated into MBPD representation for efficient dissimilarity measure between two leaf images. Extensive experiments validate the superior performance of our method over the state-of-the-arts on FGLIR.

Fine-grained leaf image retrieval (FGLIR) is a new unsupervised pattern recognition task in content-based image retrieval (CBIR). It aims to distinguish varieties/cultivars of leaf images within a certain plant species and is more challenging than general leaf image retrieval task due to the inherently subtle differences across different cultivars. In this study, we for the first time investigate the possible way to mine the spatial structure and contextual information from the activation of the convolutional layers of CNN networks for FGLIR. For achieving this goal, we design a novel geometrical structure, named Triplet Patch-Pairs Composite Structure (TPCS), consisting of three symmetric patch pairs segmented from the leaf images in different orientations. We extract CNN feature map for each patch in TPCS and measure the difference between the feature maps of the patch pair for constructing local deep self-similarity descriptor. By varying the size of the TPCS, we can yield multi-scale deep self-similarity descriptors. The final aggregated local deep self-similarity descriptors, named Structural Deep Patch Representation (SDePR), not only encode the spatial structure and contextual information of leaf images in deep feature domain, but also are invariant to the geometrical transformations. The extensive experiments of applying our SDePR method to the public challenging FGLIR tasks show that our method outperforms the state-of-the-art handcrafted visual features and deep retrieval models.

Fine-grained leaf image retrieval (FGLIR) aims to search similar leaf images in subspecies level which involves very high interclass visual similarity and accordingly poses great challenges to leaf image description. In this study, we introduce a new concept, named fan-beam binarization difference projection (FB-BDP) to address this challenging issue. It is designed based on the theory of fan-beam projection (FBP) which is a mathematical tool originally used for computed tomographic reconstruction of objects and has the merits of capturing the inner structure information of objects in multiple directions and excellent ability to suppress image noise. However, few studies have been made to apply FBP to the description of texture patterns. Rather than calculating ray integrals over the whole object area, FB-BDP restricts its ray integrals calculated over local patches to guarantee the locality of the extracted features. By binarizing the intensity-differences between the off-center and center rays, FB-BDP enable its ray integrals insensitive to illumination change and more discriminative in the characterization of texture patterns. In additional, due to inheriting the merits of FBP, the proposed FB-BDP is superior over the existing local image descriptors by its invariance to scaling transformation, robustness to noise, and strong ability to capture direction and structure texture patterns. The results of extensive experiments on FGLIR show its higher retrieval accuracy over the benchmark methods, promising generalization power and strong complementarity to deep features.

In this study, we address the challenging issue of fine-grained leaf image retrieval which focuses on distinguishing different cultivars within the same species. We propose a novel local binary pattern, named pairwise rotation-difference LBP (PRDLBP), for the characterization of leaf image patterns. Different from the conventional LBP which measure the local grayscale contrast between the center pixel and its circular neighboring pixels, we consider the grayscale contrast between the circular neighboring pixels that are rotational symmetric about the center pixel. The proposed PRDLBP is a co-occurrence LBP feature representation which can not only encode spatially symmetric co-occurrence information, but also be inherently invariant to rotation. Its stronger discriminative power over the state-of-the-arts has been validated on two challenging fine-grained leaf image retrieval tasks, soybean cultivar identification and peanut cultivar identification. This work may attract considerable attention to fine-grained leaf image retrieval and advance the research of leaf image pattern identification from species to cultivars.

This paper addresses the issue of fine-grained leaf image retrieval (FGLIR) which focuses on differentiating between different leaf cultivars within the same species. We investigate a novel bag-of-visual-words approaches (BoVW) to FGLIR. Firstly, we treat each leaf boundary point as the key-point from which to spread a chord pair for measuring the local characteristics including shape, gray-level and gradient co-occurrence texture features of the leaf image. By varying the length of the chord, we obtain multiscale local features which are then used to form two local shape and texture feature vectors. Secondly, we separately collect all the local shape and texture vectors from the database images to learn a leaf shape vocabulary and a leaf texture vocabulary by k-means clustering algorithm. By mapping the two kinds of local feature vectors to visual words in their corresponding leaf vocabularies, we can represent each leaf image as two bags of visual words (one for shape, another for texture). Finally, we convert them into two visual-word vectors by counting the occurrence of each leaf visual word in the image and concatenate them as the final image representation. The proposed leaf vocabulary representation is applied to two challenging FGLIR tasks, soybean cultivar identification and peanut cultivar identification. The experimental results indicate its superior performance over the state-of-the-art leaf descriptors and show its potential to address the issue of FGLIR.

Sketch-based image retrieval (SBIR) is a growing research field due to its potential application in both commercial value and academic value. Using sketches to retrieve images is an ideal substitute to text retrieve in capturing the fine-grained visual information. However, allowing to retrieve natural images from ambiguous and abstract sketch remains a challenging problem in computer vision and image processing. The main challenge of this task is not only the domain difference between free-hand sketch and natural image but also the abstraction of sketch. In this paper, in order to tackle this challenging task, we introduce a novel strategy named stroke harmonization, which is aim at effectively narrow the sketch-image domain gap by explicitly aligning strokes of sketches and natural images to the same stroke style domain. Based on the stroke harmonization, we propose a novel network, named StrokeNet, which consists of generative model and feature extraction model, to harmonizing free-hand sketch and natural image to the same intermediate domain, i.e., stroke domain, and then perform sketch-based image retrieval in this domain. The proposed method is comprehensively evaluated on two representative fine-grained SBIR datasets, QMUL-ShoeV2 and QMUL-ChairV2. The experiments consistently indicate that our method gains the superior accuracies over recent state-of-the-art methods without relying on complex architectures.

Leaf image patterns have been actively researched for plant species recognition. However, as a very challenging fine-grained pattern identification issue, cultivar recognition in which the leaf image patterns usually have very subtle difference among cultivars has not yet received considerable attention in computer vision community. In this paper, a novel leaf image descriptor, named local angle co-occurrence histograms, is proposed for addressing this issue. It is a kind of co-occurrence descriptors that encoding both shape and texture features which make them more informative than the existing individual descriptors and co-occurrence features. A feature fusion scheme is proposed to integrate the handcrafted descriptors with deep learning features for further boosting the retrieval performance. The experimental results on the challenging soybean cultivar recognition and peanut cultivar recognition both indicate the superiority of the proposed method over the state-of-the-art methods on leaf image pattern characterization and validate the effectiveness of the proposed method for fine-grained leaf image retrieval.

Identifying butterfly species by image patterns is a challenging task in computer vision and pattern recognition community due to many butterfly species having similar shape patterns with complex interior structures and considerable pose variation. In additional, geometrical transformation and illumination variation also make this task more difficult. In this paper, a novel image descriptor, named Gaussian convolution angle (GCA) is proposed for butterfly species classification. The proposed GCA projects the butterfly vein image function and intensity image function along a group of vectors that start from a common contour points and ends at the remaining contour points which results a group of vectors that capture the complex vein patterns and texture patterns of butterfly images. The Gaussian convolutions of different scales are conducted to the resulting vector functions to generate a multiscale GCA descriptors. The proposed GCA is not only invariant to geometrical transformation including rotation, scaling and translation, but also invariant to lighting change. The proposed method has been tested on a publicly available butterfly image dataset that has 832 samples of 10 species. It achieves a classification accuracy of 92.03% which is higher than the benchmark methods.