Occlusions often lead to underestimated grape phenotypes, thereby impeding accurate vineyard management and yield prediction. In grape de-occlusion tasks, existing amodal completion methods perform poorly due to complex cluster structures and fine-grained local textures. To this end, we propose a shape and prototype guided diffusion model for high-fidelity amodal completion, by developing a variational shape embedding learning strategy and a dynamic prototype prior extraction module. We model the shape embedding as a mixture of von Mises–Fisher distributions, indicating the potential complete shape of occluded grape clusters. Subsequently, we formulate prototype priors as discrete embeddings that are consistent with patch features, which represent local texture characteristics of grape berries. The shape and prototype embeddings are integrated into the reverse diffusion process via cross-attention mechanisms, stabilizing structural predictions and mitigating common artifacts such as deformation and adhesion. We construct and release a grape amodal completion dataset collected from real-world vineyard environments. Experimental results on the grape dataset and public KINS dataset demonstrate the superiority of our method in terms of perceptual fidelity and phenotypic accuracy, highlighting its effectiveness for vision-based phenotyping in practical vineyard applications.

BACKGROUND: A growing body of researches indicate that the disrupted expression of long non-coding RNA (lncRNA) is linked to a range of human disorders. Therefore, the effective prediction of lncRNA-disease association (LDA) can not only suggest solutions to diagnose a condition but also save significant time and labor costs. METHOD: In this work, we proposed a novel LDA predicting algorithm based on graph convolutional network and transformer, named GCNFORMER. Firstly, we integrated the intraclass similarity and interclass connections between miRNAs, lncRNAs and diseases, and built a graph adjacency matrix. Secondly, to completely obtain the features between various nodes, we employed a graph convolutional network for feature extraction. Finally, to obtain the global dependencies between inputs and outputs, we used a transformer encoder with a multiheaded attention mechanism to forecast lncRNA-disease associations. RESULTS: The results of fivefold cross-validation experiment on the public dataset revealed that the AUC and AUPR of GCNFORMER achieved 0.9739 and 0.9812, respectively. We compared GCNFORMER with six advanced LDA prediction models, and the results indicated its superiority over the other six models. Furthermore, GCNFORMER's effectiveness in predicting potential LDAs is underscored by case studies on breast cancer, colon cancer and lung cancer. CONCLUSIONS: The combination of graph convolutional network and transformer can effectively improve the performance of LDA prediction model and promote the in-depth development of this research filed.

Though beneficial for encouraging the visual question answering (VQA) models to discover the underlying knowledge by exploiting the input-output correlation beyond image and text contexts, the existing knowledge VQA data sets are mostly annotated in a crowdsource way, e.g., collecting questions and external reasons from different users via the Internet. In addition to the challenge of knowledge reasoning, how to deal with the annotator bias also remains unsolved, which often leads to superficial overfitted correlations between questions and answers. To address this issue, we propose a novel data set named knowledge-routed visual question reasoning for VQA model evaluation. Considering that a desirable VQA model should correctly perceive the image context, understand the question, and incorporate its learned knowledge, our proposed data set aims to cut off the shortcut learning exploited by the current deep embedding models and push the research boundary of the knowledge-based visual question reasoning. Specifically, we generate the question-answer pair based on both the visual genome scene graph and an external knowledge base with controlled programs to disentangle the knowledge from other biases. The programs can select one or two triplets from the scene graph or knowledge base to push multistep reasoning, avoid answer ambiguity, and balance the answer distribution. In contrast to the existing VQA data sets, we further imply the following two major constraints on the programs to incorporate knowledge reasoning. First, multiple knowledge triplets can be related to the question, but only one knowledge relates to the image object. This can enforce the VQA model to correctly perceive the image instead of guessing the knowledge based on the given question solely. Second, all questions are based on different knowledge, but the candidate answers are the same for both the training and test sets. We make the testing knowledge unused during training to evaluate whether a model can understand question words and handle unseen combinations. Extensive experiments with various baselines and state-of-the-art VQA models are conducted to demonstrate that there still exists a big gap between the model with and without groundtruth supporting triplets when given the embedded knowledge base. This reveals the weakness of the current deep embedding models on the knowledge reasoning problem.

Abstract Real‐world classification often encounters a problem called class imbalance. When the data of some classes are redundant than that of other classes, traditional classifiers usually bias their decision boundaries to the redundant majority classes. Most proposed cost‐sensitive strategies often ignore the hard‐to‐learn examples or have a large amount of hyper‐parameters. This article proposes an adaptive learning cost‐sensitive convolutional neural network to solve this problem. During the training process, the proposed method embeds a class‐dependent cost to each class in the global error, making the decision boundary bias to the minority classes. Meanwhile, a distribution weight is assigned to each example to enhance the learning of the hard‐to‐learn examples. Both the class‐dependent costs and distribution weights are learnt automatically in the net. This cost‐sensitive approach makes the algorithm focus on the examples in the minority classes as well as the hard‐to‐learn examples in each class. Besides, this approach can be applied to both binary and multi‐class image classification problems without any modification. Experiments are conducted on four image classification datasets to evaluate this algorithm. The experimental results show that the proposed method achieves better performance than the baseline algorithms and some other algorithms.

Collaborative reasoning for understanding image-question pairs is a very critical but underexplored topic in interpretable visual question answering systems. Although very recent studies have attempted to use explicit compositional processes to assemble multiple subtasks embedded in questions, their models heavily rely on annotations or handcrafted rules to obtain valid reasoning processes, which leads to either heavy workloads or poor performance on compositional reasoning. In this paper, to better align image and language domains in diverse and unrestricted cases, we propose a novel neural network model that performs global reasoning on a dependency tree parsed from the question; thus, our model is called a parse-tree-guided reasoning network (PTGRN). This network consists of three collaborative modules: i) an attention module that exploits the local visual evidence of each word parsed from the question, ii) a gated residual composition module that composes the previously mined evidence, and iii) a parse-tree-guided propagation module that passes the mined evidence along the parse tree. Thus, PTGRN is capable of building an interpretable visual question answering (VQA) system that gradually derives image cues following question-driven parse-tree reasoning. Experiments on relational datasets demonstrate the superiority of PTGRN over current state-of-the-art VQA methods, and the visualization results highlight the explainable capability of our reasoning system.

Ensemble learning is an excellent method for imbalance classification. However, the existing ensemble methods often ignore noise in the dataset, which may reduce the accuracy of classifier. In this paper, we propose a density-based undersampling algorithm (DBU) and integrate it with AdaBoost (DBUBo ost) to improve the classification performance. The major contribution of this paper is the development of an undersampling strategy for dealing with both noise and class imbalance problem. We first divide the examples from each class into three categories: useful examples, noise and potentially useful examples. Then we introduce a similarity coefficient to distinguish the examples from each category. Through a selection mechanism based on similarity coefficients, we retain the useful examples and remove the noisy examples. To demonstrate the effectiveness, we compare our DBUBoost with four ensemble methods and three anti-noise methods. The experiments were conducted on 9 KEEL datasets and their noise-modified datasets. Experimental results have shown that our DBUBoost performs better than other state-of-the-art methods.

The collaborative reasoning for understanding each image-question pair is very critical but under-explored for an interpretable Visual Question Answering (VQA) system. Although very recent works also tried the explicit compositional processes to assemble multiple sub-tasks embedded in the questions, their models heavily rely on the annotations or hand-crafted rules to obtain valid reasoning layout, leading to either heavy labor or poor performance on composition reasoning. In this paper, to enable global context reasoning for better aligning image and language domains in diverse and unrestricted cases, we propose a novel reasoning network called Adversarial Composition Modular Network (ACMN). This network comprises of two collaborative modules: i) an adversarial attention module to exploit the local visual evidence for each word parsed from the question; ii) a residual composition module to compose the previously mined evidence. Given a dependency parse tree for each question, the adversarial attention module progressively discovers salient regions of one word by densely combining regions of child word nodes in an adversarial manner. Then residual composition module merges the hidden representations of an arbitrary number of children through sum pooling and residual connection. Our ACMN is thus capable of building an interpretable VQA system that gradually dives the image cues following a question-driven reasoning route and makes global reasoning by incorporating the learned knowledge of all attention modules in a principled manner. Experiments on relational datasets demonstrate the superiority of our ACMN and visualization results show the explainable capability of our reasoning system.

Traditional image features are not able to effectively represent railway fasteners under varied illumination and conditions. We propose the line local binary pattern encoding method that considers the relationship between the center point and its upper and lower neighborhoods. The method can effectively represent the key components of fasteners. In comparison with several state-of-the-art methods, the proposed method has good performance on detecting the completely missing and partly missing fasteners on real data sets, especially when the illumination and background are not ideal.

The original local binary pattern (LBP) operator or LBP variants adopt the difference between the neighboring pixels and the center pixel to describe the pixel that does not consider the relationship between the neighboring pixels. The block region characteristics of an image are determined by the relationship between neighboring pixels, not just the neighboring pixels and the center pixel. In this letter, a new local neighborhood encoding method is proposed, which we call random sampling LBP (RSLBP). Based on the distribution of the image difference signal, point pairs are randomly selected in the local neighborhood, and LBP encoding is carried out after comparing the sums of pixels neighboring the random point. Image local difference is more obvious and noise resistance is better. By comparing the classification results of LBP and LBP variants with the proposed method, we show that the proposed method achieves better classification performance on the standard images library and the real fastener images, and the performance gain is significant when the noise level is high.

Intensity inhomogeneity often causes considerable difficulties in image segmentation. In order to tackle this problem, we propose a novel region-based active contour model in a variational level set formulation. We first define a data fitting energy with a local Gaussian distribution fitting (LGDF) term, which induces a local force to attract the contour and stops it at object boundaries, and a local signed difference (LSD) term based on local entropy, which possesses both local separability and global consistency. This energy is then incorporated into a level set formulation with a level set regularization term that is necessary for accurate computation in the corresponding level set method. Experimental results show that the proposed model can not only segment images with intensity inhomogeneities and weak boundaries but also be robust to the noise, initial contours.

In this letter, we explore local image descriptors for texture classification. We mainly propose two novel contributions: an effective sampling structure based on Pixel To Patch (PTP) to mimic the retinal sampling pattern; and a novel Local Neighboring Intensity Relationship Pattern (LNIRP) descriptor to extract texture feature by exploring neighboring gray-scale properties. The LNIRP descriptor is extended by using the PTP sampling structure which aims to capture not only micro-patterns but also macro-patterns, while reducing feature dimensionality and improving computational efficiency. The proposed descriptor has advantages of computational simplicity, no texton dictionary learning step and training-free. Moreover, the LNIRP descriptor is complementary to the Local Binary Pattern (LBP) descriptor. Extensive experiments were conducted on Outex database to evaluate the proposed descriptor and sampling structure. The proposed descriptor can achieve superior classification performance compared to most of the state-of-the-art methods, including what we believe to be the best results reported for Outex, while offering a smallest feature dimension.