Fruit counting is a fundamental task for fruit yield estimation. Though semi-supervised counting methods have received increased attention in recent years, due to the high data utilization of unlabeled data, they suffer from two limitations. Firstly, difficult weight selection is a limitation, as these methods rely on manually selected fixed weights for both the supervised learning loss and the consistency learning loss, resulting in limited performance. Secondly, biased pseudo-labeling is another limitation, as they may predict biased pseudo-labels that result in small consistency learning losses, leading to training being dominated by supervised learning with large losses. To tackle these two limitations, in this paper, we propose a novel method named ParetoSSL to automatically derive weights of losses from the perspective of multi-task learning. Specifically, ParetoSSL formulates a multi-objective optimization problem for weight derivation by maximizing the similarity between weighted gradients of losses and a customized gradient preference vector, in which, the vector can guide weight derivation. Moreover, to relieve the effect of pseudo-label biases on consistency learning, we propose a bias-aware gradient preference vector. This vector considers gradient biases brought by the pseudo-label biases, which will down-weight the supervised learning loss while high-weighting the consistency learning loss. Meanwhile, to improve the robustness of ParetoSSL, an inequality equation regarding the norm of the gradients of the consistency learning loss is designed to control the range of gradient biases. Extensive experiments are conducted on the Clustered-Fruit dataset and Fruit-2019 dataset to evaluate the effectiveness of ParetoSSL on semi-supervised counting. Experimental results show that our ParetoSSL is superior to state-of-the-art methods. Note to Practitioners—This work is motivated by the emerging need for semi-supervised counting methods in fruit yield estimation. The difficulty of selecting loss weights for training semi-supervised counting algorithms is exacerbated by the pseudo-label bias issue that pseudo-label biases mislead the weight derivation while maximizing the similarity between the weight gradient and gradient preference vectors. The proposed bias-aware gradient preference vectors help users derive loss weights automatically and save the time of choosing loss weights for model fine-tuning. The proposed ParetoSSL is generic as it can be employed as a fruit yield estimation component of crop management support systems, while at the same time being applied to counting frameworks of other objects.

Point cloud completion aims to infer the complete point clouds from incomplete ones. In real-world scenarios, where the paired data is absent, self-supervised methods have emerged as a promising solution. Although existing self-supervised methods perform well at relatively low resolutions, they suffer significant performance degradation at higher resolution primarily because they focus on point cloud reconstruction at patch-level or point-level. In this paper, we propose a self-supervised method based on Geometric Continuity and Consistency Learning (GCCL) at multi-scale level to improve the accuracy of predicting local details and global shapes of point clouds. Specifically, to capture local details, we employ a patch-topoint strategy and a coarse-fine manner for geometric continuity learning. To constrain the global shapes, we construct multiple branches for mutual supervision and utilize class priors to build a memory queue for contrasting current features, enhancing the network focus on geometric consistency learning. We evaluate GCCL on multiple datasets, and the results show that our method outperforms existing self-supervised methods by a 4.4 improvement in CD-$\ell_2$on the synthetic PCN dataset and can generate more uniformly distributed completion results on realworld datasets

Fruit counting is a fundamental and challenging task of automatic fruit yield estimation in the field of intelligent agriculture. In recent years, to relieve the burden of data annotation, semi-supervised counting methods have been studied. Though significant progress has been achieved, the state-of-the-art method estimates the uncertainty of binary segmentation to guide the consistency training of density maps, being prone to deficient uncertainty estimation. Moreover, the method treats pixels with different difficulty equally in each training iteration, being troubled by inflexible consistency training which results in high supervision loss at the beginning of training and even causes network collapse. To alleviate the above limitations, in this paper, we propose a novel semi-supervised counting method CMCNet for fruit counting. CMCNet designs image colorization as an auxiliary task to estimate the uncertainty for density map consistency. Note that this work is the first effort to utilize image colorization for uncertainty estimation in semi-supervised counting. To obtain accurate uncertainty estimation for density map consistency, CMCNet estimates density uncertainty on density maps to depict the difficulty of fruit pixels from the semantic perspective, while using image colorization for constructing colorization uncertainty to measure the difficulty of part of fruit pixels and background pixels from the visual perspective. Then we obtain a comprehensive uncertainty by mixing density uncertainty and colorization uncertainty. Further, we propose a mix-uncertainty-adaptive consistency (MUAC) module for consistency training of density maps. With mix-uncertainty, uncertainty distribution is estimated. By adaptively adjusting the uncertainty threshold, harder pixels will be selected first and easier ones will be added into consistency training gradually. To evaluate the effectiveness of CMCNet, extensive experiments are conducted on two fruit datasets. Experimental results show that our CMCNet is superior to state-of-the-art semi-supervised counting methods.Note to Practitioners—This work is motivated by the emerging need for semi-supervised counting methods in fruit yield estimation. The difficulty of training semi-supervised counting methods with unlabeled images is exacerbated by the noisy supervision issue that pseudo-labels of unlabeled images are noisy. The proposed colorization-aware uncertainty estimation strategy and mix-uncertainty-adaptive consistency approach help the fruit planter sufficiently utilize the information of a large amount of unlabeled data and save the annotation cost in fruit quantity estimation. The proposed method is generic as it can be employed as a fruit yield estimation component of crop management support systems, while at the same time being applied to counting frameworks of other objects.

As a common cause of anemia and gastrointestinal bleeding, angiodysplasia (AD) diagnosis in wireless capsule endoscopy (WCE) images is important in clinical. Current manual review requires undivided concentration of the gastroenterologists, which is laborious and time-consuming. The development of computational methods that can assist automated diagnosis of angiodysplasia is highly desirable. In this paper, we present a new approach, ADNet, for angiodysplasia segmentation using convolutional neural networks (CNNs). Compared with previous learning strategies, ADNet gains accuracy from attentionguided and domain-adversarial training via a multibranch CNN architecture. Specifically, the core branch is constructed for AD segmentation in a fully convolutional manner. Then we propose an attention module embedded in the attention branch to enhance network feature learning, which allows ADNet to focus on the most informative and AD relevant regions while processing. Furthermore, an adaptation branch is built to learn domain-invariant features by adversarial training, aiming to improve the performance when datasets are expanded while preventing the degradation induced by the variations in WCE image acquisition. ADNet is evaluated using two WCE datasets with angiodysplasia and the results show the accuracy gains we obtain, where the state-of-the-art segmentation performance on the public dataset of GIANA’17 is achieved.

Polyp recognition in colonoscopy images is crucial for early colorectal cancer detection and treatment. However, the current manual review requires undivided concentration of the gastroenterologist and is prone to diagnostic errors. In this article, we present an effective, two-stage approach called PLPNet, where the abbreviation “PLP” stands for the word “polyp,” for automated pixel-accurate polyp recognition in colonoscopy images using very deep convolutional neural networks (CNNs). Compared to hand-engineered approaches and previous neural network architectures, our PLPNet model improves recognition accuracy by adding a polyp proposal stage that predicts the location box with polyp presence. Several schemes are proposed to ensure the model's performance. First of all, we construct a polyp proposal stage as an extension of the faster R-CNN, which performs as a region-level polyp detector to recognize the lesion area as a whole and constitutes stage I of PLPNet. Second, stageII of PLPNet is built in a fully convolutional fashion for pixelwise segmentation. We define a feature sharing strategy to transfer the learned semantics of polyp proposals to the segmentation task of stage II, which is proven to be highly capable of guiding the learning process and improve recognition accuracy. Additionally, we design skip schemes to enrich the feature scales and thus allow the model to generate detailed segmentation predictions. For accurate recognition, the advanced residual nets and feature pyramids are adopted to seek deeper and richer semantics at all network levels. Finally, we construct a two-stage framework for training and run our model convolutionally via a single-stream network at inference time to efficiently output the polyp mask. Experimental results on public data sets of GIANA Challenge demonstrate the accuracy gains of our approach, which surpasses previous state-of-the-art methods on the polyp segmentation task (74.7 Jaccard Index) and establishes new top results in the polyp localization challenge (81.7 recall).

Fruit detection is a fundamental task for automatic yield estimation. The goal is to detect all the fruits in images. The-state of the art of fruit detection algorithm, Faster R-CNN, shows a lack of detection advantage on small fruits. One of the reasons is only that single-level features and a classifier are used for localization of proposal candidates. In this article, we propose to incorporate a multiple classifier fusion strategy into a Faster R-CNN network for small fruit detection. We utilize features from three different levels to learn three classifiers for objectness classification in the stage of proposal localization. Probabilities from classifiers are combined by a simple convolutional layer to generate final objectness classification for proposal candidates. During training, in order to train a model with strong generalization capability, we propose to use correlation coefficients to measure the diversity of multiple classifiers. A novel loss function with classifier correlation is introduced to train the region proposal network. We evaluate the proposed model on two data sets of small fruits. Extensive experiments show that the proposed model outperforms the state-of-the-art detectors for fruit detection.

Modern agricultural robots like drones have been studied in automatic yield estimation in recent years. Fruit counting is a fundamental task in the automatic yield estimation, on which significant progress has been achieved by detection-based methods and segmentation-regression-based methods. However, for clustered-fruit counting, the existing methods lack advantages on the localization of small and occluded fruits or discrete number regression. In addition, it is observed that existing deep neural network based counting methods have high variances on fruit density map estimation. Aiming at solving these two problems and decreasing the regression variance, in this paper, we propose a density-map-estimation model with DropBlock regularization. For evaluating the proposed model, we propose a new Clustered-Fruit dataset. Extensive experiments show that the proposed model is effective and outperforms the state-of-the-art counting methods on the Clustered-Fruit dataset. Our dataset is available at Clustered-Fruit.

The-state-of-the-art of fruit detection with Faster R-CNN shows lack of detection advantage on small fruits. One of reasons is only single level features is used for localization of proposal candidates. In this paper, we propose to incorporate a multiple classifier fusion strategy into a Faster R-CNN network for small fruit detection. We utilize features from three different levels to learn three classifiers for objectness classification in the stage of proposal localization. Probabilities from classifiers are combined by a simple convolutional layer to generate final objectness classification for proposal candidates. In order to keep diversity of multiple classifiers, a novel loss term of classifier correlation is introduced into original loss function. Experimental results show that our model is feasible for detecting small fruits.