Learning from crowds aims to train a robust and generalizable model with a noisy crowdsourced dataset from multiple annotators. Due to its simplicity and practicality, the target model and label correction mechanism are co-trained in a parameter-coupled manner. However, this end-to-end training suffers from the inherent flaws in data. The crowdsourced dataset is naturally noisy and redundant: 1) some annotations may violate the annotator’s transition matrix and mislead the training procedure and 2) the size of the annotation set is much larger than the instance set, and some instances may be backpropagated multiple times without performance gain. To address these issues, we propose a sample selection method called CrowdSketch for deep learning from crowds. Specifically, to mitigate the annotation noise, we find possibly high-quality data with small local contrastive loss (clean) and high divergence loss between prediction probabilities (important) on a two-branch network. After that, to alleviate redundant data, an importance score is developed based onl2-norm of errors. The originality of this work stems from the designed selection criteria and specified two-branch architecture for crowdsourcing. Besides, by removing the noisy data, the risk of nonoptimum in dataset pruning is reduced. Extensive experiments are conducted on real-world crowdsourcing datasets. The experimental results, which show average accuracy improvements of 1.27% on LabelMe and 1.52% on CIFAR-10H, demonstrate the effectiveness of CrowdSketch.

The research examines the value and potential usefulness of using systemic thinking, which looks at the interconnectedness of things, to comprehend the complexities of software development projects and the technical and human factors involved. It considers two different aspects of systemic thinking - psychological and sociological - and posits that these can assist in understanding how software teams function and attain their objectives, as well as the goals of the entities for which they work. Our research aims to provide a novel contribution to the field by investigating the use of systemic thinking in software development teams and organizations. We evaluate the reliability and validity of the survey applied to different groups of relevant participants, relate our findings to existing literature, and identify the most representative factors of systemic thinking. Despite the popularity of various factors that fall under the umbrella of ’systems thinking’, there is limited understanding of their effectiveness in improving organizational performance or productivity, particularly when it comes to psychological and sociological systemic factors. The relationship between the use of systems thinking and organizational performance is often based on anecdotal evidence, rather than the identification and application of specific factors. Our work emphasizes the importance of understanding and applying such factors in order to build a solid foundation for the effective use of system dynamics and systems thinking tools, which is crucial for software development teams.

Automatically prioritizing software development tasks extracted from codes could provide significant technical and organizational advantages. Tools exist for the automatic extraction of tasks, but they still lack the ability to capture their mutual dependencies; hence, the capability to prioritize them. Solving this important puzzle is the goal of the presented industrial challenge.

Simple detection of bugs, defects or anomalies during software development is not enough - it is necessary to apply corrective actions to eliminate them. To find out whether an anomaly exists in any software, we can measure the quality attributes using software metrics. The main goal of this paper was to find out and explain how to meaningfully attribute metrics to useful corrective actions.