Automated unit test generation consists of two complementary challenges: Finding sequences of API calls that exercise the code of a class under test, and finding assertion statements that validate the behavior of the class during execution. The former challenge is often addressed using meta-heuristic search algorithms optimising tests for code coverage, which are then annotated with regression assertions to address the latter challenge, i.e., assertions that capture the states observed during test generation. While the resulting tests tend to achieve high coverage, their fault finding potential is often inhibited by poor or difficult observability of the codebase. That is, relevant attributes and properties may either not be exposed adequately at all, or only in ways that the test generator is unable to handle. In this paper, we investigate the influence of observability in the context of the EvoSuite search-based Java test generator, which we extend in two complementary ways to study and improve observability: First, we apply a transformation to code under test to expose encapsulated attributes to the test generator; second, we address EvoSuite's limited capability of asserting the state of complex objects. Our evaluation demonstrates that together these observability improvements lead to significantly increased mutation scores, underscoring the importance of considering the class observability in the test generation process.

Mobile applications play a relevant role in users' daily lives by improving and easing daily processes such as commuting or making financial transactions. The aforementioned interactions enhance the usability of commonly used services. Nevertheless, the improvements should also consider special execution environments such as weak network connections or special requirements inherited from the user's condition. Due to this, the design of mobile applications should be driven by improving the user experience. This tutorial targets the usage of inclusive and accessibility design in the development process of mobile apps. Making sure that applications are accessible to all users, regardless of disabilities, is not just about following the law or fulfilling ethical obligations; it is crucial in creating inclusive and fair digital environments. This tutorial will educate participants on accessibility principles and the available tools. They will gain practical experience with specific Android and iOS platform features, as well as become acquainted with state-of - the-art automated and manual testing tools.

In software engineering pedagogy, a persistent challenge is the comprehensive assessment of student contributions within software repositories. This study delves into the investigation of the Git-Truck tool, initially designed for professional software engineers, and explores its adaptability and effectiveness within an academic setting. We specifically focus on the tool's potential for educators when assessing Capstone software repositories. Our results emphasize that educators found bubble chart visualization and metrics such as ''Top Contributor'' and ''Number of Commits'' helpful in understanding group dynamics and contribution. We also discuss the tool's limitations among visual techniques and metrics used. As the educational landscape shifts towards increased virtual and remote modalities, tools like Git-Truck are poised to augment the intricacy and depth of software project evaluations. For those considering adopting or adapting such tools in similar contexts, our study offers the challenges and lessons learned from this experience.

In this challenge, we explore the evolution of the Linux kernel’s performance during compilation by comparing versions 5.19.17 and 6.14 through sampling-based CPU profiling. We collect profiling data using perf, transform into Chromecompatible .cpuprofile format, and analyze through a novel spatial visualization called FlameGraph AR.FlameGraph AR extends traditional flamegraphs beyond the limitations of IDE panels and conventional screens by rendering visualizations with augmented reality on a Microsoft HoloLens 2 device. By offloading the flamegraph to physical space, the FlameGraph AR tool enables developers to walk through wide and deeply nested call stacks, examine function frames through gesture-based interactions, and gain spatial awareness of the runtime behavior of a software system.In effect, we found immersive visualization especially valuable for analyzing architectural changes between the two kernel versions. We found that version 6.14 exhibits a significantly higher number of samples in several functions, such as native_write_msr, indicating intensified low-level CPU interactions. In addition, functions such as intel_pmu_enable_all and x86_pmu_enable also increased in frequency, suggesting increased reliance on performance monitoring. The stack depth analysis revealed that certain functions in version 6.14, including fpregs_assert_state_consistent and account_user_time, appear at significantly deeper levels than in earlier versions. Indeed, some reach the maximum stack trace depth of the profiling tool. The results indicate a growth in both modularity and the depth of instrumentation within the kernel execution paths.Multiple performance changes become visible and interactive with Flamegraph AR. For example, time-consuming functions show up as wide frames that span over desks or walls, and deep call stacks are explored physically by approaching or gazing upward. By mapping performance traces into the spatial domain, our tool provides a compelling method for understanding systemic evolution in large-scale software like the Linux kernel.Video URL: https://vimeo.com/1092935027/7d09676a83

Performance analysis is essential to identify bottlenecks and improve software responsiveness. Flame graphs are widely used for this purpose, offering compact summaries of stack traces and execution times. However, as applications grow, flame graphs become large and dense, competing for space within IDEs already crowded with code editors and panels. We propose FlameGraph AR, a tool that offloads flame graph visualizations from the IDE to the physical environment using augmented reality. By integrating a Visual Studio Code extension with an AR application, developers can arrange interactive flame graphs on desks, walls, or in peripheral view. This immersive setup expands visualization space, supports gesturebased interaction, and enables parallel performance analysis without disrupting the coding flow.Video URL: https://vimeo.com/1089364433/e41cfa13c4

StandUp Reports play an important role in capstone software engineering courses, facilitating progress tracking, obstacle identification, and team collaboration. However, despite their significance, students often grapple with the challenge of creating StandUp Reports that are clear, concise, and actionable. This paper investigates the impact of the use of generative AI in producing StandUp report recommendations, aiming to assist students in enhancing the quality and effectiveness of their reports. In a semester-long capstone course, 179 students participated in 16 real-world software development projects. They submitted weekly StandUp Reports with the assistance of an AI-powered Slack, which analyzed their initial reports and provided suggestions for enhancing them using both GPT-3.5 and the early access GPT-4 API. After each submitted report, students voluntarily answered a survey about usability and suggestion preference. Furthermore, we conducted a linguistic analysis of the recommendations made by the algorithms to gauge reading ease and comprehension complexity. Our findings indicate that the AI-based recommendation system helped students improve the overall quality of their StandUp Reports throughout the semester. Students expressed a high level of satisfaction with the tool and exhibited a strong willingness to continue using it in the future. The survey reveals that students perceived a slight improvement when using GPT-4 compared to GPT-3.5. Finally, a computational linguistic analysis performed on the recommendations demonstrates that both algorithms significantly improve the alignment between the generated texts and the students' educational level, thereby improving the quality of the original texts.

The software engineering community has produced numerous tools, techniques, and methodologies for practitioners to analyze and optimize memory usage during software execution. However, little is known about the actual needs of programmers when analyzing memory behavior and how they use tools to address those needs. We conducted an exploratory study (i) to understand what a programmer needs to know when analyzing memory behavior and (ii) how a programmer finds that information with current tools. From our observations, we provide a catalog of 34 questions programmers ask themselves when analyzing memory behavior. We also report a detailed analysis of how some tools are used to answer these questions and the difficulties participants face during the process. Finally, we present four recommendations to guide researchers and developers in designing, evaluating, and improving memory behavior analysis tools.

Diverse studies have analyzed the quality of automatically generated test cases by using test smells as the main quality attribute. But recent work reported that generated tests might suffer from a number of quality issues not considered previously, thus suggesting that not all test smells have been identified yet. Little is known about these issues and their frequency within generated tests. In this paper, we report on a manual analysis of an external dataset consisting of 2,340 automatically generated tests. This analysis aimed at detecting new quality issues, not covered by past recognized test smells. We use thematic analysis to group and categorize the new quality issues found. As a result, we propose a taxonomy of 13 new quality issues grouped in four categories. We also report on the frequency of these new quality issues within the dataset and present eight recommendations that test generators may consider to improve the quality and usefulness of the automatically generated tests. As an additional contribution, our results suggest that (i) test quality should be evaluated not only on the tests themselves, but considering also the tested code; and (ii) automatically generated tests present flaws that are unlikely to be found in manually created tests and thus require specific quality checking tools.

Analyzing source code dependencies between components within a program is an essential activity in software development. While various software visualization tools have been proposed to aid in this activity, most are limited to desktop applications. As a result, the potential impact of augmented reality (AR) on improving dependency analysis remains largely unexplored. In this paper, we present DGT-AR, a node-link visualization tool for code dependencies in immersive augmented reality. DG T-AR extends the physical screen space of IDEs to the infinite virtual space. That is, developers neither have to sacrifice screen space nor leave the IDE and use third-party applications. We present the preliminary results of a pilot user study along with four key lessons learned. Additionally, we have made DGT-AR publicly available.

Detecting and repairing memory issues is still a challenging task. One reason is that understanding a program's memory usage involves a diverse and related set of dynamic and static aspects. Over the years, multiple tools have been proposed to assist practitioners in these activities. However, detailed information about how a tool helps users when analyzing memory usage is missing.This article introduces Vismep, an interactive visualization prototype to help programmers analyze Python applications' memory usage, and presents an exploratory study to understand the behavior and perception of users when using Vismep. As a result, we reported five information needs when participants analyze memory consumption and how they use Vismep to satisfy these needs. Besides, participants positively perceived Vismep due to their valuable views and high overall usability.

Information about source code changes is important for many software development activities. As such, modern IDEs, including, IntelliJ IDEA and Visual Studio Code, show visual clues within the code editor that highlight lines that have been changed since the last synchronization with the code repository. However, the granularity of the change information is limited to a line level, showing mainly a small colored icon on the left side of the lines that have been added, deleted, or modified.This paper introduces Spike, a source code highlighting plugin that uses the font color to visually encode fine-grained version difference information within the code editor. In contrast to previously mentioned tools, Spike can highlight insertions, deletions, updates, and refactorings all in a same line. Our plugin also enriches the source code with small icons that allow retrieving detailed information about a given code change. We perform an exploratory user study with five professional software engineers. Our results show that our approach is able to assist practitioners with complex comprehension tasks about software history within the code editor.

Genetic algorithms are an efficient mechanism to generate unit tests. Automatically generated unit tests are known to be an important asset to identify software defects and define oracles. However, configuring the test generation is a tedious activity for a practitioner due to the inherent difficulty to adequately tuning the generation process. This paper presents TestEvoViz, a visual technique to introspect the generation of unit tests using genetic algorithms. TestEvoViz offers the practitioners a visual support to expose some of the decisions made by the test generation. A number of case studies are presented to illustrate the expressiveness of TestEvoViz to understand the effect of the algorithm configuration.Artifact - https://github.com/andreina-covi/ArtifactSSG.

Software performance may be significantly affected by source code modifications. Understanding the effect of these changes along different software versions is a challenging and necessary activity to debug performance failures. It is not sufficiently supported by existing profiling tools and visualization approaches. Practitioners would need to manually compare calling context trees and call graphs. We aim at better supporting the comparison of benchmark executions along multiple software versions. We propose Performance Evolution Matrix, an interactive visualization technique that contrasts runtime metrics to source code changes. It combines a comparison of time series data and execution graphs in a matrix layout, showing performance and source code metrics at different levels of granularity. The approach guides practitioners from the high-level identification of a performance regression to the changes that might have caused the issue. We conducted a controlled experiment with 12 participants to provide empirical evidence of the viability of our method. The results indicate that our approach can reduce the effort for identifying sources of performance regressions compared to traditional profiling visualizations.

Monitoring software performance evolution is a daunting and challenging task. This paper proposes a lightweight visualization technique that contrasts source code variation with the memory consumption and execution time of a particular benchmark. The visualization fully integrates with the commit graph as common in many software repository managers. We illustrate the usefulness of our approach with two application examples. We expect our technique to be beneficial for practitioners who wish to easily review the impact of source code commits on software performance.

Profiling the memory consumption of a software execution is usually carried out by characterizing calling-context trees. However, the plurality nature of this data-structure makes it difficult to adequately and efficiently exploit in practice. As a consequence, most of anomalies in memory footprints are addressed either manually or in an ad-hoc way. We propose an interactive visualization of the execution context related to object productions. Our visualization augments the traditional calling-context tree with visual cues to characterize object allocation sites.We performed a qualitative study involving eight software engineers conducting a software execution memory assessment. As a result, we found that participants find our visualization as beneficial to characterizing a memory consumption and to reducing the overall memory footprint.

Glyphs are automatically generated visual icons, commonly employed as an object identification technique. Although popular in the Human Computer Interaction community, glyphs are rarely employed to address software engineering problems. We extended the VisualID glyph technique to cope with structural software elements and used it to address two issues in software maintenance: identify classes with the same dependencies and classes with a similar set of methods. We have compared VisualID against three visual representations: textual, graph (nodes and edges), and dependency structural matrix. Our experiments indicate that VisualID significantly helps identify classes with the same dependencies and classes with similar methods when compared with visual techniques commonly used in software maintenance.

Source code changes may inadvertently introduce performance regressions. Benchmarking each software version is traditionally employed to identify performance regressions. Although effective, this exhaustive approach is hard to carry out in practice. This paper contrasts source code changes against performance variations. By analyzing 1,288 software versions from 17 open source projects, we identified 10 source code changes leading to a performance variation (improvement or regression). We have produced a cost model to infer whether a software commit introduces a performance variation by analyzing the source code and sampling the execution of a few versions. By profiling the execution of only 17% of the versions, our model is able to identify 83% of the performance regressions greater than 5% and 100% of the regressions greater than 50%.

Little is known about how software performance evolves across software revisions. The severity of this situation is high since (i) most performance variations seem to happen accidentally and (ii) addressing a performance regression is challenging, especially when functional code is stacked on it. This paper reports an empirical study on the performance evolution of 19 applications, totaling over 19 MLOC. It took 52 days to run our 49 benchmarks. By relating performance variation with source code revisions, we found out that: (i) 1 out of every 3 application revisions introduces a performance variation, (ii) performance variations may be classified into 9 patterns, (iii) the most prominent cause of performance regression involves loops and collections. We carefully describe the patterns we identified, and detail how we addressed the numerous challenges we faced to complete our experiment.

Understanding the root of a performance drop or improvement requires analyzing different program executions at a fine grain level. Such an analysis involves dedicated profiling and representation techniques. JProfiler and YourKit, two recognized code profilers fail, on both providing adequate metrics and visual representations, conveying a false sense of the performance variation root. We propose performance evolution blueprint, a visual support to precisely compare multiple software executions. Our blueprint is offered by Rizel, a code profiler to efficiently explore performance of a set of benchmarks against multiple software revisions.