In software development, domain models are conceptual blueprints that capture the structure, relationships, and key entities of a problem domain. Automated techniques can support analysts and developers by extracting such models from existing artifacts. However, this is a non-trivial task, especially when the input consists of informal artifacts such as user stories. This paper investigates how providing an initial, automatically generated domain model influences novice developers’ ability to construct their own domain models. We conducted an experiment involving 127 undergraduate students, divided into three groups: one receiving an LLM-generated model that maximizes precision (validity), one receiving an LLM-generated model that boosts recall (completeness), and a control group that did not receive any initial domain model. Our findings show that novices who received an initial LLM-generated model produced more complete class identification in simple user story projects and improved relationship identification accuracy in both simple and complex projects. While initial domain models appear to aid novices in refining domain models effectively, our results also suggest a strong tendency among participants to rely heavily on these initial models. On average, students included 85% of the correct classes and 67% of the incorrect classes from the initial domain models in their own derived models. Such reliance can scaffold novice learning and refinement, but this reliance may limit creativity and hinder the deeper reasoning required to develop robust domain modeling skills.

Domain models play a crucial role in software development, as they provide means for communication among stakeholders, for eliciting requirements, and for representing the information structure behind a database scheme or at the basis of model-driven development. However, creating such models is a tedious activity and automated support may assist in obtaining an initial domain model that can later be enriched by human analysts. In this paper, we propose an experimental comparison of the effectiveness of various approaches for deriving domain models from a given set of user stories. We contrast human derivation with machine derivation; for the latter, we compare (i) the Visual Narrator: an existing rule-based NLP approach; (ii) a machine-learning classifier that we feature engineered; and (iii) a generative AI approach that we constructed via prompt engineering. Based on a benchmark dataset that consists of nine collections of user stories and corresponding domain models, the evaluation indicates that no approach matches human performance, although a tuned version of the machine learning approach comes close. To better understand the results, we qualitatively analyze them and identify differences in the types of false positives as well as other factors that affect performance.

The increasing adoption of agile software development has led to a shift toward lightweight and informal requirements documentation. While creating prototypes for automating requirements analysis and management, researchers often overlook the flexibility of requirements documentation in agile settings. We aim to explore the gap between the assumptions that researchers make and real-world documentation practices. Since this gap causes challenges for locating requirements, we plan to develop a tool to help practitioners extract this information.