A Test-Driven Approach for Refining Use Case Specifications of Software Requirements with LLMs

Automatically generating test cases from software requirements is an important task in software engineering [1]. Acceptance testing is a critical phase for ensuring that the software system, as a whole, meets its functional requirements and business objectives, and for determining its readiness for delivery [2]. Effective acceptance testing requires well-defined test criteria, which help test engineers understand what to test and how to conduct tests. However, such criteria depend on clear user requirements. Natural language, with its inherent ambiguities, often introduces interpretational biases when describing requirements, making it impractical to directly derive test cases from such descriptions. Existing manual methods—including the Given/When/Then structure, rule-based criteria, and scenario-based approaches—are still widely used but are labor-intensive and prone to inconsistencies.

To address these issues, researchers have turned to software requirements modeling techniques [3, 4] such as Unified Modeling Language (UML). UML use case modeling combines visual diagrams with detailed textual specifications, offering a comprehensive view of requirements. Use case specifications, in particular, include a main success scenario and alternative/exception paths. Despite their strengths, the descriptions of use case specifications are only guided at the principle level, including the use of simple logic and freedom from design-oriented interactions. The means of describing them remains natural language. This gap underscores the need for better ways to bridge requirements engineering and acceptance testing.

Recent advancements in large language models (LLMs) offer promising opportunities to address these weaknesses in requirements engineering [5, 6]. LLMs, known for their ability to process and generate human-like text, can be fine-tuned through iterative prompting to perform complex reasoning tasks [7]. For example, researchers have explored using LLMs to generate acceptance criteria directly from user stories [8]. While these efforts are promising, providing a comprehensive solution for requirements characterization and testing as a whole remains a challenge. Moreover, most existing studies use LLMs only for natural language processing, limiting their ability to tackle the deeper structural and semantic challenges of defining test cases [9].