Thesis Project
From theory to industrial application: a principled selection and evaluation of regression testing methods
My PhD research focused on the field of software testing, specifically on the Regression Testing Problem (RTP). Regression testing aims to verify whether all functionalities of a software system continue to work correctly after the introduction of new features or modifications. Such tests are crucial to detect regressions in the source code. However, they can be time-consuming, and when the test suite is large, it becomes necessary to decide which tests should be executed for each new version. To address this challenge, three main approaches have been identified in the literature: test prioritization, selection, and minimization.
Regression testing (RT) optimization has been the subject of extensive research, resulting in a wide variety of methods. However, in industrial contexts, it remains difficult to determine which of these methods best fit a given situation. This work tackles that challenge by proposing a taxonomy-based approach that helps practitioners characterize their RT problems, select appropriate candidate solutions, and assess their effectiveness in real-world industrial environments.
The main contributions are as follows:
- A taxonomy-based approach that builds on a recent taxonomy of RT concepts, enabling a systematic alignment between RT problems and candidate RT solutions.
- An empirical assessment of the approach through a real-world case study involving two software systems embedded in connected vehicles.
- Feedback on taxonomy usability, highlighting strengths and areas for refinement.
- Insights into real-world constraints encountered when evaluating RT methods, along with the corresponding evaluation results.
The proposed approach proved effective in identifying suitable RT methods for the studied problem. However, this success came with challenges, such as ambiguities and omissions in the taxonomy and occasional misclassifications of methods. We conclude that the alignment between problems and solutions is indeed valuable in principle, but that the taxonomy itself requires further maturation. Our work thus highlights several directions for improvement.
This thesis was conducted in collaboration between Renault Group and the LAAS-CNRS (Laboratory for Analysis and Architecture of Systems). At LAAS, I was a member of the TSF group (Dependable Computing and Fault Tolerance). The research was supervised academically by Hélène Waeselynck and guided industrially by Fernand Cuesta.