Using Rule Mining for Automatic Test Oracle Generation

Abstract

Software testing is essential, but also one of the most costly and time-consuming activities in the software development process. The mechanism to determine the correct output of the system under test for a given input space is called test oracle. The test oracle problem is a known bottleneck in the software testing process. It describes the sophisticated task of distinguishing between correct output and faulty output of a system under test, for which no universal solution exists that works for any system under test. One of the artifacts that can be used to derive test oracles automatically is a previous version of the system under test. Such oracles can be used to validate new versions, for example, the versions where some corrective modifications were made. In this thesis, we develop a methodology for test oracles generation using internal state data extracted from the system under test during the test suite execution. We derive a test oracle in the form of rules using the association rule mining technique. The oracle can be used to validate new or modified versions of the system under test. We test our method on a Stack class from a custom version of the Java collection classes and discuss the lessons learned during our experiments with the class under test. We were able to show the feasibility of our method, but it has several disadvantages that restrict its capability to generate oracles. We also discuss the limitations of our study. As a main result, we were able to develop a new method and an experimental setup to generate test oracles using internal states of the system under test. We were able to derive a partial test oracle for the Stack class using our method.