A Survey on Mutation Testing Methods

Software testing is an important but expensive process because about fifty percent of the total development cost is spent for it. However, this part is the first one to miss by software developers if there is a limited time to complete the project. Tests are commonly generated from program source code, graphical models of software (such as control flow graphs), and specifications/requirements. Creating test cases manually is a huge work for software developers. It is time consuming and error prone. A solution which automatically generates test cases and test data can help the software developers to create test cases from software designs/models in early stage of the software development (before coding). High quality software can not be done without high quality testing. Heuristic techniques can be applied for creating quality test data. Exhaustive testing of a software is not possible so different techniques are invoked. Mutation testing is a powerful testing technique for generating software tests and evaluating the quality of software. Mutation testing measures how "good" our tests are by inserting faults into the program under test. This article presents a survey on various mutation testing tools available. Basic concepts and notations of mutation testing as well as how mutation testing works are described. Problems of automatic generation are identified and explained. Finally important and challenging future research topics are presented.