PhD Defense


Akbar Siami Namin

Mutation Analysis in Software Testing


Thesis Examiners:

External Examiner:
Monday, August 18, 2008
9:30 a.m.
Middlesex College, Room 320
Dr. Jamie Andrews
Dr. Michael Bauer
Dr. Hanan Lutfiyya

Dr. Reg Kulperger (Statistics)
Dr. Gregg Rothermel (University of Nebraska)



Mutation is a fault-based technique by which we generate a set of slightly altered and possibly faulty versions (mutants). Mutation operators are a set of mathematically well-defined transformations simulating simple faults. We tune the quality of mutants by designing various types of mutation operators. Applying mutation operators, we are able to generate a considerably large number of mutants. These properties of mutation make it a suitable tool for conducting repeatable experiments. The idea, albeit interesting, opens up several challenging research questions.

First, we investigate the behaviour of mutants and faults in various circumstances. Research shows that for random-based test suites mutants are good representatives for real faults. We extend previous research by studying their similarities for coverage-based test suites.

Second, the thesis investigates the feasibility of using mutation in assessing the cost-effectiveness of testing techniques. Addressing the trade-off in cost-effectiveness of different criteria assists a test practitioner in choosing the most appropriate criterion.

Third, we tackle the infeasibility of mutation. Mutation is an expensive technique mainly caused by generating enormous number of mutants. The sufficient set problem is the problem of finding a smaller subset of mutants while maintaining the properties of the entire set of mutants. We determine the sufficient set for both random and coverage-based test suites.

The main theme of this thesis is empirical study and statistical analysis. The results show that mutants can be a good representative of real faults. We adapt several statistical techniques such as variable reduction and subset model selection for determining a sufficient set. The thesis introduces Cost-based Least Angle Regression (CbLARS) by which a set of best fitted regression models are constructed. pages must not be used by students. They are for internal CSD use ONLY.