Mutation Testing – PHP Web Applications

We covered the basics of mutation testing. Now it’s time to have a look at why you may have never seen it in any web application you’ve worked on and what tools are available to use.

While sophisticated software testing is available for web technologies and also used in applications, Mutation Testing as a method to evaluate test quality is not. Suffice to say, there are only few publications concerning web related software. A point apparently overlooked is though, that not only more and more applications are moved to the domain of the web, but their importance and demand for reliability increases, too. Just looking at cloud services, it is possible for an entire company to rely on such software. Obviously, test quality should be a concern for those applications, too. After all, when using an object oriented programming language (e.g. PHP, JSP, JavaScript) and given a maintained and commercially used system, needs and problems are very similar compared to those of a non-web technology. Regardless, it took until 2010 when Praphamontripong proposed a technique to apply mutation testing to a set of websites in the form of HTML documents [1].

In practice, despite of a variety of available alternatives, PHP based software is and has been the foundation for many commercial online services.

And because PHP has been around for quite a while now, systems have often been developed and maintained for a longer period of time than other web software built on newer or more short-lived technologies. The object orientation of PHP allows for complex structures that have to be tested as thoroughly as any other software would. Seeing that, PHPUnit, a popular testing framework with built-in code coverage metrics for PHP, enables developers to create tests with feasible effort. Concerning test quality assessment then, Danawale and Kadam [2] finally proposed a mutation testing tool for the PHP language in their publication in 2016. Coupled with this relatively weak research interest, hardly any tools or frameworks are available which further hardens the breakthrough of mutation testing in web development.

Tools

The first Mutation Testing tool for PHP was Humbug written by Pádraic Brady and initially made available in 2015. While it was the only tool available, it offered downsides that eventually led to the development of Infection, initialized in 2017 by Maks Rafalko. In the meantime, Humbug has been abandoned, is no longer maintained and suggests Infection as the only available alternative for Mutation Testing PHP code.

Luckily, Infection is thriving and under active development. The tool is available as a composer package and ready to use for continuous integration processes.

References

[1] Upsorn Praphamontripong and Jeff Offutt. Applying mutation testing to web applications. In Software Testing, Verification, and Validation Workshops (ICSTW), 2010 Third International Conference on, pages 132–141. IEEE, 2010.

[2] Jyoti J Danawale and Sandeep Kadam. Survey on different approaches for mutation testing. 2016.