How Agile, MVP, Cloud and DevOps are Transforming Performance Testing Practices

For decades, QA is primarily thought of as a gatekeeper. For the product to reach the customer, it has to go through this gate. The underlying tenet is; the product comes to the gate after it is fully done. This model is completely disrupted by new developments and as a test or software engineer; do you deeply understand these methodologies and underlying technologies? Does your organization have modern platforms and tools to deliver the results?

Ready to Ship mode
In the early 90s, one of most dynamic software organizations of its time started experimenting with rather radical ideas. The most controversial of those is; always be in ‘Ready to ship’ mode. In this model, the product team has to ship the product on very short notice. The misunderstood (and controversial) part of this idea is the role of QA. For this model to work, QA needs to be an integral part of product development.

The advantages of this are that product teams are much more agile with adding/matching competitor features, and production costs are much lower. The emphasis is on producing ‘good enough’ product which meets certain criteria.

Test-Driven Development
The idea spread and by late 90s and early 2000s many concepts were pioneered; Test-Driven Development, Xtreme programming and others. However, agile methodology was the most widely adopted standard and QA was central in making this popular. For instance, in the earlier models, product design is done at the beginning of product cycle and some critical testing like performance testing was done at the end. If performance tests revealed an issue that needs a major design change, then it’s a major endeavor to incorporate the fix and most certainly the release dates will be affected. Agile, when properly done, solves these issues.

Minimum Viable Product
Minimum Viable Product (MVP) emphasizes things like keeping the feature list short/simple, build products in an iterative cycles, and keeping in close touch with consumers even at the very early stages of product development. QA processes need to be aligned with agile methodology to be successful with MVP strategy.

Performance testing
Performance testing in an agile environment is truly about performance. Not load per se, but how an application performance (transaction times) at each build. And is there continuous improvement in those metrics. Performance testing workloads are one of most suited applications for cloud computing. Performance testing generally needs substantial computing power, the need for these computing resources is intermittent, and for some very short periods, performance testing teams needs really huge amounts of computing resources. If we setup lab resources for this peak load, then for most part of the year around 80% of the resources are not used and rest of them are marginally used. Moving these peak loads to cloud drastically reduces operation costs, by cutting the need for capital expenditure on lab, and personal costs to maintain the software and hardware.

And lastly, the most popular buzzword today is DevOps. In 2009, when I was working as application performance management consultant, my favorite gripe with IT managers is how siloed engineering, QA and OPS teams were. And how easy it is to reach performance goals, if they work more closely. I was not aware that a revolution is brewing to fix that gap. Major breakthroughs came when IT organizations started visualizing infrastructure as code. Now, a complex IT environment can be encapsulated in a ruby or JSON script. This script can now be version controlled, shared with anybody in an email, and updates/upgrades can be added by just adding lines in a script. The DevOps phenomenon accelerated IT release cycles, by removing complexities/time in provisioning hardware and software to test, host and run the software.

QA organizations, as always, are in the center of all these exciting changes happening in the IT world. And the way QA reacts to these changes determines to a large extent whether your company can leverage these technologies and prosper in the future.