Want good, cheap, and fast software delivery? Budget for testing!

An adage in the IT services world says, “We offer three kinds of service—good, cheap, and fast—but you can only pick one.” Shifting testing into the heart of an agile development project takes us some way to the holy grail: high-quality outcomes on time and within budget.

Scope, cost, and time are the three competing constraints when delivering a software project (Exhibit 1). The problem is that everyone still wants all three! Testing is no longer an afterthought to software development; unfortunately, many organizations consider it expensive, particularly when they view it as the direct cost of fixing a bug rather than mitigating business risks. Businesses need to consider the primary and secondary business failure costs that software issues cause. The primary cost is a direct cost to the business as a result of the defect, and the secondary cost is the cost resulting from stopping the service users’ business activities. Thankfully, Agile software development methodologies, testing automation, and testing as a service (TaaS) offer solutions for improved quality and, if done correctly, cost savings.

Exhibit 1: The Constraint Tug of War

Source: HFS Research

In the ultra-competitive digitally enhanced world, customers will increasingly make buying decisions on the quality of a brand’s apps

As apps’ importance to buyers increases, software development is now crucial to almost all businesses. At the core of good software is a solid testing function with a well-structured framework supporting it. Modern digital services consumers are fickle; if something is slow or doesn’t work well, they will not hesitate to look elsewhere for a platform that meets their expectations.

National Institute of Standards and Technology found that software bugs cost the US economy $59.8 billion annually

A software bug discovered in the production environment A famous example in the UK was in April 2018, when TSB (a major UK high street bank) locked out almost 1.9 million customers during a major system upgrade. Some customers were presented with incorrect balances, and others were shown accounts belonging to other customers. This failure cost the bank £300 million and caused 80,000 customers to switch to a competitor. Imagine if a fraction of this lost revenue had been directed toward a more robust testing strategy!

In the hunt for efficiency and timeliness, companies often take risks that they misunderstand, or they fail to hold the risk-takers accountable for the consequences of their decisions. In September 2014, Apple had to pull its iOS 8 update due to several software bugs. Its share price on Nasdaq fell on the same day (approximately $60 million!).

Don’t treat testing as an afterthought; it must be a front and central part of your SDLC

What the agile development methodology and agile testing have brought to the party is the ability to test iteratively or continuously so that costly defects are discovered and fixed early. As Exhibit 2 shows, once a defect makes its way to production, its resolution cost grows exponentially. Although the National Institute of Standards study is from 2002 and based on waterfall delivery, the principle is still valid—a production defect will have a much higher business impact. The importance and effort put in to quality assurance have steadily increased, as is clear from the World Quality Report.

However, the report also shows a decrease in QA budget allocation from 35% in 2015 to 23% this year; in part, budgets decreased because the line between testing and development has blurred.

Exhibit 2: How much does a defect cost?

Source: National Institute of Standards and Technology

Automating this will result in cost savings

An example of where the testing function becomes comparatively expensive is when software development is not at the core of the company’s business. The costs of hosting test environments, buying the latest testing tool software, and keeping a team employed soon rack up. In this situation, finding a provider that offers TaaS will bring quality improvements and cost savings. It makes much more sense to pay for a service when you need it, especially when experts with the latest tools provide it. If software is at the heart of what you do, bringing in a third party to run your QA function still makes good sense. It can still be cheaper than hosting the function in house, and it stops any conflicts of interest between developers and testers if they are sourced from the same team.

The Bottom Line: We need to evolve the budget conversation beyond cost.

Although good, cheap, and fast have traditionally been contradictions for IT projects—quality is largely defined by how you view the word “cheap.” Software that fails and causes primary and secondary business losses is not cheap. Having a good testing regime may have more upfront costs, but it will ultimately be cheaper if it generates customer satisfaction and works for your customers and staff. Having the right tools, people, and processes can ensure a secure and satisfying customer experience that brings return customers to your business and justifies the expense of doing it.