How much Testing is required?

Another Million-dollar question. No amount of testing is good enough to say that an application is completely bug-free. In fact, there is one confirmed, common fact in all software development projects and the bug is ‘there will be defects’. There is no software application that doesn’t contain any defect.

Probably, by the way, our (programming) languages are structured or the way in which the requirements are defined or the process in which we agree and approve what we perceive is required – all these ensure that there will be a number of defects in software development.

Wikipedia lists out these different methods of testing – Black box testing, White box testing, Grey Box Testing, Acceptance Testing, Non Functional Software Testing.

But, do all these testing assure that an application is clear after these are completed? Not probably. So, where does one stop?

It is essential for the Customer to be involved while creating the test-cases that are absolutely essential to be successfully passed for go-live. This is the last toll-gate where one can notice the features of the application and compare it with what was perceived. Hence, creation of test-cases are very important.

Of course, one can also go with the assumption that the application will be maintained in the long-run and hence, prioritize the test-cases that will test for 90% of the functionality. Once the application goes live, the remaining 10% can be covered. One has to strike a balance on the amount of testing required with the functionality that is covered so that the point of diminishing returns is not reached.

Of late, the trend has been to create an independent team that will work in parallel with the development team (reporting to a different Manager). This is done to give a competitive edge to the development life-cycle. I am not so sure about the benefits of having this team as the merits of having a combined team is much more than the other option.

Even during Acceptance Testing, there has to be an agreement in place with the Customer about the number of defects that can be ‘agreed’ for an application to go-live (also called in some places as Acceptance Criteria). Typically, the customers would agree for

• No Fatal,
• Few Majors (dependent on the complexity of the application),
• Minor defects (cosmetic mainly)

This will ensure that the testing will stop when this threshold is reached. Then, only a verification of these defects will be done and the application is allowed to go-live. Otherwise, the testing team will continue their cycles without knowing when to stop.

Regression is another aspect to consider. Most often what happens is that the team does not worry about regression at the beginning of the project. At a later point of time, they realize that the tests should be executed whenever there are changes to the application (big or small) – to check whether whatever was working earlier has been broken.

Whatever be the means of testing methods selected for a project, the Project Manager has to budget appropriate time (as per some of the pointers mentioned above) for testing his application.