The Bug Busters

Ramakrishnan, a retired schoolmaster in Egmore, almost fainted when he
received a telephone bill of Rs 18499.50 from the Chennai Telephones a year
back. It was almost 30 times more than his usual monthly bill and it also had
logs of calls that were never made by him or anyone in his family.

There was commotion at Ramakrishnan’s home till the telephone authorities
explained about "an error in the billing software".

In another incident, far away in the United States, a software failure
paralyzed the New York Mercantile Exchange and telephone service to several East
Coast cities in February 1998 leading to huge public outcry.

Instances such as these highlight the mission critical nature and the human
dependence on software.

 According to estimates made by the US-based National Institute of
Standards and Technology (NIST), the economic impact arising out of faulty
software in the US alone range in the tens of billions of dollars and constitute
1% of its GDP. An error in software is commonly referred to as bugs. The impact
of defective software is pervasive and the ramifications are not just limited to
the enterprises, but also at the individual level.

For instance, a function every desktop user does many a times during the day
is double clicking on an icon to open a program or an application. But sometimes
the PC freezes and the user has to restart and in the bargain loses all the
unsaved information. The problem most likely is due to a bug deep down in the
OS. This happens on PCs, which have not had updates, but still a flaw embedded
in the primary version, installed by the user.

Why Testing?

Testing is not a new phenomenon, but has been an embedded task in any
software development exercise.  But with computing becoming pervasive, the
workflow and functionality of the IT solutions have become imminent for meeting
the wide-ranging business requirements of the enterprise. For instance, when
manufacturing resource planning (MRP) of the 1980’s graduated into ERP in the
1990’s, it seemed that ERP was the panacea to all enterprise IT problems. But
in the formative years of ERP boom, many implementations failed for two reasons
- one the ERP solution would be very rigid and hence it failed to interface with
existing processes. The second major pain area was the solutions conflict with
the country specific issues. The failed implementations are the result of lack
of functionality testing, both by the vendor and the end user at the BPR level.
The hypothesis is clear; for any vendor, testing becomes integral and they can
fix the bugs inherent in their solution by entrusting the activity to a third
party tester. The testing companies rationalize that the third party tester is
actually a beta user of the product, who drills down on the various layers of
the solution and find critical faults.



They recommend appropriate patches and remedies for the loopholes in the
solution. A third party tester makes more sense, as in most software development
companies the programmer himself plays the role of a tester and the possibility
of error identification is very less.

"Basically applications have fallen short on the required functionality, performance and usability factors..."

N Mahesh, executive director, Maveric Systems

Recently, both the vendors and the end users have started realizing the
benefits of testing their solution mainly because the economic impact of a
software failure have been resulting in huge cost implications.

The Big Picture–Outsourcing Edge

According to Gartner Group, the global software testing market is estimated
at $13 billion (Rs 62,000 crore).  Industry estimates pegs the current
global market size of outsourcing testing services at around $3 billion.

Says Sashi Reddy, CEO, AppLabs, "Since testing is one of the safest and
most non-intrusive activity to outsource, there is a significant amount of
testing business coming to India. This business is across all types of
industries, but the financial services sector has been one of the most
aggressive in outsourcing in this space."

AppLabs for instance is India’s leading software testing company with over
350 employees. It is also the only CMM-Level 5 independent testing company in
the country.

Says Aswini Kumar, CEO, Thinksoft Global Services, " In the past
enterprises have to rely on in house testing done by their IT vendors. But this
has not been producing an acceptable quality of software at a sustained level.
Testing which is one step in

the value chain of IT application development and roll out is now being viewed
as a potential candidate for outsourcing."

India has an enormous opportunity to corner the testing market. So
outsourcing-testing services will become a big opportunity.

Says Reddy, "Entrusting testing activity to the same developers who do
the programming leads to disastrous results, as they tend to overlook
things."

The emergence of pure play testing companies is creating a new business
domain in Indian software space. Unlike routine software development companies,
the testing companies operate at a totally different level. Take the case of
Ready Test Go (RTG), a Chennai-based performance engineering and software
testing company that provides testing services to enterprises like Citibank,
Deutsche Bank, SBI, American Express, Dendrite, IFlex among others.

Says Sridhar Kulasekharan, COO, RTG, "Development companies do not
adequately invest in architectural benchmarking. As a result, problems are
usually detected at the eleventh hour.  Without rigorous testing
capabilities in-house, the end product or solution delivered to the customer
does not perform in line with SLAs." 

Agrees N Mahesh, executive director, Maveric Systems, "Quite a few
questions have been raised with regard to ROI on IT investments in the last
couple of years.

Today several large Indian technology companies have started depending on
independent testing vendors to certify their software before deployment with end
customers. Similarly end users like banks have now started, in a small way, to
use the services of independent testing vendors to test the software provided by
technology companies. These include tests for scalability, as well as various
functionalities promised by the provider of software.

While there is no specific study made so far on the Indian market size for
outsourced testing, many believe the opportunity is huge. And the pure play
testing companies clearly have the early mover advantage in this space. As
enterprises expect more from every IT implementation, the vendors are
increasingly facing problems meeting their expectations. Given that, more and
more software development companies will outsource the testing part, and this
likelihood creates immense opportunity to pure play testing companies in India.

Shrikanth G in Chennai

Testing Nuts And Bolts

Testing happens at every stage of the development process. The testing team
checks for the functionality of each module of the software product initially.
Defects tracked are classified, based on the priority and reported to the
engineering team. Once the defects are rectified, a ‘regression testing’ is
done to check the software. In the next stage, all the modules are tested after
they are integrated, which is known as ‘integration testing’. Upon
rectifying the defects after integration, the software is tested across multiple
operating systems, configurations for installation and functionality issues.
This entire process can be broadly termed as ‘Functional Testing’. In the
next stage, called Performance or Load Testing, testing is conducted to evaluate
the compliance of a system or component with specified performance requirements.
Often this is performed using an automated test tool to simulate large number of
users. This process is known as ‘Performance Testing’ or ‘Load Testing’.