Software Testing: After Thought or Necessity?

It is easy to see what poor testing costs. eBay experienced a catastrophe in
1999. The catastrophe resulted in a 21-hour outage of their web site. That one
outage cost eBay $5 mn in revenue. The $5 mn loss precipitated an 11% drop in
share price. The outage affected 1.2 mn customers who were either trying to sell
or buy something at auction.

Another disaster was with Toys R Us who lost $12.5 mn, and spent $4 mn in
performance testing to try and prevent the lost. That represents a testing
investment of 25% against the risk with no measurable return. Toys R Us did not
get good value from their testing.

By contrast, businesses that test their software well can expect to invest
about 10% of the risk and reduce the actual failures to about 20%. Using the
Toys R Us example, if $12.5 mn is at risk, a testing investment of $1.2 mn
should hold the actual failure costs to about $2.5 mn.

Is Software Testing A Low Priority?

It is a fact that software testing was not the highest on the priority list
in the past. But that was the Mainframe era, and for legacy systems, when
vulnerability and complexity was low. Over the last two decades, open systems
and single users moved to multi-user environments, increasing the complexity
manifold. This brought testing into mainstream activities.

Says Venkatesh Chillara, Asst VP, Quality Consulting, Satyam, "The
industry has transcended from client-server environments to web-based
environments and risks increase exponentially at many levels such as
scalability, security, portability and recoverability in integrated as well as
distributed environments This made testing complexity extremely high. All this
led to the emerging criticality of early defect detection. Thus, testing changed
its position in the priority list and came to occupy a position of extreme
priority in software life cycle."

According to Deepak Rao, head, global R&D Practice, Blue Star Infotech,
"The independent software testing industry is riding on a high wave in both
the domestic and international markets. Initially, filliped by the Fortune 500,
banking and financial services and ISV segments, independent testing has now
caught the fancy of several large and medium sized organizations in varied
industries beyond the BFSI."

Wipro has a testing team of 2500. Testing has become so crucial that it has
been included as a new and independent service line at Wipro. The recently
concluded 'Wipro Strategym Roadshows' that were attended by 200-plus CXOs,
ranked testing services as the second most preferred area after ADM.

Cognizant has its own independent software testing practice that handles
across the board testing assignments. "We have done complex testing for
high velocity, complex applications that have thousands of concurrent
users," explains R Chandrasekharan, managing director, Cognizant.

Patni has a well-established Verification and Validation Center of Excellence
(V&V CoE) that balances excellent execution and ingenious innovation. The
CoE has built capability in various flavors of eTesting, remote performance
testing, usability testing, etc. The CoE has also established alliances with
leading tool vendors like Mercury and IBM Rational and has developed an array of
sourcing and training organizations.

"One of the good fallouts of the dotcom or technology slump is that CIOs
and CTOs are focusing on two key aspects-ROI and Reduction to bugs in
production," says NN Subramaniam, director, Maveric.

The Indian Testing Market

The size of the Indian testing market is estimated at $300 mn and is said to
be growing faster than the global average; the global testing market is
estimated at around $13 bn.

There are two different markets for testing in India. One is the end-user
institutions that comprise institutions and those having their back-office
facility in India. The 0technology spends by Indian financial institutions is
expected to exceed $1 bn in the coming year and therefore that should generate
roughly $150-200 mn in testing business. Typically, 20-30% of the engineering
budget is QA testing, says Anand Deshpande, CMD, Persistent Systems.

Industry experts point out that testing has seen excellent growth in India in
the past 18 months. Testing is expected to see growth rates in excess of 100%
for the next 5 years.

Essentially, the software testing market in India would include:

• Testing related to IT spending within the domestic market.
• Testing related to IT development currently offshored to India
• Independent dedicated testing services for large IT users for IT
  development activity across the globe.

Need for Independent Software Testing Companies

You can't let a fox guard the henhouse. How often have we heard developers
say "It works on my machine, you must be doing something wrong!" to
dismiss reported defects? According to Arindum Basu, VP and head of Independent
Testing Practice, Kanbay, the rigor, incisiveness and `nothing is holy belief'
ensures that defects are trapped before they hit the clients. Sharad Sharma,
vice president, product operations, Veritas Software India says, "the need
of independent software testing firms is required by the market space".

Independent testing brings many advantages to companies, asserts V
Chandrashekaran, CEO, Aztec Software and Technology Services. For one, it brings
in objectivity, rigor and transparency to the testing and defect reporting
process. In the process, companies can also gain time-to-market by outsourcing
parallel activities in the product life cycle development. Most importantly, it
provides checks and balances against the development team by being the 'eyes
and ears' of the end-user/client. Aztec Software and Technology Services
recently acquired Disha Technologies-a software product testing company.

India has around 8 to 10 independent software testing firms. SNS
Technologies, for instance, has focused purely on product testing since 1989.
Managing director Ashish Shah has big plans for his company. His company hopes
to touch the 250 people mark by 2005, and is targeting the UK market this year.

Verisoft was originally a division of Prabodhan Info Systems. This was hived
off as an independent company this year. Verisoft has completed over 139 testing
projects and has worked for more than 70 clients. There are other software
testing firms as well: RelQ, AppLabs, Maveric, ReadyTestGo, Quexst Associates,
Pure Testing.

Testing Challenges

Time, quality and cost. They are principal challenges, says Arindum Basu of
Kanbay. What makes it tricky is that the three go hand in hand and it frequently
means that to meet the goals of one of these parameters, one or more of the
other benchmarks have to be compromised.

According to Sarath Sua, managing director, Sierra Atlantic, product
development companies are increasingly finding it difficult to pack in more
features into the product with the pre-agreed quality standards. Developing
specific testing competencies remains a big challenge, he feels.

Product testing demands the availability of a multi platform test environment
at one location. Surveys show that 55% of projects get rolled back, 70% are late
and over budget, 65% of problems are the result of patches and 75% of projects
use excess hardware in the product development industry. Implementing and
testing fast changing business requirements against a slow moving development is
a big challenge the industry is trying to cope with, says Chillara of Satyam.

Another vital concern is data security and IP Protection of sensitive
personal and business data. Without real data usage, acceptance testing is
vulnerable to failures, say experts.

The Outsourcing Opportunity

Gartner estimates the global software testing market at $13 bn. The current
global market size of the outsourced testing services could be pegged at around
$6.1 bn and India has the potential to corner 70% of the worldwide market, says
Parminder Miglani, managing director, Keane India.

A META Group Study reveals that the testing business in India was $200 mn in
2003-04 and this could increase to $700 mn-$1 bn by the year 2007. IDC has
predicted the testing services market to be $2.6 bn by the end of 2004.
"So, India is beginning to play a dominant role in this business as
well," says V Chandrashekaran of Aztec.

Many MNCs and software product companies are looking at outsourcing their
application testing to Indian companies. Over 50% of the SCI CMM Level 5 IT
organizations in the world are located in India, points out Surendra Agarwal,
president, Neilsoft.

L&T Infotech has seen significant returns on investment by outsourcing
testing to an independent testing provider both in terms of low cost and high
quality output. Kiran Pai, GM, Securities, L&T Infotech reveals that a
larger part of the company's outsourced work includes regression testing and
testing of their ERP implementation.

“India is beginning to play a dominant role in this business as well” -V Chandrashekaran Aztec

According to Deepak Rao of Blue Star Infotech, outsourced software testing is
a low-risk model, where companies do not have to worry about data security, IP
and other issues which usually accompany outsourced development. It is a
non-core activity, which can be easily executed remotely. Outsourced
destinations like India provide an added advantage while providing superior
testing services and team ramp up and ramp-down flexibility, he explains.

Anand Deshpande, CMD, Persistent systems, however, believes there are many
aspects to outsourcing testing. Testing requires very sophisticated knowledge
and a lot of experience and expertise. One must realize that testing goes hand
in hand with product development. Currently, it is the lower end of testing that
is being outsourced, he says.

Domestic Opportunity

We expect a growth of 15-20% from the domestic software testing market.
"The domestic demand will come from the BFSI, government, healthcare and
manufacturing segments, which have the maximum IT spend," says, Sridhar
Kulashekaran, COO, ReadyTestGo.

This year has also seen the small and medium enterprises become the first
time purchasers of ERP and SCM systems and, in course of time, they will also
need testing services as SMEs will find it difficult to invest in in-house
testing, he adds.

Ashish Shah of SNS Technologies sees testing becoming the norm in the next
five years. All banks are centralizing their depositories and databases.
Moreover, information audits are moving from the manual to automated systems.
These require risk mitigation strategy, he points out.

Emerging Growth Areas

The spread of verticals for general outsourcing will continue to apply to
testing. This means that financial, banking, insurance and healthcare will lead,
says Chandrashekaran of Aztec. With the expected size of the testing market,
growth in opportunities will come from software companies that outsource core
product testing, he says.

With regulatory bodies across Europe and US introducing more stringent
compliance norms, this segment will continue to offer growth opportunities to
companies specializing in testing in this vertical, says Subramaniam of Maveric.

According to Pradeep Waychal, head, corporate quality and delivery
innovation, Patni, India is strong in the BFSI segment. The other growth
verticals are in the ISV segment and manufacturing, he says.

Agrees Kulashekaran: "Banks and Financial sectors are big investors in
IT. We see banks, financial institutions, insurance service providers,
independent software vendors, manufacturing, healthcare and life sciences as
domains with highest potential for testing services."

Nanda Kasabe inputs from Sunitha Natti, Hyderabad Priya Padmanabhan,
Bangalore

The Intel Pentium Bug

Enter the following in your PC calculator: (4195835/3145727)*
3145727-4195835. If the answer is not zero, you have an old Intel Pentium CPU
with the floating-point division bug

• Discovered in 1994
• Intel's test engineers had found the problem before release.
• When pressured, Intel offered to replace the faulty chip, but only if a
  user could prove that he was affected
• Ultimately, there was a public outcry. Intel had to spend $400 mn to
  replace the faulty chip

Testing Automation Cuts Costs for P&C Insurer

Fact File