EkaThe Story of an Indian Champion

October was cruising toward its close. The clock was viciously steering
toward the midnight stroke. Sweat-swathed eyebrows, nervous eyeballs, twitching
palms, pounding heartbeats, crossed fingers, distraught glances, and agitated
minds. The air was bubbling with tension, anticipation, fears, prayers, and
tumultuous hope. An unknown campus in Punes Hinjewadi outskirts was literally a
hairs distance from making history. It was nothing short in stature or
excitement though. The 90% run of Indias supercomputer-in-making was
successfully through, but the clincher was actually nowthe last 10% run-time.
It might happen, it might not. Some kilometers away on his way and continuously
on the phone, Dr N Seetha Ram Krishna, project manager, CRL and a key architect,
understandably still kept arming his team against the Murphys ways, It may
fail, be prepared for everything. As the reverse countdown began, every heart
and hope in the jitter-packed room started racing high. Five, four, three, two,
one and

Yes! The supercomputer hit the 117.9 teraflop mark. At about 11 pm on October
31, at a TAT facility in Punes Hinjewadi IT park outskirts at CRL, shrieks of
joy, sighs of achievement, and euphoria was all that could be heard, seen and
felt. Indias technological razor had made its sharpest cut again. The dream was
finally alive. And one hour later when Dr Krishna looked around the same room he
met another once-in-a-lifetime sight. Exhausted with 22 hours of grinding toil
for the past six months and worn out of a peak of excitement and tension just
some minutes back, everyone in the same room dozed off into a blithe and
well-accomplished sleep. Thats a lifetime experience.

It surely was. India, through CRL (Computational Research Labs), a Tata Sons
wholly owned subsidiary, had claimed its space in the world top 10 supercomputer
league. The fourth in the global ranking and fastest in Asia. The 120 teraflop
(sustained rating) supercomputer with a peak hit of 172 teraflop was actually a
reality in October 2007, with the added pride of being the largest privately
funded supercomputer in the world.

Eka Incubates

Tatas HPC (high performance computing) initiative dates back to June 2006 with
the aim of becoming the one-stop-shop and achieving the iconic journey from
atoms to applications. It armed CRL, its subsidiary, with the mandate of the Eka
(Sanskrit for the number One) Dream. This seventy-five-member team, which was
divided into hardware, system software and applications, had beyond the obvious
challenge of achieving the supercomputing power as set, also the nigh-impossible
goalpost of doing all that in flat six months. It had Dr Sunil Sherlekar, head,
R&D, CRL, and also one of the founders who, with Dr Narendra Karmarkar,
initially developed this concept, made the business plan and presented it to the
Tata Sons Board to get the funding. Since then Dr Karmakar left and Dr Sherlekar
stayed as the remaining founder.

Incidentally, Eka also claims the distinction of being the only supercomputer
funded by a corporate. CRL had the task of fully integrating and designing Eka
with an in-house developed technology.

The race flagged off. In June 2007, Hinjewadi (a 4,000 sq ft floor area set
in record time) became the data center to house Eka. August saw the initial
machines with 16 teraflop peaks and the first prototype going operational.
September saw the building blocks being ordered, set in place in due time and
October saw the 172 teraflop peak system operation. In six weeks record time,
the actual 120 teraflop (or trillions of calculations per second performance)
happened, and Eka was born.

Eka is built with 1,794 blade servers using common off-the-shelf hardware
using quad-core Intel clovertown processors. It has 400 ton cooling capacity and
2.5 MW power requirement. Its benchmark is 117.9 teraflop, and achieved final
performance of 120 teraflop on a sustained basis with 172 teraflop as the peak
score. There are 28 terabytes of memory with a storage counterpart of 80
terabytes.

The Revolution

There were quite many new approaches and dimensions during the making of Eka.
The key was in the design of the architecture, then designing algorithms for
application classes (a term that itself is nascent in the scientific domain and
an ongoing research game), and the mapping of algorithms to the architecture.
CRL also chose the not-tried-before circular layout that serendipitously took
the form of an octagon for stacking servers and switches.

Supercomputers dont come without their share of burdens. Scalability is a
major issue. CRL addressed scalability by doing away with the need of connecting
all nodes to all. Judicious balancing, computing and communicating, and
guaranteed load balancing were some of the ideas attempted here.

Another concern was the cost per teraflop which the Eka team managed to
handle with reduction in interconnect switches, cables and connectors so that
interconnect scales up linearly. Similarly, usability, another issue with
supercomputers, was faced in the eye with innovations in library and mathematics
kernels, thus, hiding complexity of the underlying hardware, as shared by Seetha
Ram Krishna. Additionally were solutions like novel interconnect architecture
based on projective geometry that takes care of complexity as well as near
linear speed-up of applications. There were also better algorithms for specific
applications that helped Eka.

Props and Poles

As Dr Sherlekar points out, a fair share of credit of Eka goes to enabling
technologies in networking, storage, power, and cabling. While some of them had
evolved to an adequate extent already, thanks to ongoing innovations, some of
the path-breaking innovations that made Eka possible, in fact, happened during
the making of Eka, with the concerted efforts of CRL and other partners in
respective areas. Cabling, for instance, is one major example.

Eka became the testing ground of early fibercable technology since it was not
feasible to use erstwhile copper cables beyond seven meters due to breakage
concerns. The new technology that covered twenty meters is already in the
production mode now. From processors, multi-core, programmable voltage,
Interconnect Infiniband sensors to cooling technology, every paraphernalia was
as cutting-edge as Eka itself turned out to be.

Milking the Cow

The actual work starts now when the supercomputer can start working on
problems that have been waiting for the power of a superhero. Ekas usage is
outlined wide and deep. Its applications, current and potential, cover a long
spectrum. From system architecture research, system software research,
mathematical library development, large scientific problems, application
porting, optimization and development to future technology development and data
center development, the possibilities have just started surfacing. Talking of
simulations, Ekas purview covers computational fluid dynamics, nanomaterial
simulation.

As head, Applications Group, Dr Rajendra Lagu rightly says, The young
scientific minds are the ones to be looked onto now. There are for sure many
applications that nobody has ever thought of yet. Eka is open for grand
challenges. The exciting areas ahead range from aeroacoustics, weather modeling,
carbon nanotube modeling, CFD or computational fluid dynamics, number theory,
Motif discovery, molecular docking, aircraft simulation, bio-medical simulation,
to business applications like SCM (supply chain management), BI (business
intelligence), email scanning, pattern detection, video surveillance and so on.
Maybe the next Tata car would be completely simulated on Eka.

Some Grey Matter

Eka is not without its share of criticisms and correction points. At a CSI
seminar, an elderly and senior scientist in this domain pointed out areas like
use of analog devices that were conspicuous by their absence in Eka; the complex
use of FPGS; the perception that differential calculus is the ultimate in
mathematics. CRLs explanation to them goes thus. The latter is still a topic of
research and even CRL is doing work on inverse problems in this area. As to the
use of analog devices, there are constraints on programmability and limits of a
fixed function in an era where digital is in vogue. Even switched capacity
filters, the hybrids are not in vogue, explained Dr Lagu in response.

The Human in it

The real beauty behind the mystery of Eka, the supercomputer, is the very
absence of mystery. As Seetha Ram Krishna emphatically demystifies it, Any
company can achieve this. Theres nothing non-enablingly complex behind it. Just
some knowledge, expertise, logic, ideas and electronics put together in a system
I can make possible. Today, with the Internet as the ever-accessible ocean of
knowledge and science, anyone can make their own supercomputer, albeit with
levels of scale, he believes.

The future of supercomputing is something that will unfold interestingly. A
supercomputer is a force multiplier and the third pillar of science. It is
becoming all-pervasive to the games we play and the digital content we consume,
says Dr Lagu.

The Top 500 supercomputer list would remain on Eka and CRLs dashboard. But
the list, updated every six months, would be as fickle as the speed with which
faster systems keep popping up. The top-ranker IBMs BlueGene/L System has
achieved a benchmark of 478.2 TFLOPS. Pitted against computing superpowers like
China, Japan, and the US computing, Indias sprint will be something to watch
out for. Competition with the likes of Chinas Dawning, Japan NECs Earth
Simulator, and IBMs Blue Gene stirps would not be easy. The race is going to
turn ruthless but will surely have India on the fast tracks. Next on the cards
for CRL are plans of building a bigger machine that incorporates accelerators in
addition to building domain specific software libraries tuned for Eka
architecture. Besides, innovations on the software stack and productivity tools
for the imminent many-core revolution are on the anvil.

Whats a factor of concern, however, is the critical need of development of
human resources in this space. It was tough to make about a 75-80 member Eka
team and we know how we did it. The future problems wont be hardware or
software, but peopleware, Dr Lagu stresses.

Eka and its likes could buck the trend and drive more Indian cerebra to
experience the inimitable thrill that CRLs team experienced that historical
night. After all, it was indeed an eka moment. Its intriguing at this juncture
then to think of some words that the father of supercomputing, Seymour Cray,
said: Its always easy to do the next step and its always impossible to do two
steps at a time.

Pratima Harigunani

maildqindia@cybermedia.co.in

(Acknowledgements to Dr Sunil Sherlakar, head, R&D, in CRL; Dr Rajendra Lagu,
head, Applications Group, CRL, and N Seetha Ram Krishna, project manager, CRL,
all key members of Eka team and CSI Computer Society of Indias Gireendra
Kasmalkar Pune for enabling the sharing of the Eka story)