EkaThe Story of an Indian Champion

October was cruising toward its close. The clock was viciously steering

toward the midnight stroke. Sweat-swathed eyebrows, nervous eyeballs, twitching

palms, pounding heartbeats, crossed fingers, distraught glances, and agitated

minds. The air was bubbling with tension, anticipation, fears, prayers, and

tumultuous hope. An unknown campus in Punes Hinjewadi outskirts was literally a

hairs distance from making history. It was nothing short in stature or

excitement though. The 90% run of Indias supercomputer-in-making was

successfully through, but the clincher was actually nowthe last 10% run-time.

It might happen, it might not. Some kilometers away on his way and continuously

on the phone, Dr N Seetha Ram Krishna, project manager, CRL and a key architect,

understandably still kept arming his team against the Murphys ways, It may

fail, be prepared for everything. As the reverse countdown began, every heart

and hope in the jitter-packed room started racing high. Five, four, three, two,

one and

Yes! The supercomputer hit the 117.9 teraflop mark. At about 11 pm on October

31, at a TAT facility in Punes Hinjewadi IT park outskirts at CRL, shrieks of

joy, sighs of achievement, and euphoria was all that could be heard, seen and

felt. Indias technological razor had made its sharpest cut again. The dream was

finally alive. And one hour later when Dr Krishna looked around the same room he

met another once-in-a-lifetime sight. Exhausted with 22 hours of grinding toil

for the past six months and worn out of a peak of excitement and tension just

some minutes back, everyone in the same room dozed off into a blithe and

well-accomplished sleep. Thats a lifetime experience.

It surely was. India, through CRL (Computational Research Labs), a Tata Sons

wholly owned subsidiary, had claimed its space in the world top 10 supercomputer

league. The fourth in the global ranking and fastest in Asia. The 120 teraflop

(sustained rating) supercomputer with a peak hit of 172 teraflop was actually a

reality in October 2007, with the added pride of being the largest privately

funded supercomputer in the world.

Eka Incubates



Tatas HPC (high performance computing) initiative dates back to June 2006 with
the aim of becoming the one-stop-shop and achieving the iconic journey from

atoms to applications. It armed CRL, its subsidiary, with the mandate of the Eka

(Sanskrit for the number One) Dream. This seventy-five-member team, which was

divided into hardware, system software and applications, had beyond the obvious

challenge of achieving the supercomputing power as set, also the nigh-impossible

goalpost of doing all that in flat six months. It had Dr Sunil Sherlekar, head,

R&D, CRL, and also one of the founders who, with Dr Narendra Karmarkar,

initially developed this concept, made the business plan and presented it to the

Tata Sons Board to get the funding. Since then Dr Karmakar left and Dr Sherlekar

stayed as the remaining founder.

Incidentally, Eka also claims the distinction of being the only supercomputer

funded by a corporate. CRL had the task of fully integrating and designing Eka

with an in-house developed technology.

The race flagged off. In June 2007, Hinjewadi (a 4,000 sq ft floor area set

in record time) became the data center to house Eka. August saw the initial

machines with 16 teraflop peaks and the first prototype going operational.

September saw the building blocks being ordered, set in place in due time and

October saw the 172 teraflop peak system operation. In six weeks record time,

the actual 120 teraflop (or trillions of calculations per second performance)

happened, and Eka was born.

Eka is built with 1,794 blade servers using common off-the-shelf hardware

using quad-core Intel clovertown processors. It has 400 ton cooling capacity and

2.5 MW power requirement. Its benchmark is 117.9 teraflop, and achieved final

performance of 120 teraflop on a sustained basis with 172 teraflop as the peak

score. There are 28 terabytes of memory with a storage counterpart of 80

terabytes.

The Revolution



There were quite many new approaches and dimensions during the making of Eka.

The key was in the design of the architecture, then designing algorithms for

application classes (a term that itself is nascent in the scientific domain and

an ongoing research game), and the mapping of algorithms to the architecture.

CRL also chose the not-tried-before circular layout that serendipitously took

the form of an octagon for stacking servers and switches.

Supercomputers dont come without their share of burdens. Scalability is a

major issue. CRL addressed scalability by doing away with the need of connecting

all nodes to all. Judicious balancing, computing and communicating, and

guaranteed load balancing were some of the ideas attempted here.

Another concern was the cost per teraflop which the Eka team managed to

handle with reduction in interconnect switches, cables and connectors so that

interconnect scales up linearly. Similarly, usability, another issue with

supercomputers, was faced in the eye with innovations in library and mathematics

kernels, thus, hiding complexity of the underlying hardware, as shared by Seetha

Ram Krishna. Additionally were solutions like novel interconnect architecture

based on projective geometry that takes care of complexity as well as near

linear speed-up of applications. There were also better algorithms for specific

applications that helped Eka.

Props and Poles



As Dr Sherlekar points out, a fair share of credit of Eka goes to enabling

technologies in networking, storage, power, and cabling. While some of them had

evolved to an adequate extent already, thanks to ongoing innovations, some of

the path-breaking innovations that made Eka possible, in fact, happened during

the making of Eka, with the concerted efforts of CRL and other partners in

respective areas. Cabling, for instance, is one major example.

Eka became the testing ground of early fibercable technology since it was not

feasible to use erstwhile copper cables beyond seven meters due to breakage

concerns. The new technology that covered twenty meters is already in the

production mode now. From processors, multi-core, programmable voltage,

Interconnect Infiniband sensors to cooling technology, every paraphernalia was

as cutting-edge as Eka itself turned out to be.

Milking the Cow



The actual work starts now when the supercomputer can start working on

problems that have been waiting for the power of a superhero. Ekas usage is

outlined wide and deep. Its applications, current and potential, cover a long

spectrum. From system architecture research, system software research,

mathematical library development, large scientific problems, application

porting, optimization and development to future technology development and data

center development, the possibilities have just started surfacing. Talking of

simulations, Ekas purview covers computational fluid dynamics, nanomaterial

simulation.

As head, Applications Group, Dr Rajendra Lagu rightly says, The young

scientific minds are the ones to be looked onto now. There are for sure many

applications that nobody has ever thought of yet. Eka is open for grand

challenges. The exciting areas ahead range from aeroacoustics, weather modeling,

carbon nanotube modeling, CFD or computational fluid dynamics, number theory,

Motif discovery, molecular docking, aircraft simulation, bio-medical simulation,

to business applications like SCM (supply chain management), BI (business

intelligence), email scanning, pattern detection, video surveillance and so on.

Maybe the next Tata car would be completely simulated on Eka.

Some Grey Matter



Eka is not without its share of criticisms and correction points. At a CSI
seminar, an elderly and senior scientist in this domain pointed out areas like

use of analog devices that were conspicuous by their absence in Eka; the complex

use of FPGS; the perception that differential calculus is the ultimate in

mathematics. CRLs explanation to them goes thus. The latter is still a topic of

research and even CRL is doing work on inverse problems in this area. As to the

use of analog devices, there are constraints on programmability and limits of a

fixed function in an era where digital is in vogue. Even switched capacity

filters, the hybrids are not in vogue, explained Dr Lagu in response.

The Human in it



The real beauty behind the mystery of Eka, the supercomputer, is the very

absence of mystery. As Seetha Ram Krishna emphatically demystifies it, Any

company can achieve this. Theres nothing non-enablingly complex behind it. Just

some knowledge, expertise, logic, ideas and electronics put together in a system

I can make possible. Today, with the Internet as the ever-accessible ocean of

knowledge and science, anyone can make their own supercomputer, albeit with

levels of scale, he believes.

The future of supercomputing is something that will unfold interestingly. A

supercomputer is a force multiplier and the third pillar of science. It is

becoming all-pervasive to the games we play and the digital content we consume,

says Dr Lagu.

The Top 500 supercomputer list would remain on Eka and CRLs dashboard. But

the list, updated every six months, would be as fickle as the speed with which

faster systems keep popping up. The top-ranker IBMs BlueGene/L System has

achieved a benchmark of 478.2 TFLOPS. Pitted against computing superpowers like

China, Japan, and the US computing, Indias sprint will be something to watch

out for. Competition with the likes of Chinas Dawning, Japan NECs Earth

Simulator, and IBMs Blue Gene stirps would not be easy. The race is going to

turn ruthless but will surely have India on the fast tracks. Next on the cards

for CRL are plans of building a bigger machine that incorporates accelerators in

addition to building domain specific software libraries tuned for Eka

architecture. Besides, innovations on the software stack and productivity tools

for the imminent many-core revolution are on the anvil.

Whats a factor of concern, however, is the critical need of development of

human resources in this space. It was tough to make about a 75-80 member Eka

team and we know how we did it. The future problems wont be hardware or

software, but peopleware, Dr Lagu stresses.

Eka and its likes could buck the trend and drive more Indian cerebra to

experience the inimitable thrill that CRLs team experienced that historical

night. After all, it was indeed an eka moment. Its intriguing at this juncture

then to think of some words that the father of supercomputing, Seymour Cray,

said: Its always easy to do the next step and its always impossible to do two

steps at a time.

Pratima Harigunani



maildqindia@cybermedia.co.in


(Acknowledgements to Dr Sunil Sherlakar, head, R&D, in CRL; Dr Rajendra Lagu,
head, Applications Group, CRL, and N Seetha Ram Krishna, project manager, CRL,

all key members of Eka team and CSI Computer Society of Indias Gireendra

Kasmalkar Pune for enabling the sharing of the Eka story)