Posted on Leave a comment

The new Titan supercomputer is badass but will China’s Tianhe-2 blow past it in 2015? [UPDATE]

image thumb250

Cray XK7 Titan Supercomputer at Oak Ridge National Laboratory

Titan supercomputerChina’s Tianhe-2 supercomputer is expected to arrive in 2015. At 50 petaflops it would be three times faster than the new Cray XK7 Titan supercomputer but let’s face it, vaporware doesn’t count in the supercomputer race and China has only held the title once (in 2010 when its Tianhe-1 briefly held the title of the world’s fastest computer). For now, Titan, at a scorching 18 petaflops or 18,000 trillion calculations per second, is the fastest computer on earth. The official announcement should come out next week.

Titan, developed by supercomputer specialist Cray Inc, is housed at the U.S. Department of Energy’s Oak Ridge National Laboratory in a room the size of a basketball court. It was turned on for the first time on Tuesday, October 29, 2012. The high-performance supercomputer took the title of world’s fastest computer from IBM’s Sequoia (16.32 petaflops) which is housed at Lawrence Livermore National Laboratory (and used exclusively by the National (Nuclear) Security Administration).

Titan is actually an upgrade of Jaguar which uses the same number of nodes and cabinets. Titan replaces Jaguar’s 224,256 12-core processors with 300,000 faster AMD Operon 16-core processors paired with 18,688 Nvida Kepler GPUs, giving the supercomputer a third more processors than the older model. Titan will also have 710 terabytes of memory to keep it ticking over. Each compute node (CPU + GPU) includes 32GB of high-speed DDR3 memory for the CPU and 6GB of even higher-speed GDDR5 for the K20 GPU. Storage is equally impressive – a total of 10 petabytes of storage consisting of 10,000 standard 1TB 7200 RMP 2.5” hard drives. The IO subsystem can push 240GB/s of data through the wires.

Since most of the work processed on Titan is done remotely, there are dozens of 10GbE links inbound to the machine. Titan is also linked to the DoE’s Energy Sciences Network (ESNET) 100Gbps backbone.

The beast fills over 200 custom-built server cabinets (similar to the standard 42RU datacenter racks) and consumes 7-9 megawatts of power at cost of $7 million in power each year. Inside each cabinet are Cray XK7 boards, each of which has four AMD G34 sockets and four PCIe slots.

Titan runs Cray Linux which is based on SUSE 11. The Cray variant has been hardened and modified for efficient operation on a large scale. For instance, some CPU cores are dedicated solely to OS tasks. Jobs are batch scheduled using Moab and Torque.

The United States expects to upgrade Titan in 2016 to 200 petaflops or 10-times the speed of Titan.

Table comparing Jaguar supercomputer to upgraded Titan

Oh, and China’s Tianhe-2 relies primarily on U.S. manufactured chips and software so we’ll always have at least some bragging rights…

11/12/12 UPDATE: Sequoia, which uses processors from IBM, became the top computer in June 2012 with a performance of 16.32 petaflops a second. Titan beat that showing today, sending Sequoia to second place on the list, with a result of 17.59 petaflops per second (not quite as fast as the 20 petaflops that were expected).