The Anatomy of a Supercomputer called Bruce.
DownUnder GeoSolutions (DUG) recently revealed the power of their Perth-based supercomputer and its role in processing one of the oil industry’s largest ever multi-client seismic projects (Bruce Almighty Adds Power to the Process: PNR Online April 22, 2015.).
Since then, there has been great interest in this computer, touted to be the largest supercomputer in the Southern Hemisphere and five times larger than the better-known installation at the Pawsey Centre and three times larger than the proposed CSIRO BOM system to be built in 2016.
At 5 PFlops capacity (that's 5,000,000,000,000,000 floating point operations per second) DUG’s SGI based supercomputer, affectionately known as 'Bruce', is not only the largest super computer in Australia, but possibly the largest privately owned super computer in the world.
Bruce possibly processes seismic data faster than any other machine on earth and is currently processing one of the single largest-ever acquired seismic surveys (24,000 km2 across the Roebuck Basin), along with many other significant projects.
Incredibly, DUG has managed to compress 5PFlops of compute into a tiny 110m2 of floor space, making Bruce one of the most compute-dense systems on earth. Unsurprisingly, Bruce generates considerable heat, pumping out over 600 kW through his super-efficient water chillers, which use magnetic-levitated compressors spinning at 48,000 rpm.
The quest for a more reliable, cost effective and energy efficient cooling solution has led DUG to introduce revolutionary oil cooling technology. Much of Bruce is submerged in a cooled mineral oil bath, which makes him one of the greenest machines around. All the cool technology enables Bruce to be 40% bigger within the same power envelope and offers far greater reliability under extreme temperature conditions. Bruce can continue to process data, uninterrupted, even when the mercury tips 43 degrees centrigrade!
With an extreme high speed, low latency, full bisection bandwidth 10 Gb network, each part of Bruce can communicate with the others without contention. This, combined with a Lustre-based cluster file system running on Netapp hardware, enables Bruce to process data from disk at a massive 35GB/s (35,000,000,000 bytes per second).
Bruce is no ordinary machine. His brain consists of hundreds of thousands of processing units, packed up in thousands of Intel Xeon Phi Co-processors: one of the most innovative, high-performance processors on the market. Bruce uses more Xeon Phis than any other commercial super computer on earth and runs highly-optimised software to extract every last flop. In fact, through DUG’s recoding of their software to optimise the power of the Phi, they are able to take advantage of 95% of the Xeon Phi’s theoretical power.
Not content with just a good brain, Bruce has excellent memory too. Not a single bit goes unnoticed. His ECC memory ensures exactly the same result every time - unlike commodity GPUs for example, which do not use ECC RAM and therefore sacrifice accuracy for speed and price.
Bruce doesn't eat quiche! He is big, brawny and brainy. And he’s helping DUG turn around large projects at speeds like never before. Bruce is a little concerned about his younger brother 'Bubba' in DUG’s Houston office. By year-end it looks like he will be bigger than Bruce!
The rows of oil tanks in which the computers are suspended.