MIT Lincoln Laboratory computer achieves 0.5 petabyte calculation

MIT Lincoln Laboratory has demonstrated a 0.5 petabyte calculation on the HPC Challenge STREAM benchmark (www.hpcchallenge.org) using its TX-2500 computer and pMatlabXVM (eXtreme Virtual Memory) software. This is the largest single problem ever run on a computer.

The TX-2500 system consists of 432 connected nodes, 3.4 terabytes of RAM, and 0.78 petabytes (a petabyte equals one million gigabytes) of local disk/virtual memory. With the Lincoln Laboratory pMatlabXVM software, the entire storage can be treated as a single large global array, thereby enabling the processing of datasets far larger than would otherwise be possible.

The TX-2500 and pMatlabXVM are being used to prototype next-generation, scalable, sensor processing architectures. This system can store and process a single 0.5 m resolution, 250-trillion-pixel image of the entire earth.

The STREAM benchmark was run on 380 nodes to validate the system concept and achieved a bandwidth of 25 gigabytes/sec on a 0.5 petabyte problem and 120 gigabytes/sec on a 75 gigabyte problem. In addition, the TX-2500 became the record holder for sorting a terabyte when Bradley Kuszmaul, research scientist at MIT’s Computer Science and Artificial Intelligence Lab (CSAIL), used it to run his TokuSampleSort in the 2008 Sort Benchmark, an annual challenge that recognizes the best sorting systems.

The TX-2500 is a part of Lincoln Laboratory’s LLGrid system, the largest interactive on-demand supercomputing system in the world. LLGrid supports more than 200 users at Lincoln Laboratory, approximately 85% of whom run parallel MATLAB® codes using the Lincoln Laboratory–developed pMatlab library (see sidebar on right). The TX-2500 was provided to Lincoln Laboratory under a Department of Defense High Performance Computing Modernization Program (HPCMP) Distributed High Performance Investment in collaboration with Dell Computer, Inc.

Posted September 2008


MATLAB is a registered trademark of The MathWorks. Reference to commercial products, trade names, trademarks, or manufacturer does not constitute or imply endorsement.

 

top of page