Slaying costs:Linux Beowulf

For decades, computer companies have made lots of money selling big computers. The equation was simple: if you wanted big iron, you had to pay big bucks.

That rule is no longer true. Now you can build your own supercomputer, using commodity components, for a fraction of the cost of a Cray, IBM SP2, or SGI Origin 2000. Using the open source Unix variant Linux and off-the-shelf PCs, a growing number of people are doing just that.

Clustering superpower

The first Linux supercomputer appeared in 1994, when two researchers at NASA's Goddard Space Flight Center in Greenbelt, MD, coupled together 16 DX4 PCs running Linux into a single system. The scientists, Thomas Sterling and Don Becker, called the system Beowulf, after the heroic figure who slew the monster Grendel in the early English epic poem.

Beowulf describes a class of computers made up of a cluster of standard PCs running Linux. Most of the cluster is usually composed of machines dedicated solely to number crunching and often run without keyboards or monitors. A server node feeds data to the rest of the cluster for processing, and serves as an administration system. The cluster is generally hooked together using off-the-shelf ethernet cards, although a variety of higher-speed networking arrangements are used in various systems in an attempt to improve performance.

Cost-cutting and customizing

That first Beowulf cluster at NASA was quickly followed by similar systems at other research labs and universities. To scientists on tight research budgets, Beowulf's low cost is very attractive. The fact that Linux is open source is a big plus for them as well, because they can write their own device drivers for telescopes, magnetic imaging devices, and other specialized scientific instruments.

Access to the source code also lets them optimize Linux for cluster computing by stripping out unnecessary parts of the kernel when they compile it. In a cluster where only the server node has a monitor, for example, cutting the device drivers for monitors out of the kernel used by the rest of the cluster can free up memory for other uses.

Today, www.beowulf.org lists 55 universities and research laboratories with Beowulf clusters. The Los Alamos National Laboratory in Los Alamos, NM, for example, has a Beowulf cluster named Avalon, which is currently ranked number 160 on a list of the world's 500 fastest supercomputers.

Corporate adoption

While many of these groups built their own Beowulf systems, you can now buy ready-to-go Beowulf systems from companies like Paralogic in Bethlehem, PA, Carrera Systems of Corona, CA, and even IBM. That has led to Beowulf clusters springing up in the computer rooms of major corporations: The Boeing Co.'s Applied Research and Technology group, in Seattle, for example, is experimenting with a 16-cpu Beowulf cluster for designing new airplanes. Pharmaceutical company Bristol Myers Squibb has been running a 20-node cluster since February 1999. Proctor & Gamble has a 32-node system running in a research facility near Cincinnati.