Big Iron

Beowulf clusters, running up to hundreds of nodes of cheap, commodity computers running the free Linux operating system, are quickly becoming a mainstay of science and business projects that need a lot of computing power, and fast. The "Big Iron" market, a realm that even denied the formidable Unix variants, is making use of the unique opportunities offered by Linux -- a freely available, freely modifiable and reasonably flexible solution -- to create solutions that fit niches that have previously been the sole domain of multi-million dollar projects and expensive research projects. Beowulf clusters have considerable potential in several areas; the ability to produce supercomputing power using cheap, freely available hardware and open, free software is something that will enable many new applications and opportunities for computing, science, and business.

IBM, once the giant behemoth of the software industry, is attempting to take advantage of Linux in the area of cheap supercomputing in several ways. Its research teams are actively developing Beowulf software and implementing it into solutions that will soon be commercially available. The most recent of these projects, a 1.5 million US dollar project designed to produce a cluster capable of making the number 24 spot in the Top 500 list.

Until very recently, Linux was something that simply did not exist in the clustered computing arena. The market was dominated by a relatively few proprietary solutions and was limited to a few institutions that could afford the implementations. Linux has the potential to change all that, to become part of a platform that brings cheap computational power to a far wider array of people and institutions. This is a very critical development, one that could turn out to have enormous implications for research in a wide array of fields. Projects that currently are languishing due to a lack of computing time; fields as diverse as computational linguistics, sociology, and electronic commerce are all languishing behind the bottleneck of cheap data processing power. The introduction of Linux into Big Iron computing could change all that dramatically, for three reasons:

1. Commodity hardware: Beowulf clusters run quite happily off of computing hardware available readily to the regular consumer, thus bringing in economies of scale that are undreamt of in the previously arcane world of mainframes and clustering.
2. Commodity software: Freely modifiable and redistributable software removes a huge bottleneck. A research team can modify existing clustering software to their heart's content, and contribute those changes to make their tools useful to a yet wider group of people. Free software makes this synergy possible. IBM's research labs, for instance, have historically worked very close with Microsoft and other operating systems providers to fix bugs and contribute to the development of those OSs. Similar development teams now exist for Linux software, and are able, due to the open nature of the free software paradigm, contribute to a far higher degree than is possible with other OS. IBM gets a far higher value for its research dollar in its Linux efforts than it ever could with its Windows efforts.
3. Linux provides a common skill base with a ubiquitous computing platform that exists in several different areas. The skills that a person learns on his or her desktop computer, or in a network environment, transfer directly over to a Beowulf cluster environment, necessitating a far lower learning curve than previously required with large-scale commercial clustering systems. When the skills to run large data-processing machines are ubiquitous, it becomes far, far more expedient for projects to use those skills in a useful manner.

It's safe to predict that given the huge demand for machines capable of handling heavy computational loads, and the readily available solutions available with Linux and the Beowulf technology, that there will be an explosion in the market for such large-scale machines. IBM, for instance, is attempting to establish a name early on in this marketplace, and will be selling Beowulf implementations quite soon. They will very likely make a metaphorical 'killing' in this regard -- all with freely available technology.

While the Beowulf software will always be freely available, and the technology behind it is readily understood and implemented, companies will make money not from the software itself, but from the in-house expertise they have developed in implementing and supporting these systems.

Now, the whole Beowulf thing is an excellent case study of exactly what people have got wrong in viewing Linux' potential for commercial success. It isn't Linux that will be making money, or supporting Linux systems directly. While some money will be made in that arena, it will be minimal. The real money lies in the applications that free software, that Linux, will be able to enable. In economic terms, free software's secondary economic effects are what will be the most important in the long term. In enabling new applications and commoditising technology, new opportunities, new fields of endeavour will constantly be uncovered. These benefits will always be larger than the very limited, trivial attempts at making money directly off of free software.

In the end, companies that try to make money directly off of Linux are doomed to fail. It's quite simply a losing proposition. In the end, free software will and is changing everything it touches, expanding and rejuvinating science, commerce, research, engineering.. but it will always be a difficult proposition to make money off of it directly.

Linux development simply will not fail because of that, however. Research teams and individuals sponsored and supplied by their own efforts, working toward specific, selfish goals, will drive forward its development as an OS and an environment. VA, in making hardware solutions involving Linux, has the right idea. IBM, in developing Beowulf implementations, has the right idea. Academic institutions, in adopting Linux, have the right idea.

There's a bright future for Linux and the people who use it. Who's to say what area of our lives will be touched by this free OS next? Who knows what area of computing will be enabled? So much work will be saved, so much duplication of effort eliminated. But that effort won't be simply discarded, but put into advancing the state of the art.

Rob Bos (rbos@linux.com) is a student at Simon Fraser University and has midterms soon. Yay.