Originally Published: Monday, 12 July 1999 Author: Robert J. Berger
Published to: featured_articles/Featured Articles Page: 1/1 - [Printable]

The Linux Probability Wave

Whether or not Y2K has any meaning other than a temporal odometer reaching an arbitrary milestone, it is clear that we are in a time of great endings, transitions and new beginnings. Though the seeds were planted in the 18th and 19th centuries, the 20th Century has been a period unlike any known in recorded history....

   Page 1 of 1  

Whether or not Y2K has any meaning other than a temporal odometer reaching an arbitrary milestone, it is clear that we are in a time of great endings, transitions and new beginnings. Though the seeds were planted in the 18th and 19th centuries, the 20th Century has been a period unlike any known in recorded history.

Exponential Growth and Change leads to Non-Linear Transitions

The core trends have been the interaction of technology development and human population growth. Worldwide population went from 2 billion in 1930, to 3B in 1960, 4B in 1975, 5B in 1987 and we are expected to reach 6 billion by Y2K. When plotted out the population growth curve has the same "hockey stick" growth that investors like to see in Internet stock prices.

During the same time, the growth in technology has had a similar curve. Not only have we had huge breakthroughs in new technologies but the time from the invention or discovery of a new technology to when it is available to a mass market as finished goods or services has gone from decades to months.

Undercurrent of Changes in Fundamental Scientific World Views

Both in the case of human population and technology, the bulk of the growth has been in only the last 3 to 5 decades, pretty much since the explosion of the first atomic bombs and the introduction of television. Much of the technological growth spurt has stemmed from a fundamental shift in the underlying scientific paradigms of physics. The primary transition has been from the Newtonian "Cause and Effect" world view to the Einstein / Bohr Relativistic and Quantum Mechanics (QM) perspectives. The Industrial Revolution of mechanization and specialization followed the spread of Newtonian teachings. Material sciences, nuclear energy, molecular biology, electronics, and computers propagated from the new perspectives offered by Quantum Mechanics.

The Legacy Mechanisitic Newtonian Model

The Newtonian model has a mechanical view of the universe consisting of discrete pieces (planets, bodies, particles, etc.) whose interaction is fully determined by mechanical push-pull actions and forces. This mechanistic view that separates the observer from what they are observing has fostered a dualistic world culture. This perspective is one that focuses on differences and separation. Us / them, mine / yours, friend / enemy and so on. The observer and what is observed is distinct and separate.

World View: Next Generation

The Quantum Mechanics view is quite different. It says that the observer and the observed are inexorably linked. That it is the intent and action of observing that effects and even determines what is observed. The classic example is of the photon behaving as a particle with a measurable location independent of any medium at one moment and a wave with a non-specific location, existing as a disturbance of a medium at another moment with the only difference being the intent of the observer. According to the famous Quantum Mechanic Werner Heisenberg: "what we observe in our experiments is not nature itself but nature exposed to our methods of questioning nature. " In short, the photon does not exist until we observe it, until then, it is just a probability wave!

These scientific paradigm shifts have not just effected the seemingly esoteric discussions between scientists and engineers. The combination of the new perspectives and the technologies that this new world view fosters also impacts the world views and daily activities of just about everyone. There is a propagation delay from the time a major scientific shift is generally accepted in the scientific community until it becomes part of the "common sense world view" of the general population. Of course with everything being sped up, the Quantum Mechanics world view will penetrate the collective consciousness much faster than the Newtonian one did 300 years ago.

At this point, though, we are in transition where most people are still thinking and acting with Newtonian minds while at the same time living in a world whose technology is being driven by Quantum Mechanics, leaving a lot of people confused and fearful.

By now, you are probably wondering "What's the big deal about Newtonian vs. Quantum Mechanics, and what's all this have to do with Linux anyway!?" Well, besides the physicists, the population that is most exposed and comfortable with the new mental model based on Quantum Mechanics are the people who are developing and working with the technologies that have blossomed from the theories and mathematical models of Quantum Mechanics. The oldest and most successful of these is semiconductor-based electronics that has given us the integrated circuit, with its cheaper, faster, better, more complex and smaller CPUs, DSPs, switches and memories. These have given us computers and digital communications such as PCs and networks. On top of all this we have an even more abstract layer of protocol/API standards and software.

Changing Scientific Paradigms Lead to Societal Upheavals

It is my contention that just as Quantum Mechanics demands radical changes in the mental models of physicists and mathematicians, technology based on Quantum Mechanics leads to radical shifts in the thoughts, beliefs and actions of individuals and societies that embrace such technology.

For example, Newtonian technologies are things like petroleum, planes, trains and automobiles. These all are macroscopic and industrial. They consume huge resources and generally get more expensive, change slower, and haven't improved much in decades. An educated person from the 19th century could understand them with just a little bit of explanation, Trains ran better back at the turn of the century, automotive average ground speeds have not significantly improved since the 1930s and air travel has not had any real speed improvement since the introduction of the first jet airliners in the late 50's. Though there is progress, the basic societal thought model is one of bureaucracy, large centralized organizations / governments, limitation and scarcity.

On the other hand, it has become clear to just about everyone how rapid and continuous the advancement has been in electronics and computers in the last few decades. We even have Moore's Law to canonize this phenomenon: Integrated Circuits capacity doubles every 18 - 24 months. This has held true since Gordon Moore (Intel co-founder) stated it in 1965. The raw semiconductor technology is driving the computer, communications and any other industry that can harness these technologies to products and services that are getting cheaper, faster, more complex and smaller. This creates a new pattern in society, one that will require shifts not just in the technology industry but in all facets of business, economics, governments and religion. Technologies based on the QM technologies lead to a world where abundance is the norm and scarcity is the exception.

Technologies of Transformation and Abundance

QM tech based semiconductors with its doubling of capabilities every 2 years, offers us the abundance of digital compute and signal processing power. This wellspring of capability is now reaching levels of capacity that were undreamed of only a short time ago (remember: "You'll never need more than 640kbytes of RAM"?). The end of Moore's law is not in sight and we are now approaching processors with 50M transistors and GigaMIPS of compute power. We are faced with the pleasant problem of what can one do with computing capacity as it reaches astronomical complexity in smaller and smaller packages? Its clear that there is more to do than running bloated Windows applications!

An example of the synergy that can come from adding processing power based on semiconductors to do things that were previously impossible is that of modern jet fighter aircraft. A modern jet fighter has requires very small wings for low drag and extreme maneuverability. Traditional aerodynamics does not allow for such a plane to be stable at all its speeds. By adding the processing power of distributed microprocessors to manage the real-time non-linear corrections to the positions of the control surfaces of the wings the aircraft designers were able to go beyond conventional aerodynamic limitations and create a device capable of feats that would have been thought impossible a short time ago.

This kind of synergistic capabilities of doing more with less and even doing things that were once impossible with less, doesn't just give us incrementally better technology. It also forces us to come up with new ways of thinking and acting both to develop and to utilize this new power.

Trends and changes in technology impact on society and business

A major trend has been the dissolution of centralized control and the empowerment of individuals and self-organizing small groups. This can be seen on the world stage as the destruction of the Berlin Wall, the disappearance of the Soviet Union, the trivialization of the US Congress and Presidency and the eventual dissolution of the Microsoft Monopoly. On the industrial front we see fervent new production of wealth (both real and "on paper"). On the surface, it looks like a return to gigantism with all the mergers and acquisitions. But the real growth in new jobs and true wealth production is happening in the small to medium businesses and other entities.

This does finally lead us back to Open Standards, Linux and the Open Source movement which I contend is a harbinger of a whole new way of human organization and behavior that expects and creates a world of abundance.

The Wonderful World of Software

The world of software is almost as strange as the world of Quantum Mechanics that has enabled software's emergence. Software development is an exercise in human thought and creativity. The process of creating software is the embodiment of the programmers imagination and logic into symbols that are animated by the layers of virtual machines that sit on top of the somewhat physical logic circuits manifested in the quantum effects from the junctions formed by the etchings in silicon. People who are into software tend to be "different" than the "average" person. This difference is not necessarily good or bad, but it does seem to be an ability to deal with this new kind of world. The best software developers (the top 10 percentile) tend to be significantly more productive than the average developer. Many developers and people who use computers find themselves immersed in this world. It is one where they participate in its creation and evolution. In many cases, particularly where there is significant innovation, they get to set the rules, break the rules and make new rules.

In many cases, when people who spend much time immersed in the world of software step back into the so called "real" world, there is significant dissonance between the experience in the two worlds. Many of the expected behaviors, thought modes and societal structures begin to seem absurd and severely limiting for no reason other than tradition or outmoded beliefs and fears based on expectations of scarcity.

There have been many responses to this dissonant experience. Most people in this position just go back into their computing world and keep doing what they've been doing because it seemed that there was nothing they could do to change "reality" themselves or they did not want to expose themselves to ridicule.

Taking a Stand for Freedom

Fortunately, there have been a few who have had a run in with "reality" and decided to do something about it. In the late 1970's, Richard Stallman was one of the first to rise up and proclaim the need for new behavior and the guarantee for the ability of programmers to be in control of their environment as well as to have the ability to extend and improve the shared software environments as well. Stallman began to articulate the fact that Software is a different resource than the physical resources of the past. It is something that you can share yet not reduce your own possession. In fact just the opposite occurs, when you share your software with others in source form, it allows the other programmers to at least help you find bugs and may even let you increase your software wealth when another programmer passes you back an improved version of the software or another piece of software that builds on top of your original. This is indeed a source of ever growing abundance with inherent positive feedback that accelerates the growth of abundance as more sharing and less hoarding goes on!

As someone who started the open systems idea (by the way the most criticized part of the SUN business plan) it seems like open source is the next logical evolution of offering each user or set of users just what they need and not everything everybody might want.

More importantly, it forces a development methodology that is modular not monolithic. With increasing complexity and we cant build spaghetti systems anymore where everything depends and is impacted by everything else. By virtue of the methodology open source will force more complemented development methodology - a methodology change that is required in all of our engineering. Incidently the methodology change also helps one of the key requirements of the modern engineering. Change isn't a event anymore (version 2,3...) - it is a process. Adaptability and evolution are far more important goals for systems as opposed to the old goal of optimization (I could give you a whole dissertation on this). Open source and Linux have fortunately fallen into this new paradigm and are benefitting from rapid evolution, incremental changes, modularity which also implies customization and personalization.

Vinod Khosla, founding CEO of Sun Microsystems, Partner Kleiner Perkins Caufield and Byers

At the same time that Richard Stallman was codifying the philosophy of Free Software ("Free software" is a matter of liberty, not price. To understand the concept, you should think of "free speech", not "free beer."- Free Software Foundation http://www.fsf.org/philosophy/free-sw.html), which was made concrete in the "CopyLeft" or GNU General Public Licence (GPL) and manifested in a project to create a Free operating system / software environment in the form of GNU Software, another important example of openness based on assumption of abundance was beginning to emerge into public. This was the ARPANET which in the early 1980's moved from being a Military R&D project to an Academic network of researchers funded by the National Science Foundation and called the Internet.

Here is another case where a mostly post nuclear / television generation of developers come up with radically new concepts that are built on top of semiconductor and digital logic technology that lead to inherent abundance. In this case the developers dreamed of protocols that fostered robust and unlimited communications between one to many, many to one, one to one and many to many nodes. These protocols were described independent of the implementation and were made available to anyone. Diverse implementations were encouraged and human processes were developed to encourage interoperability and sharing at the hardware, software and human levels. Open, consensus and result driven processes were organically evolved into the Internet Engineering Task Force which had no centralized authority, or legal foundation became the incubator within which the standards were able to be grown. Interop became the forum where the interoperability between implementations and devices was tested, debugged and proven.

At each stage the "common wisdom" of the day said that it would be impossible to use the model of "rough consensus and running code" to make a system that would be robust and scalable. When AT&T was approached by ARPA to implement the original network, the AT&T engineers told ARPA that they had already proven that packet switching will not work. The International Telecommunications Union (made up of the world's telephone monopolies) mocked the "unprofessionalism" of the IETF process and spent decades creating the OSI family of protocols which in the end was abandoned by most of the world in favor of the "unprofessional" but robust, scalable and ubiquitous Internet Protocols.

The Internet is another case of a technology built on top of semiconductor and thus QM technology with a human layer that is based on sharing, cooperation and an assumption of abundance that creates more abundance and wealth.

"To me it [Open Source] is no different than ARPA funding developments, like TCP/IP, and granting them to the world at large. In the case of Linux the money/resources came from personal donations, instead of being washed by the government. People make money when they do something others want badly enough to buy it instead of something else. Linux is not a charity case. It produces useful stuff or it vanishes.

Microsoft is a mechanism for creating, distributing and servicing software. It has been extremely successful. One could almost say it got started with free software. Didn't they pay something like 50K for Qdos? And they have done well for themselves and for the world. IBM sure as hell would not have made computing as pervasive as MS has done. And for the same reasons people were mad at IBM people are mad at MS for squelching innovation by virtue of their real monopoly.

So, something like Linux has a chance. But will it be squandered like Unix(tm) was, due to loads of squabbling? Remains to be seen.

Open interfaces promote diversity. Money is a great example of that. But money is rather sterile. Linux is more fecund."

- Dan Lynch, founder of Interop, CyberCash, private investor / board member in several key Internet companies

Interestingly enough, it was the intersection of the Internet and the GNU Free Software concepts that gave rise to Linux in the early 90's. Linus Torvalds, then a computer science student in Finland, got sick of not having an OS that he could explore, learn from and freely extend. So he put out some messages onto the Internet through the comp.os.minix (Minix was an early not quite free, not quite unix clone) and asked for input and help on creating a free OS:

From: torvalds@klaava.Helsinki.FI (Linus Benedict Torvalds)
Newsgroups: comp.os.minix
Subject: What would you like to see most in minix?
Summary: small poll for my new operating system
Message-ID: <1991Aug25.205708.9541@klaava.Helsinki.FI>
Date: 25 Aug 91 20:57:08 GMT
Organization: University of Helsinki

Hello everybody out there using minix -

I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu) for 386(486) AT clones.This has been brewing since april, and is starting to get ready.I'd like any feedback on things people like/dislike in minix, as my OS resembles it somewhat (same physical layout of the file-system (due to practical reasons) among other things).

I've currently ported bash(1.08) and gcc(1.40), and things seem to work.This implies that I'll get something practical within a few months, and I'd like to know what features most people would want.Any suggestions are welcome, but I won't promise I'll implement them :-)

Linus (torvalds@kruuna.helsinki.fi)

PS.Yes - it's free of any minix code, and it has a multi-threaded fs.It is NOT protable (uses 386 task switching etc), and it probably never will support anything other than AT-harddisks, as that's all I have :-(.

This began the group collaboration via the Internet that grew into Linux. Linus facilitated volunteer software developers from around the world to work on a GPL'd Unix kernel clone. Others integrated the work being done with the Free Software Foundation's GNU software that neatly surrounded and supported the Linux Kernel. The fact that all of this software was developed under the GPL was key to keeping the work done by all these volunteers in the public domain and prevented hoarding. The Internet which has a complementary philosophy of open standards acted as a medium to foster a distributed development team and environment allowing thousands of developers and 100's of thousands if not millions of testers and users to cooperate in ways never before seen.

"The open source model of "development at a distance" in a compelling solution to complexity management in software creation. It forces modularity, resulting in code that is generally more elegant, more secure, and more reliable that alternative techniques of software development."

- Hal Varian Dean of the School of Information Management and Systems at the University of California at Berkeley and co-author, with Carl Shapiro, of "Information Rules" from Harvard Business School Press

Not surprisingly, GNU, Linux and other Open Source Software has given back to the Internet in the form of being the implementations for many of the servers and services that make the Internet run. From the most popular web server on the Internet, the Apache Web Server running on Linux and other Unix systems, to BIND which runs much of the DNS system, to Sendmail which moves and delivers the bulk of the Internet mail, Open Source Free Software and the Internet have through their symbiotic and synergistic relationships helped spread more abundance.

After several years being tempered in this new forge, GNU/Linux emerged as an extremely robust, portable, scalable platform and scaffolding for the next wave of Open Source software and services to be built on top of. In the last year, GNU/Linux and Open Source in general has emerged from the schools, ISPs and developer's basements to the commercial and world at large. Besides its already dominant position as a platform with Apache for the majority of ISP and business web servers, Linux started to be injected into corporate enterprise networks by system administrators who were developing or using Linux at home. They quickly saw its flexibility and robustness and were sick and tired of having to come in and reboot the Windows NT servers at all times of the day and night. With the development of Samba Linux was a better file and print server for Windows (and Apple and NFS) than Windows NT.

"Linux snuck in the back-door, and corporate IT often doesn't know it's there," says Dan Kusnetzky, an analyst at International Data Corporation. At first management doesn't know that anything has changed except that their network is more reliable with higher performance. Eventually the system administrators get brave enough to show management along with the dramatic cost savings that a Linux servers delivers.

Kusnetzky amplifies: "Compare the costs of a file and print server for a 25-person group using Linux or NT: NT Server has a street price of $809, including a license for 5 clients. Two more 10-client packs, at $1,129 apiece, brings the total to $3,067.

A copy of Linux from Red Hat--one of several companies that offer Linux support--costs $49.95, and the cost doesn't go up if clients have to use the server. Or, for that matter, if you want to install the same copy of Linux on another server, or five other servers, or 50 other servers. And Linux lets you do the job with hardware that Microsoft and Intel have declared obsolete."

Open Source has another inherent positive feedback loop. Open Source becomes the natural playground for kids and students that are into computers and want to learn how to program them. Linus' original desire to have an OS that he can study, learn from and improve is now a rich living library, tutor and mentor for kids, students and adults who want to learn and experiment. This is breading a large and ever growing population of developers and technically savy linux users.

"Linux likely will have a surge of support when the current crop of Linux-savvy students starts looking for jobs. These students have been trained in their computer science classes to play with Linux's source code, hacking the kernel and trying out new software ideas.

A similar phenomenon happened with Unix and, later, Windows, Kusnetzky said. "It seems to point to a successful, rosy future."

Linux is pushing Windows out of the nest where new programs are born and bred. and so I will close with a look at what comes hither for Microsoft's future as the winds of change and philosophies of abundance begin to dissolve Microsoft into irrelevance.

The Beginning of the End of Microsoft as we know it

With the US Department of Justice Anti-trust court case underway, Microsoft likes to paint Linux as a competitor, particularly in the more public forums such as at the trial itself as shown in this excerpt of the testimony of Paul Maritz, Sr. VP of Microsoft:

"Linux is rapidly emerging as a major competitor to Windows. Indeed, "the number of developers working on improving Linux vastly exceeds the number of Microsoft developers working on Windows NT"

But in the more private trade journals the story gets twisted back to the usual Microsoft PR message:

"For information technology personnel, the up-front cost of the operating system is a relatively minor component of the total cost of ownership of a system," a Microsoft spokesman said.

"Microsoft sees Linux as a competitor, and we see that as good for the market," the spokesman said. But Linux competes more with other Unix systems, the spokesman said. "It's unlikely someone would move from NT to Linux. It's more likely they'd move from a Unexposed system to a Linux-based system."

Of late though, it seems that the Microsoft's illusion of invulnerability that it has so carefully built up since it bluffed its way into the PC Operating System arena with its trick purchase of the CP/M clone QDos has been pierced and the its domineering presence is starting to fade.

"Software, both proprietary (Microsoft Word) and non-proprietary (Linux), often exhibits network effects: the larger the number of users, the more valuable it becomes to any one user. The dark side of network effects is the tendency towards "winner take all", and the resulting lack of variety. Open source lets you have your cake and eat it too, offering a standardized product that, at the same time, is highly customizable"

- Hal Varian Dean of the School of Information Management and Systems at the University of California at Berkeley and co-author, with Carl Shapiro, of "Information Rules" from Harvard Business School Press

The fundamental difference between Open Source and Proprietary systems such as Microsoft products is that the former creates a positive feedback loop that encourages fun, innovation and abundance whereas the later creates boredom, lack of diversity, fear and scarcity. In the long run, people will want to choose the former over the later. Developers are the first to see the positive effects, particularly developers whose initial interest is the love of programming. These are also the people who tend to develop the breakthroughs and "killer apps".

Hal Varian observes in his book with Carl Shapiro, "Information Rules":

"We present the following equation: your reward = total value added to industry x your share of industry value. When open source adds a lot of value to an industry by increasing reliability and reducing development costs, even a small market share can be worth a lot."

This shows that its possible to have a big impact and that the reward can be bigger than you might expect. In some cases the reward is in praise and recognition which leads to access to even more interesting opportunities. In other more conventional cases, it may lead to financial reward. But as we move into a world of abundance, financial reward becomes less and less meaningful compared to personal satisfaction and growth. The Open Source movement is well suited for delivering this kind of experience while the propriatary, fear driven organizations tend to do the opposite and burn people out.

Now that the DOJ is exposing the predatory practices of Microsoft for all to see while at the same time giving Microsoft's competitors and customers some breathing space to act without fearing instant retaliation, it seems that the larger world is turning its back on Microsoft and embracing alternatives at a rapidly growing rate.

"Linux Threatens Commercial Operating Systems and Provides Model for Freeware Industry. Previously confined to the fringe of the computer industry, Linux is breaking out, with a huge potential impact." says Tom Kucharvy an analyst for Summit Strategies. "Even if it does not capture the operating system market, Linux serves as a model for open source software and is thereby laying the seeds for a revolution in the software industry."

Hardware system vendors who only a few months ago would not even acknowledge an alternative to Windows and Windows NT are now trying to outdo each other with their Linux announcements.

"Demand for applications and support for Linux is growing rapidly. Customers appreciate the ease and reliability of purchasing integrated Internet-based solutions through the HP Covision program, and through this alliance with Red Hat, they will now have the flexibility to select solutions built on the Linux operating-system platform."

- Greg Mihran, head of Internet Business Development for HP's Personal Systems Group.

"Our customers have told us they want standards-based systems. This means they want Windows NT, UNIX and Linux," "We will lead the low-end server market with full support for Linux from a leading UNIX systems vendor. SGI will provide the same high level of quality and support for Linux on its low-end IA-32 servers that it currently provides for its industry leading high-end MIPS processor-based systems. This will include the full support and backing of our extremely technical field teams and our technically sophisticated internal support organization."

- John R. "Beau" Vrolyk senior vice president, Computer Systems Business Unit, Silicon Graphics.

The MIT Sloan School of Business rated Linux important enough in 1999 to include it in its Digital Time Capsule:

The Sloan School Digital Time Capsule symbolizes the leadership role of MIT and the Sloan School in technology and business today. It will contain digitized artifacts and memorabilia capturing The Internet and Business in 1999, such as:

...the Euro, Amazon.com, Lotus Notes with Domino, Linux, the Asian economic slowdown, online auctions, skyrocketing IPOs, Microsoft trial, bartering online, Monica Lewinsky, online resumes, personal homepages, portals, Dilbert, RealPlayer, Lester Thurow, MP3, Impeachment, South Park, Slate, Amazon.com...

The digital time capsule will also contain Predictions for the Future of the Internet by celebrities and by everyday people. The capsule will be opened in the year 2004.

All I need to know I learnt from Quantum Mechanics and Open Software

In closing, it is critical to state the importance of the Open Source community to not repeat the mistake of Marc Andreesen and Netscape where they directly confronted Microsoft and lost their competitive edge by stepping directly into the Microsoft owned and operated arena. Netscape got its original giant leap ahead by creating a new game in an arena that was not being visited much by Microsoft. They were well on their way to creating a defendable new marketplace. By directly challenging Microsoft, Netscape stopped focusing on doing cool new things and instead focused on competing with Microsoft and then with not getting steamrolled by Microsoft.

It is also important that we learn from the mistakes of the Unix Wars of the 1980s where petty differences, arrogance, ignorance and greed stopped the first attempt at open systems and paved the way for mediocratic monopoly to fill the void. Stop and think before you flame a KDE person if your into GNOME, or a BSDI guy if your into Linux or even a Microsoft addict if you are into open source. Religious wars always lead to unnecessary conflict.

The positive lesson for the Open Source community is to stay focused on creating and extending open source software that brings joy and satisfaction for ourselves, our friends and our customers. It is important that we continue to observe and learn from what makes GNU/Linux and Open Source a movement, at why the Internet continues to defy logic in growth, capabilities, reach and valuations and how the new QM based technologies snowball and expand our horizons and possibilities. That fear / scarcity is a negative vicious circle which leads to more scarcity / fear, whereas sharing fun, joy, wonder, and abundance in how you do your work and relate to other people will create more abundance to share with more people.

Robert J. Berger is a consultant for Internet Bandwidth Development. A version of this article first appeared as the cover article for the premier issue of Linux Magazine.

   Page 1 of 1