Originally Published: Monday, 27 August 2001 Author: The Staff of Linux.com
Published to: develop_articles/Development Articles Page: 1/1 - [Std View]

Interview with Eric Allman, CTO of Sendmail Inc.

Today, the first day of Linux World, San Francisco, Sendmail Inc. one of the world's most successful open source companies and IBM (big blue) announced a joint partnership to deliver powerful software for mainframes on Linux at a scale the world has perhaps not seen before. This terrific validation of the open source development method is cause for celebration. Linux.com caught up with Eric Allman, CTO of Sendmail, Inc. (who wrote the original sendmail back in 1981, arguably the most important piece of network software ever written) to ask the guy some questions.

As Sendmail, Inc.'s chief technology officer and co-founder, Eric Allman leads the company's technology strategy and developments. Allman authored sendmail, the world's first Internet Mail program, in 1981 while at the University of California at Berkeley. He continues to spearhead sendmail.org, the global team of volunteers that maintain and support the sendmail Open Source platform.

Today Sendmail Inc. and IBM announced a partnership to develop a suite of products for the IBM eServer z900 mainframe, including Sendmail Switch, Sendmail Advanced Message Server and Sendmail Mobile Message Server on Linux for mainframes. These solutions provide ISPs, telcos and other large network organizations the opportunity to provision high performance software that can handle more load (up to two million users on a single machine), more reliably (99.999% uptime) and at a lower total cost than ever before. It's an agreement that represents a terrific validation of everything the Open Source movement holds to be true.

Linux.com caught Eric Allman to ask him some questions intense about morality and puddings, among other things.

Linux.com: Why did you write the original sendmail, back in 1981?

Eric Allman: Berkeley had connections to the ARPAnet and UUCP, with the ARPAnet being on a PDP-11 owned by the INGRES (Interactive Graphics and Retrieval System) database management project, which I worked on, and UUCP being on a departmental VAX machine. There was also a third home-grown network called BerkNET, written by Eric Schmidt (now CEO of Google).

But none of these networks talked to any of the others. In particular, to send or receive ARPAnet mail, you had to have an account on the INGRES machine, which was quite overloaded as it was. We actually did bring up accounts for all the faculty and some of the graduate students, but the situation was untenable for the long run.

At some point I realized that these networks were (primarily) software, so all I had to do was write some "glue software" to make them talk to one another. I created a predecessor to sendmail called delivermail. Use quickly exploded at Berkeley, and after not too long many other sites started using it as well.

When Berkeley got the ARPA contract to do VAX software for the new (as yet unbuilt) "Internet" thing, designed to replace the ARPAnet, I added SMTP client and server support to that program and redubbed it as sendmail.

Linux.com: Can you tell the Linux.com readers who don't know very briefly what sendmail is?

Eric Allman: Sendmail is a Message Transfer Agent (MTA) - the software that knows how to route mail from one place to another. This is in contrast to the Mail User Agent, which is what people see on their screen, and the Message Store, which holds your inbox until you get a chance to read your mail.

Linux.com: When did you form Sendmail, Inc.?

Eric Allman: We opened our doors in March of 1998.

Linux.com: That was before many people were trying to combine "Open Source" - software development by individuals and the academic community with - commercial businesses. What gave you the idea to make the leap?

Eric Allman: Two reasons really. First, I personally viewed that as a requirement. I may have written sendmail to solve a local problem at Berkeley, but I continued working on it because I saw that e-mail was going to change the world - or rather, continue to change it - and I felt that change was going to be fundamentally positive, and that sendmail was a big part of that change. However, I did need to make money because sendmail had become such a success that a small crew of volunteers couldn't keep up with e-mail support and do development at the same time. But getting rich wasn't my primary goal, and ultimately I felt that the best way to contribute to the Internet was to keep sendmail Open Source.

Greg Olson, my business partner and the "business mind" behind Sendmail Inc. did some market research and ultimately agreed with me. He was adamant that Sendmail, Inc. needed to be a "real" company right from the start, and a big part of being a real company is making a real profit - something I very much agreed with. Being Open Source gave us market share, name recognition, and street creds: intangible perhaps, but very valuable. Frankly, I think that closing up the sendmail source code would have been suicide.

Linux.com: Sendmail, Inc. is often cited as a model commercial open source company. To what do you attribute your success in that area?

Eric Allman: As noted above, we started up the company with a goal of building a real company, unlike so many Internet Bubble startups that seemed to take the attitude of "let's get mind share, and we'll figure out a way to make money later." But maintaining true to the Open Source users was also essential. And it helped that Greg and I came from two very different perspectives. We had many "frank and comradely" discussions, and we continue to do so. Far from weakening things, these different perspectives made us stronger. Another important point is that Greg and I agreed early on that we needed experienced management. Too many bubble companies (which isn't the same thing as open source companies at all, I hasten to add) thought that dropping out of school was the best qualification to be CEO. We looked for people who had some background, and I think that was a good decision. However, I will admit that it took some time to educate some of these people about open source, which is often considered to be antithetical to making money - which of course it is not.

Ultimately, we created a "hybrid" company, rather than a totally "open source" company - that is, we have open source software in the form of the sendmail MTA, which remains the core of our products. But we have embellished that with monitoring and management tools, a message store, POP and IMAP servers, web and mobile access, and so forth, which gives us a much broader product line.

By the way, Greg and I are still friends - a somewhat unusual situation for co-founders, I'm told.

Linux.com: How did Sendmail, Inc. successfully marry the needs of a commercial entity with the needs of the Open Source developer community while maintaining the integrity of the core technology?

Eric Allman: First of all, we decided early on not to do a "pure" open source play. That is, while we are committed to keeping the core MTA technology as open source, we are selling other parts of the software in a more traditional old-school business model. For example, the graphical user interface is not open source. On the other hand, this is not software that would typically be of special value to the developer community, who want power-tools, not simple tools.

This division makes it reasonably easy to build a Chinese wall between the open source technology and the rest of the company. There have been fights (er, "discussions") about how this should work, particularly with schedules. For a while we tried to keep the open source releases synchronized with commercial releases, but that didn't work well, and we've reverted to releasing the open source MTA more-or-less when the engineers say that it's done, which seems to work best for everyone.

Linux.com: How did the engineering practices change when moving from a pure Open Source environment to a commercial environment?

Eric Allman: In some ways in the best possible way: we have more people able to work on sendmail full time than we had before. In other ways, the changes have been much more subtle.

One of the first things that I noticed is that there are far more versions of the code in play at any given time. In the pre-company days, we pretty much finished one release (that is, released it and let it stabilize) before starting the next. Today, we have both the current and next release active at any given time, plus the versions that go into commercial releases (these are very close to the open source releases, but in some cases may have slight variations; it turns out to be easier to just create a separate branch early on), plus any special customer branches, plus experimental or porting branches, and so forth. We've been using CVS for source management since we started the company, but that's starting to run up against the wall, so we'll probably have to re-evaluate soon.

There are also, necessarily, people like management and marketing involved. They fulfill a valuable role, but they have very different perspectives on things, some of which impact engineering.

Linux.com: Has the Sendmail Consortium (i.e. the Open Source initiative) enjoyed benefits from the commercial venture? How involved are you with the Sendmail Consortium?

Eric Allman: Absolutely. As I mentioned above we now have more people working on open source development than we did before, and if we need to buy commercial software to help us develop, debug, or support sendmail we have resources to do so.

Perhaps more importantly, we now hold an annual conference of the core people in the Consortium. This costs a non-trivial amount of money, because these volunteers come from all over the world. But a huge amount of idea exchange and review gets done in a very short amount of time. It's well worth the effort.

Personally, one of the down sides of founding a company is that there is always too much work to do, and sadly I find I don't have much time to code any more. I do some occasionally, but I try not to get pulled into any schedules - these days, it seems like everything I do is late.

The annual meeting of the Sendmail Consortium and friends (the "Meeting of the Minds") focuses almost exclusively on the open source MTA. However, these people are far from stupid, and they understand and appreciate the needs of a profit-driven company. Inevitably, advice about commercial releases is offered. But the focus is very definitely open source.

Linux.com: When you started the commercial division sendmail already had a massive dominance - it was used all over the world. How important do you think that was to building Sendmail, Inc. as a company?

Eric Allman: I think it was very useful to us. It certainly helped us during the funding stage - it never hurts when the investors have already heard of you. But it has also been, and remains, useful to us during the sales cycle. It doesn't automatically sell the product, but it helps get us in the door. Just like investors, customers like to buy things they have heard of to get the job done. Linux.com: Recently you watched the debate between Microsoft's Mundie and RedHat's Tiemann, what did you think of the debate in general?

Eric Allman: I thought Mr. Mundie's prepared remarks were quite good, but he seemed to not do anywhere near so well handling off-the-cuff questions. Mr. Tiemann has a tendency toward hyperbole which I thought diminished the impact of his statements, although clearly the crowd loved it. And speaking of the crowd, I was disappointed by some of them - they were clearly there to make a point (by trying to embarrass Mr. Mundie) rather than investigate the issues, so I felt that the Q&A portion was less interesting.

On the other hand, I was able to go to part of the press conference after the open session, which was much more interesting. Unfortunately, I didn't see the whole thing, but what I did see involved much more thoughtful (and harder) questions, with quite a bit of discussion. It was pretty clear that the press wasn't on Microsoft's side, but they were letting questions be fully answered and then asking follow-up questions; this was able to get a bit deeper into real issues.

Perhaps the person I was most impressed by was Tim O'Reilly, who very effectively moderated both sessions, and was truly interested in creating more light than heat.

Linux.com: I saw the debate also, and thought Mundie did well, all things considered. His description of an ecosystem, which includes an intellectual commons and a commercial software business makes a lot of sense. What doesn't he understand?

Eric Allman: Perhaps his description of an ecosystem is a bit too close to the mark. For example, one commercial enterprise might be lumber. There's nothing inherently wrong with lumber - it's a fabulous building material, and it is renewable if properly managed. But clear-cutting forests is not a responsible way to go about it, even if that is cheaper and more profitable in the short run. Clear-cutting often results in serious erosion, thereby making that land essentially unusable by anyone (well, except perhaps for the strip-miners) for a significant time.

To me, Microsoft has behaved like the clear-cutters of the computer software industry. He claims that Microsoft has been a major innovator. I don't believe that to be true - essentially all of their major "innovations" have been purchased rather than built - even most of Microsoft Office was built elsewhere and then purchased and incorporated. It's less risky to let the true innovators explore the territory and then reach down and pluck the winner - and crush the competition, essentially reducing genetic diversity, which slows down and in some cases may even stop evolution entirely. This doesn't enhance the ecosystem; it exploits and damages it. I have absolutely nothing against the ecosystem that Mr. Mundie describes- in fact, I think it's essential. But given that we do all share a single environment, be it a planet or an industry, the players must behave responsibly in order to ensure the perpetuation of that world.

Linux.com: Mundie made the point that our western society is formed on the idea of intellectual property ownership and getting paid for money invested in patent development. Proprietary software companies are starting to repeat basically the same message in response to Open Source: look, we all gotta eat. It's a valid question, what are your thoughts on that issue?

Eric Allman: Western society formed on that idea? Somehow I think that certain other principles such as cooperation between individuals to create a society based on principles of fairness rather than sheer strength might be closer to the mark. However, I do agree with the "we all gotta eat" principle, and I do believe that the concept of IP is intrinsically fair. However, I also believe that the IP situation, at least in the United States, has been corrupted. For example, it's possible now to file a vague patent, keep it alive by updating it periodically, and then revise it to match something interesting that someone else has done, get the patent issued, and then sue the true inventor for patent violation, even though the patent holder actually stole the idea from the inventor. This is simply broken.

Also, IP law has been revised to extend the life of many forms of IP; this is particularly true in copyright law. That seems backwards in a world where things are changing more quickly, not more slowly. This doesn't strengthen the inventor, it strengthens the companies that can afford to buy up IP and can afford large legal teams.

There are legitimate cases for extending patents - the classic case being drug companies that lose a substantial amount of their patent protection during the approval period. However, extending the life of patents isn't the way to fix this - for example, perhaps the clock shouldn't start ticking until the invention can actually be sold. There are ways to abuse this as well, but perhaps they aren't as onerous as the current system.

Linux.com: What's the main competition for Sendmail, Inc.'s products these days, and what are those products?

Eric Allman: We're finding that our sweet spot is in the higher end of the market - mostly companies that want us to come in and architect a new mail system or overhaul their existing system. In this part of the market our main competitor is probably Microsoft. Interestingly enough, we see ourselves as complimentary to them, not in competition. We're not a groupware product, and they don't specialize in high volume Internet mail routing. In fact, many of our customers use Exchange internally with sendmail as the front end to the Internet. They find that much more secure and reliable than putting Exchange directly on the net.

After that the competition impacts us far less - for example, we've gone up against Openwave with a couple of customers, but this is rare because they are mostly in a different market segment (the "carrier class" service providers) than we are. Of course, the market is changing constantly.

Linux.com: What do you see for the future of commercial Open Source software companies? Recently some people have been questioning whether there are Open Source business models at all. How do you think the next 5 years will play out?

Eric Allman: For the ones who said "it would be cool to do an open source company", I think the future is dim. From the corporate perspective, open source is a means, not an end, and if you confuse those you're going to have problems. (Note though that this need not be true for everyone; there are a number of not-for-profits that see open source as an end, and that's perfectly reasonable.) On the other hand, companies that accepted up front that they were going to have to make sure that revenues exceeded expenses will probably have a better time of it.

Of course, there is no difference on this point between open source companies and traditional companies. The market is pretty weak right now, and companies of both stripes are failing at a pretty rapid rate. Unfortunately, some of them are going to fail not because of a bad business model or incompetent management or a product that just doesn't work, but because of the luck of the draw. That's unfortunate, but that's the way life works sometimes.

Linux.com: Do you think that Open Source or free software benefits mankind in any way, or is it just an alternative development paradigm? (In other words do you see any kind of moral or ethical imperative in free software or Open Source, does it have a value beyond making money?)

Eric Allman: Yes and yes. For example, I'm pretty sure that if the early TCP/IP code, notably the stack built at Berkeley, had not been open source, the Internet wouldn't be where it is today, so (assuming you agree that the Internet has benefited mankind) there is definitely some benefit. On the other hand, for-profit companies are in business first and foremost to make money. Even Ben and Jerry's (Unilever buy-out notwithstanding) first task was to make a profit; this is necessary if they are to be able to continue to pursue their social agenda.

(By the way, this applies to not-for-profits too. For example, Usenix Association has a very explicit "good works" program that gives away on the order of a million dollars a year. But to do this, they have to continue to run successful conferences that bring in money that they can then give away. Not-for-profits don't have shareholders who have to be paid, but they do need to make sure that their bank account stays in positive territory.)

Linux.com: For those of us who live in the bay area, you've made what to me is an interesting decision and located Sendmail, Inc.'s office in Emeryville, a town in the east bay just opposite the city and at least a one hour drive to Silicon Valley. Why did you locate there?

Eric Allman: Maybe because I hate Silicon Valley? It feels like L.A. to me - not my favorite place. I live in Berkeley, and definitely wasn't willing to commute (tried that, never again), and moving from a place that I love to a place that I find tepid at best would have been painful. Greg Olson had already done the move to the valley, and was desperate to get back. Real estate was much cheaper, and programming talent (from places like UC Berkeley) was abundant. The people who have the hardest time of it are the management and marketing types, who mostly live where the money is - that is, the valley.

Linux.com: What do you do for fun?

Eric Allman: I like to eat good food. I cook and collect wine. I like going for long walks when I can - as I write this I'm in north Wales, where there are amazingly beautiful walks across rugged hills. I enjoy reading non-technical stuff - I recently finished re-reading Fellowship of the Ring, in hopeful preparation for the movie release in December. I listen to music (some modern, some classical) and go to Cal Performances (based at UC Berkeley) for dance concerts, recitals, modern opera (I love Philip Glass), and so forth.

And when I have time, I immerse myself into programming.

Linux.com: What do you like best about programming?

Eric Allman: That's like asking what you like best about sex. There are so many things and ranking them seems pointless.

I like solving puzzles. I like challenging myself. I like building real things that really work. I like being able to lose myself in the problem, work for hours without even noticing the passage of time. I like the "Aha!" moment when it all becomes obvious and you can berate yourself for not seeing it earlier. I like the especially clever solution, the really great hack, the beauty and efficiency of a problem well solved.

Linux.com: What operating system do you run?

Eric Allman: FreeBSD. Sorry, I'm not a Linux user, perhaps because of my Berkeley roots, perhaps just because I find BSD more familiar.

Unfortunately, I do need to have NT or Win2K available to exchange documents with non-Unix users. Office suites such as StarOffice are a step in the right direction, but they aren't all the way there yet. But I'm waiting with great expectations.

Of course, inside the company we have most major systems available for development and testing, and I'll use them when necessary.

Linux.com: What is your favorite pudding? I mean, if you could have any dessert you wanted, what would it be?

Eric Allman: I don't think I have "a favorite" - it depends on my mood. I tend not to go for big heavy things like chocolate torte cakes or tiramisu - I'll tend toward sorbets or something like that. Although 4th Street Grill, a long gone restaurant in Berkeley, used to have the most exquisite ice creme sundaes, and I have been known to consume a particularly good chocolate truffle.

Linux.com: Mr Allman, thank you so much with talking with us here at Linux.com.