Innovation in Open Source Software?

ndogg asks: "Many have said that there is a lack of innovation in OSS software, and tend to talk about the big projects, like Mozilla and the Linux kernel. However, I would contend that innovation is quite abound in OSS, but that the problem is the spotlight is rarely shown upon those projects that are truly innovative. For example, I would contend that Data Display Debugger (DDD) and The Boost C++ Libraries are quite unique and innovative projects. What OSS projects do you feel are innovative, but underapreciated?"

I nominate ZeroConf

[2nd Post!]

Aka Rendezvous. It is an Apple backed technology, but it is open source; albeit not the classic example of open source springing up from the commons, but it still qualifies.

The innovation is to produce unencumbered versions

[kriston]

While there are a few notable exceptions where existing trade-secret software packages are released into Open Source, such as AOLserver, Netscape, and Solaris, much effort is expended into producing unencumbered versions of existing proprietry software projects. The many Open Source projects such as glibc, HURD, GNOME, OpenSSL are duplicates of existing technologies. I do not see how these projects are innovative except for being in themselves unencumbered versions of existing, known-good, encumbered products. In the linguistics field there is a move to produce a duplicate of an existing, proprietary pronunciation lexicon; there is nothing better than the "free" version except that it's "free". In fact, the "free" version is very unlikely to become a viable alternative. Imagine if that effort could be used elsewhere how much further along we might be?

At the same time, the important technologies that are in Solaris, AOLserver, or Netscape are truly innovative. The improvements that go into these projects are even more so.

Unfortunately too much effort is spent to produce unencumbered clones of known-good projects.

Re:Synaptic

[Anonymous Coward]

Forget about dependencies which are (NOT for reasons of good design) not a problem on Windows. Show me the program for Windows that lets me add repositories of categorized software packaged specifically for my system, browse and search said software, and queue it up for installation (installation that does not require mindless clicking through some "wizard" installer), and at any time check to see if a newer version of programs I have installed are available. I think this system itself is "innovative" in many ways: People packaging up scores of software packages and making it available for download and installation in such an easy way. For all the talk of how hard it is to install stuff on Linux systems, I don't think I've seen anything remotely as easy to use for a random non-techie user than Synaptic pointed at a good set of repositories. And no need to mention how some packages are badly done and don't work etc. I've had to deal with plenty of retarded Windows installers, written by the actual people who make and sell the software being installed.

Bittorrent

[woobieman29]

'Nuff said

Re:Synaptic

[gregmac]

Windows and Mac OS X both have similar installer and update functionality. The difference is that they are both more stable platforms (in terms of whether or not certain packages are available) -- you don't need the same kind of dependency management that you need with Linux.

Is that a good thing though? Maybe you're missing the point. I haven't used OS X so I can't comment on it. I also haven't used Synaptic on Fedora -- I have, however, used Synaptic (and even more often, apt, which Synaptic is a front-end for) on Debian.

Windows Update updates windows, and possibly some other MS applications. The apt repository on my debian workstation has about 18,000 packages available to install. A lot of these packages are libraries, etc, but there are also quite a few applications.

The power of having a shared system of libraries, however, is that updates are automatic. If you're using, say, libssl, to make connections to servers, and there is a flaw or security hole in it, as the application developer, you don't have to do anything really. Once libssl is updated, your application is updated, and that's that. If anything, the next time you release your app, you specifically depend on >= the updated version of libssl. The other benefit is when a developer effectively abandons a package - it can still get updates, if there are problems with libraries it uses.

Contrast this to Windows. Since there are not really any central libraries, each application has to bundle its own - which means that the developer is responsible for updating their package to release the new version of the library. Obviously any core packages to Windows will get updated by MS eventually, but there are also a lot of 3rd party libraries in use. Some applications even put their dll files in the Windows directory, and while that would normally be a good idea, there's too many developers that don't play nice, and require a specific version (their app breaks when another updates), or install an old version and break other apps.

Sure this could be fixed, but all it takes is one developer to not adhere to the rules. On Debian, this is handled by the apt team - if an app doesn't play nice, it won't meet the requirments to get into the repository. Microsoft could do something similar with Windows Update, but I have a feeling that would end up where code signing has ended up - MS charging lots of money, and no developers will to pay for little perceived value.

Re:The innovation is to produce unencumbered versi

[Godeke]

How is this duplication of effort different from any other competing products? Proprietary competitors are exerting "too much effort" producing *encumbered* clones of encumbered products. It would seem that Open Source makes a bit more sense, as once effort is placed in an unencumbered product the product exists and can be built upon, extended or borrowed from. Is the production of a bunch of products that will compete, fail and then take any innovation with them to their grave *better* than the open collaboration that Open Source allows?

Under the argument you propose, Linux was a pointless exercise because it produced a "free" work-alike to an "existing, known-good, encumbered product". Meanwhile, do you think that if the "effort could be used elsewhere" it would have been used to produce something useful, like say another failed Word knock-off? I don't quite see how proprietary dead ends are better than open source clones, which at least have the virtue of being immortal. (I am doubious of the idea that "more effort" equals "more innovation, as you can see).

No, I don't buy the "open source for everything" mantra that is spouted here. Each development method has its advantages. I doubt that Open Source will be invading the vertical markets any time soon as the user and developer base is too small for the advantages of Open Source to be felt. But when you are talking about things that have become "commodity" such as OS, database, office suites, etc then Open Source makes sense... and that's why they aren't very innovative. Open Source is the ultimate expression of commodity good.

The extreme alternative is for all products to be commercial forever, meaning that large companies continue to cash in on less and less innovative products. Office 2003 had exactly one product than made the upgrade worthwhile to me: Outlook. (I.E. a version without the 2GB file size limit). Products like Open Office keep Microsoft honest by forcing them to try to innovate, but to be honest, how much farther can the basic office suite be pushed? If it turns out the answer is "not much farther in core functionality" then by all means allow the ultimate expression of commodity good be created in that market and start to succeed. If the answer is "all over the place" then I guess we will see just how far office suites can be pushed. Everyone is a winner, either outcome. Discounting the "innovation pressure" than Open Source is putting on the commercial vendors I think understates the value of the ultimate commodity good. No, the products are not usually massively innovative, but they force the commercial vedors to be so or perish.

Dasher

[femto]

Dasher [cam.ac.uk]

It seems innovative to me.

I would make the point that innovative does not equal successful. In today's winner takes all world, the term innovative often seems to be restricted to successful innovations. Unsuccessful innovations are valuable though, as they rule out things which don't work.

Don't forget the early "open source" projects

[ekuns]

Much of the Internet runs on software that was open source in some way early on -- such as bind, sendmail, perl, the original web browser (Mosaic), and so on. How many of the "backbone of the Internet" common RFC's have been implemented in open source from the get go?

Don't forget code from DECUS and other such collaborative projects.

Many of the open source projects that people are most familiar with (because it's software they interact with in an obvious way) may seem like a "copy of an existing closed source project," but under the hood there is a lot of innovate software that quietly runs things. Also don't forget that much of what open source is said to copy is software concepts that started out open before the commercial world threw money at it (think, Internet Explorer).

Keep in mind that the amount of software the average user encounters in an obvious way is not huge. It's things like the windowing system and an office suite and a browser, plus some other apps.

When open source or academics or other groups come up with something new and innovative, the commercial companies very often copy it themselves. People who come along later and don't know the history might look at later open source projects and say that they are just implementing what commercial companies have implemented.

Re:The innovation is to produce unencumbered versi

[Scott7477]

I believe that there is great value in having unencumbered versions of basic software. The core functionality of word processors, spreadsheets, databases, and email clients have not changed in many years. Without the feature bloat that vendors of commercial versions of the above have included to justify maintaining the prices of their products, the cost of such software would have been driven to zero quite a while ago. This would have saved consumers, governments, and businesses billions of dollars.

At the same time, MS products still have value because they fill niches that are created by software vendors in other markets. For example, many people in the business world use Excel extensively because the reporting functionality of ERP programs is so lousy. Having Excel allows me to manipulate data dumped out of an ERP to generate analyses and reports that I need. Otherwise I would have to send requests to the IT department to have reports written and I'd have to wait weeks or months to get them.

Big projects don't innovate.

[Anonymous Coward]

Almost by definition, software based on a truly new idea has to start from scratch, then if the idea works, it might grow into a big application. This is true for both free and closed software. Of course there are incremental improvements to existing projects that can be quite creative, but even they will find their way into the official version faster in a smaller project.

Not that there is any shame in making the most of other people's good ideas:
One of the most direct advantages Free Software has is that innovations from multiple sources are contributed to the one codebase, so that the resulting software has the best of everything.

Re:This is a much better example than those given.

[ComputerSlicer23]

Don't take this the wrong way, but "I don't think that word means what you think it means". Or whatever the obligatory quote from "The Princess Bride" is.

You can have an innovative stone block bridge design. You know, the same things the Romans built 2000 years ago. You can still innovate on them.

innovation
n.

1. The act of introducing something new.
2. Something newly introduced.

Innovation inside of the VM subsystem of the kernel happens. It's esoteric and 99% of all people don't care, know or see it. However, if you are "introducing something new" into the VM subsystem, then it surely fits the definition of "innovation".

I have no idea what Rendevous is, or what it does. But I can give you an example of C library that is innovative. Readline written by GNU. It is innovative. To the best of my knowledge they were the first group to introduce such a beast, and to the best of my knowledge it is still fairly unique. It's a straightforward library that you can link into any interactive program where a person might edit a single line of text. It automatically gives you keymappings, history, and all kinds of other goop. So any application I use, that uses it has pretty much the same interface as far as I'm concerned (gdb and bash are the two applications I know I use it with all the time, I'm sure there are others). I like to configure it so I can use vi style commands into it, to speed commandline editting.

Innovation isn't strictly limited to a particular level of implementation. Innovation could happen in the processing of toliet waste water. It's still innovation. So I'm curious about how you feel an innovation can't happen in a C++ library. I mean, mozilla is nothing but a bunch of interconnected C++ libraries. I'm fairly sure there's lots of innovation in there somewhere. It's got to be contained in the C++ libraries somewhere.

Kirby

Re:httpd, tex

[jbolden]

1978 Xerox releases Interpress
1978 Knuth starts work TeX and releases the first version
1982 Geshke and Warnock leave Xerox and form Adobe a company designed to take the ideas of postscript commercial
1984 Adobe release Postscript level 1
1985 Postscript laser printers hit the market this includes image setting
1986 Adobe releases postscript fonts
1989 Knuth finish TeX

Seems to me they came out at the same time driven by similar issues with different target markets

GPL

[cyphercell]

If there is anything that will ever be considered revolutionary it's the GPL. This liscense is the sort of thing our grandchildren will read about. I would also assert that this is still innovative, as most people who use computers don't know what it's about. It is our declaration of freedom and it deserves more attention from the media than it has gotten (none). I personally beleive the most innovative thing in OSS right now is the liscences and the people who are reading them for the first time.

Maybe a different definition...

[Undertaker43017]

Neither of the given examples are very innovative.

DDD was done 15+ years ago with CodeCenter/ObjectCenter. Boost looks a lot like RogueWave libraries.

Most OSS projects are playing catchup with some product in the commerical world, innovation is hard to find. A couple that come to mind are Struts and Cocoon. Both of these frameworks where different from any other web framework, at the time.

Re:Synaptic

[mountain_penguin]

nah there is nothing stopping proprietary systems using apt. In fact I wish sun + oracle + ibm would provide there own apt source for debian and include the dependencies that they need There is username and password support in apt so you can still charge for things.
It would just provide a level playing field.
It would be possible on a proprietary system to have all software installed via something similar to apt and have the installer add more locations to search. Then you could update everything in one place. The design of apt is outstanding in this regard.