Nurturing Ideas Into Open Source Projects? 109
"Until recently, I was the leader of the SquirrelMail project. When it started, we released version 0.1 and people started hacking on it. However, when we decided to do a rewrite, we attempted to start over using the bazaar model from the ground up, allowing for group discussions and decisions. We got caught in a years worth of discussion before any code was actually developed (now, however, its development is well under way and flourishing). I've seen this through personal experiece with countless other projects as well.
As I am venturing into this territory once again with a new project, I'm wondering if anyone in the community has had personal experience with this, and can lend advice as to how to avoid endless bickering about trivial issues. Having a code base to release is obviously a key factor, but in this case, that simply isn't possible due to the magnitude of the task at hand. Advice?"
tip (Score:5, Insightful)
"Beware `we should...', extend a hand to `how do I...'"
The point being that people who do nothing but talk and argue over details are not going to assist in moving forward and worse, are likely to slow things down.
Re:tip (Score:5, Insightful)
Successful projects always start out with someone (or occasionally a few people) doing a bunch of work. General George Patton once said, "It's better to have a bad plan now than a perfect plan tomorrow." Someone has to go ahead and start doing some work. Make it available, be open to accepting help. Do not, however, wait for some magic moment for everything to be perfect and have dozens of people ready to go. That moment won't ever happen.
Re:tip (Score:3, Insightful)
Wrong (Score:3, Interesting)
Try this experiment: get together with someone else, sit down at the same computer, and try to write a piece of software together. Try to write an essay together. Try to fill out a spreadsheet together.
I've been on plenty of committees. The good ones realize that a meeting is to review progress and make sure everyone is clear on the plan. The bad ones think that the meeting itself is the productive work (instead of overhead to get work done).
Re:Wrong (Score:2, Insightful)
Re:Wrong (Score:1)
However if you cram a room full of people. Varying from coders, managers, psychologists to someone who just figuer they'd get some donuts;
The first (pair programming) is a team, not a committe.
willing and eager (Score:2)
Getting a project started by committee sometimes consists of a bunch of people, each of which hopes that the others will do the actual work.
Open source projects, and other projects, that attempt to start in the latter style generally don't get anywhere fast.
Re:tip (Score:2, Interesting)
Re:tip (Score:1)
I think this may have come from The Fountainhead [amazon.com]. At least that's where I first heard it.
Re:tip (Score:1)
When an individual makes all the decisions the work done is the result of a force with a very narrow scope.
When a room full of Marketing, Support, Design (as in graphics), User Interface, DBA, Systems and other geeks work together, the work done is the result of a force so broad in scope, it can't be stopped.
It's like a slow, massive glacier. Hard to "see" the progress, but still unstoppable.
Re:tip (Score:1)
This scheme requires that the core developement/design (design/developement:)?) team agree on a majority rules veto system. This system is, stated simply:
If a problem arises that extended discussion may hinder advancement of the project
- If time permits continue discussion.
- Otherwise take a vote on the issue, majority rules.
It seems to work for me. There have been some hurt feelings but in the long run the entire group has been more productive and Happy!
Data Point (Score:5, Informative)
There have been at least a half-dozen attempts to plan such a system, but AFAIK none have made it to the point of being well-documented, let alone well-coded.
This is a shame, because its one of those "killer apps" that could rocket Linux into mainstream business use.
Re:Data Point (Score:2)
Re:Data Point (Score:2, Informative)
GameDev.net is run by a diverse group of people spread all over the U.S. A workflow system would allow us to take incoming emails for example or new news or article submissions and perform a sequence of tasks on them.
For example, all new email items could be directed to one individual who screens the email. The items that made it through the initial screening could go to whomever was designated as "email answerer" for that week. The resulting replies could be sent to another individual for approval and then the reply sent.
Incoming articles could send out a copy of the article to each person on a review committee. They could approve or decline approval but if a sufficient number approved the item it would be automatically posted to the site.
There are lots of examples of this but it basically has to do with data movement and editing among a variety of humans and automated processes. The movement is normally defined externally in some kind of program or script so individuals don't have to know what happens when they get a new email and hit the approve or disapprove button. They just do their job and the system moves the data to the next person or directly to its final destination without their having to be informed of every change in procedure.
Not entirely correct! Check out Sourceforge... (Score:3, Informative)
The project is based on J2EE standards as well as standards proposed by OMG (Object Managemnet Group). The workflow piece in particular is based on stuff from WfMC [wfmc.org].
So, have a look through that. A simple search on "Workflow" revealed many other projects on Sourceforge, otheres look like they might have a great deal of substance as well.
Re:Not entirely correct! Check out Sourceforge... (Score:2)
unfortunately any search on sourceforge reveals a hundred projects that exist in name only. They really ought to do some house clearing.
Re:Not entirely correct! Check out Sourceforge... (Score:1)
So, the number of sheer "shell" projects under "workflow" I think is less, but like I said I only examined the one I mentioned at all carefully. Some of the others sounded good - a lot of times I use the number of message posts as an initial determination for alive a project is.
In the general case though, I have to agree with you that Sourceforge could probably use a sweep for projects with no activity ina year and no code at all!
Re:Not entirely correct! Check out Sourceforge... (Score:2)
My Experience (Score:5, Interesting)
They just kept asking, "Where's the source code?"
It really is tricky to get an open source, distributed programming project started, because the new people don't have anything to hack on. There's no jumpstart or catalyst.
I wound up just writing the whole thing myself, and never got around to opening the source. It's a loss, because it'd be much better and much more widely used had the idealistic methodology actually worked.
Re:My Experience (Score:3, Insightful)
The first was to get a core of an idea and then hunt down people interested in helping. The problem with this is that the people you will attract will not be so interested in getting work done. At best you'll get a lot of armchair coders that are most interested in shaping the idea, often in directions you completely disagree with.
The second was to do it all by myself and hope others will be inspired and jump in. They won't.
What you need to do is put yourself in the shoes of your potential fellow coders. What will get them excited enough to put time into the project? The best example I've seen is GIMP. They got a framework up and working in a short time and made the project modular and extensible. They then primed the pump by getting some cool modules written. Quickly others made modules and next thing you knew there was a large group of developers making plug-ins and actively improving the architecture to make their plug-ins work better.
Re:My Experience (Score:4, Interesting)
----
Here are some points I've noticed are useful in building a successful free/open projects.
. Don't assume that you will get any help. This sounds antithetical to the whole point of opening up the source. However, there are other points; an end-user contributes a patch.. and then gets competent enough to understand some aspect of the software that she can provide help on the mailing lists. Eventually that person may contribute more. And even if not so many developers help out, you probably at this point have an infrastructure that is good for efficient (== less costly) support. As Netscape learned, opening up the source is not some magical fairy-dust, but it does have some very deep overall advantages.
. Stable mailing lists. Should have an archive, so you're not answering the same question over & over. As well as a faq. You might find that much of this infrastructure is very useful even just for purely internal projects.
. Have a pretty good first release. That's how you attract users. The point is that some of these users start liking the software, and would rather help it along rather than switch to something else that doesn't quite satisfy their needs. A secondary advantage to this is that you'll probably have a good design at this point, since you were forced to deal with the monster. Make your mistakes before releasing it, so the developers will know that you have experience and trust that you understand the sticky corners.
. Feel free to look at other projects. It clearly seems like you're developing in Java, so I'd suggest looking at the guys at jakarta.apache; they have an entirely straightforward approach to things, even if one of the mailing-list archives gets bounced around to different URLs sometimes...
And at this point, you probably will notice that you are doing the best practices that you should have been doing for any project, internal or otherwise. (Making sure to comment, for example.) Just a small generalization to external development. And it will probably be some of the best management training you will ever find.
Re:My Experience (Score:2)
"Where's the source code" is totally reasonable... it's certainly consistent with the meaning of "open source".
The world does not need any more empty SourceForge projects.
Re:My Experience (Score:1, Offtopic)
Hey, maybe that's why it didn't succeed as an "open source" project! Just a thought.
Where are the Universities? (Score:1, Interesting)
Are they able to participate in nurturing meaningful Open Source or is it beneath them?
Re:Where are the Universities? (Score:2)
huh. where are the universities? look around, see plenty of stuff going on.
at my institution (I'm a grad student, not in CS), there are strange wars around campus over operating system choice. the campus is sponsored by Sun and so we have these javaboxen all over the place. however, there are a pile of machines owned by various faculties that are all running iis, and yes, they were all hit by code red. one of my (non-CS) profs has a linux server in his office. so by no means is it a MFC house: rather it would be best characterized as a war zone. the only CS prof I know seems to be doing all his work in java. also, I know of plenty of other places where perl is running everything.
but then, I'm in library science, and here in LIS open source is considered a very good thing, because we always need to tweak stuff for our environments. (example: my library, [ncsu.edu] written in perl and GPL'ed).
Leader (Score:4, Funny)
I think most projects need a powerful leader to get them started quickly and push them in direction. Design by committee is slow. So start with a powerful leader, like a 8th or 9th level Paladin or Wizard to lead your party to success.
Re:Leader (Score:4, Insightful)
I think most projects need a powerful leader to get them started quickly and push them in direction.
This is exactly right. I was expand this idea by saying that the most important thing is to have someone leading by example rather than by direction.
What I mean is that the ideal open source leader should take it upon theirself to make it so the project is completed, regardless of whether other people join the project. This allows people to join and leave the project at their own whims, and yet things will get accomplished because the leader maintains a continual forward push.
Leadership by example is also important because the only person a programmer will listen too (besides a person that pays him money) is one who has done the most work in the project. This is why the best leaders for open source projects are, in my opinion, programmers themselves.
ESR's timelines need work (Score:1, Interesting)
No code to bring to the table? (Score:2, Insightful)
OpenSource != leaderless. (Score:5, Insightful)
There are a number of projects I would like to start when I have the time, some of which I would like to develop on SourceForge or whatever. However, I would still like some say as to what features I think fit within the scope or ambition of a project.
Re:OpenSource != leaderless. (Score:3, Insightful)
The important part seems to be working on it until you have something that is actually usable. Once it's usable other people will just start fixing things for you and adding new features.
After seeing all the things that have been implemented (or attempted) in the old version it's not hard to design a new version where again a central person or group does the brute of the work and then the masses again use as a core.
It's sort of a flux between chaos and order that repeats over and over and both parts are equally important.
it's all about the popularity (Score:5, Insightful)
Paid programmers don't necessarily have to have any interest in the program they're producing (though, admittedly, it helps). Therefore, their projects don't depend on their popularity with the community, and everyone involved (generally, PHB's excepted) has a clue. Then again, this model limits the number of minds working on the project, and thus can be detrimental.
Bazaar? (Score:2, Insightful)
Personally, I don't see a project getting off the ground this way, at least not with any semblance of a coherant project.
I also lead an Open-Source project (shameless plug: jspShop [jspshop.org]) but we are definately proceeding in a structured manner. Perhaps this is some kind of outdated model that is staying around from corporate example, but I can't really think what else we could do. In society at large, even, we have 'leaders' to guide our development.
I don't see it as a bad thing.. I think without some top-level guidance a project would, at best, lose efficiency. I wish I could remember who it was that said this in a previous slashdot post, but I can't. I'll paraphrase it here:
'Tell me what gets things done faster.. talking about implementing ideas, or implementing them'
(I think it was the AtheOS guy? Can't remember).
Anyways, the long and the short of it is.. I think top-level guidance leads to an efficient product with specific goals met. Sure it might aggrevate some people.. but they are free to take the source and turn it into what they want as well. (Ala phpNuke - PostNuke).
doh (Score:3, Insightful)
project that failed at the bazaar mode off the bat. I think people want code to play with to get them motivated, not some open source planning committee.
Discussion never creates code (Score:2, Interesting)
Discussion is good, but remember, talking design all day doesn't actually build anything.
bazaar worked for us (Score:2, Interesting)
A group of us had been experimenting with news and analysis stuff, on other sites and on a separate website. Then, someone offered a domain he owned for use.
Everything just came together. We had a vague, open idea (a news website with a particular editorial style) and we all loved it so much that we did, piecemeal, the practical work needed to get the thing up.
we found hosting, modified scoop in order to make a closed editorial queue, found an opening cadre of users.
we worked so hard, and made so many compromises, because we were all motivated by a reward -- getting the site up -- and we didn't have too much invested in each little idea individuals had about the site.
if you stop your motion, your dynamic excitement, and try to boringly work out principles, you will get dragged down.
if the nature of the technology you are working with allows it, do NOT stop. discover principles of structure for your idea as you go.
RAD (Score:1)
Ie., Many eye's (and hands) approach
Re:RAD (Score:1)
Maybe we should call it Open Programming
I think you have to have a leader (Score:2, Insightful)
Look at the discussion & infighting through the recent Linux kernel changes, especially over VM implementations. Now imagine if there was no Linus or Alan to put forth a final decision.
Discussion, debate and arguing are vital, but if you don't have someone (or a very small, focused group) to take that input and render a decision, no progress will be made. Decisions by committee are slow and may not be beneficial to the good of the project.
Re:Open Source the best option? (Score:1)
However, I have a hard time thinking that Linux itself would have received any kind of funding at it's inception. Open-Source allows you to do away with what other people think, and do something on your own. The model of business plan-funding-IPO is great, so long as your project follows the accepted public 'Norm'. Anything a bit quirky, a bit off-the-beaten-path, innovative, even, is extremely difficult to find funding for.
Not to mention that most funding agents expect some say in your project (and why shouldn't they? it's their money) so it ceases, at least on some level, to be 'your' project any more.
The IPO model has it's place, for sure. But it's not the best solution for every project. You have to evaluate the pros and cons, just like anything else.
How about KDE (Score:3, Insightful)
It seems to me that the most successful bazaar projects are very broad in scope and accompany a lot of small, almost atomic elements that are combined. A single monolithic project that requires huge amounts of coordination for each component possibly requires a strong leader to keep things under control.
Part of this has nothing to do with "cathedral vs. bazaar" anyways. It comes down to convincing other people to get things done, and setting a good example by getting things done yourself. A good manager knows when to debate the architecture, when it's time to start coding, and how to get people started too!
Re:Myth (Score:1)
I've been a part of (as well as have led) both OSS and commercial projects and it seems to depends largely on the people. The best thing you can do it work hard to find capable coders and do all you can to keep feeding what is driving them to your project.
Caveat (Score:2)
Organized software development (Score:4, Interesting)
Guidelines need to be set, studied and altered to better fit a model. Flowcharts, maybe... what about general funding? Start any kind of project, where you could look at 2 years of full-time development, is going to require some kind of revenue stream. Do you think Windows XP was released after 8 months of coding? Sure, it SEEMS like it was, but sometime back in 1997 Microsoft had the idea to merge Windows 95 and Windows NT. It took them nearly 5 years to accomplish this.
This similar thing can be said for lots of different software. Development doesn't occur overnight. The first draft isn't the BEST draft... there will always be bug fixes and feature additions. In fact, you may start coding an application and half way down the road realize it would be better done some other way and would need to rewrite half of everything you've done. This is something you need to be willing to do from the start of the project.
Perhaps that is how the new software is done, (Score:1)
No one likes to admit it though. It is a perfectly good model to work with, and I am sure ideas get translated into working prototypes before they are submitted...they are probably given the green light much more easily that way.
Re:Organized software development (Score:2)
Re-writing a chunk of your codebase sucks, but if you're undertaking that kind of task, the outcome will usually be worth the extra pain, correct? And if you know what you're doing already, have an idea of where you want to take it, and some skilled programmers, then you're set.
I think if it were possible to have open source programmers in the same geographical location, you could start from scratch open-source much more effectively; as much as meetings suck, you can cover more ground face to face in a few hours (with some pre-planning, of course) than you can in a week's worth of message-board exchanges.
Re:Organized software development (Score:1)
Re:Organized software development (Score:1)
We all know it takes M$, one of the biggest and richest software companies, 3 revisions to have a useable, semi-popular product, and another major revision to make it an industry leading software package. Then they polish it up a fifth time just to finally meet many (not all) of actual user requirements and complaints that the previous 4 versions didn't have.
Any open source project should be focusing on design and architecture first. If it takes forever for it to leave the hangar, so be it. If it's got high potential, people will jump on board to help tweak some of the smaller issues here and there as the project moves forward.
Easy (Score:2)
end advice as to how to avoid endless bickering about trivial issues
You sound like you've already had more experience in this area than most.
From what I've seen in the Linux camp, bickering is allowed to continue so long as valid points are being generated. Once the antagonists freeze their positions to merely make themselves look good, then it is time for the project leader to exercise benign dictatorship rights and pick a resolution.
And, as others have noted, participants that provide substance toward a solution are more valuable than participants that talk about solutions, and those that talk about solutions are more valuable than those who just gripe and whine about what's bad without suggesting positive improvements.
Paraphrasing from the money aphorism:
One example (Score:1)
Mozilla's one gaping example of what not to do. (Score:2, Flamebait)
What I wasn't ready for was the clashing of inflated heads, the war of egos, the cacophony of nonsense. I got in on the ground floor, got involved in some flame wars (don't ask what about, i can't even remember) and got off shortly thereafter. The bloating code and slipping deadlines are a testament to the impotence of Mozilla's development model.
The bazaar's just too bizarre for me: I'll stick with the tried-and-true methodology of project leads, senior and junior developers, thanks.
Interesting topic. (Score:1)
To bad those with good ideas have to deal with coding difficulties created by the
development talent.
[shrug] if only coding was alot easier.....
Buy In (Score:1)
Peer teams as a model? (Score:5, Interesting)
What if the person who had a new project idea advertised it in newsgroups and
Re:Peer teams as a model? (Score:1)
The only danger I see with this is the possibility that people will pass off the tricky problems as belonging to someone else.
Frederick Brooks wrote about this in "The Mythical Man Month", which is still a very relevant and occasionally funny read.
My problem is this... (Score:2)
Am I just a poor coder, or am I not putting enough effort into planning or is this normal?
Lately I've been better about it, but I can go back and look at projects I did 6 months ago and think, "What the heck was I thinking?" and redo it 10x better.
Bazaar can defeat design (Score:3, Insightful)
Realize that Open Source is a myth (Score:5, Insightful)
You can't "start and open source project" because there isn't really an open source project.
There are projects. Open Source is a type of licensing.
One of the effects of the licensing is that you may get help. This is terrific. We use open source projects, if we modify the system, we submit patches. That's the benefit of it.
However, all open source projects are run as normal projects. Many of the top (quality of code) projects started as University projects (PostgreSQL, BSD, etc.). Some of them are run by corporations, but if the anti-corporate garbage from Slashdot is an indication of the programmers (I don't believe it is, however), you won't get support because nobody wants to make anybody rich.
The trick is to build a solid foundation. If you get help, terrific. However, you'll have to focus on project management. It's like being a "real" project manager, but since you don't pay your programmers, they aren't going to take orders as well.
If this project is of use to a corporation, see if they will "sponser" the project. Maybe you can make a proposition (show them that this could make them or save them X dollars if completed, so if they can supply Y dollars or programming hours (YX) then you can get the unpleasant part done).
Be creative. However, there is no magic bullet.
Building software is building software. Whatever license you stick on the final product is separate from the process of GETTING the product.
Alex
Voluntary dictatorships (Score:2)
Open source projects start with one or two people defining the core and writing some code, then recruiting others to add to the code base. They progress if the founder is good enough at dealing with people to be able to hand out assignments to volunteers and to decide which improvements do or don't get into the code base. That is, they are run as mild dictatorships: Linus Torvalds welcomes suggestions but ultimately decides what becomes part of Linux. Anyone could go make his own changes no matter what Linus thinks, but given the respect Linus has earned, if you fork the code you will probably be working alone. Linux coders voluntarily submit to Linus's decisions because it's better to be working with Linus and 10,000 others than to be going it alone... It's a dictatorship that is benign enough that people volunteer to come under it, and dissenters can safely be ignored rather than shot.
Probably better to write something to start (Score:1)
ERP system - Open source project idea (Score:1)
Have been having problems along the same lines. Am interested in creating an ERP (Enterprise Resource Planning) system for small/mid sized businesses, based on open source. Basically a Linux/Apache/Tomcat/MySQL server, serving up web pages as needed. Lots of interest from folks that I talk to, but no-one is interested in coding. In the early research phase right now, and am ready to start creating reqts docs (spec to follow). I'd like to do this "the right way" with system design, (small but concise) docs, etc.
My questions to the
- Does such a product/project currently exist?
- Would your company be interested in (seriously )trying such a product?
- How much would they be willing to pay - round about figure: 1K, 10K, 100K, etc. (I realize this depends on # of users, features, etc.)
Re:ERP system - Open source project idea (Score:2)
How many individual programmers, working on their own time, need an ERP system?
The Problem Here Is... (Score:1)
Here is an example of a project: Linux (Score:2)
I think that this is the key: give folks something that sort of works, to fire their imagination and get them to commit, and then (assuming that there really are folks who care passionately about getting this hypothetical project working) it'll take off.
If you are proposing the 123rd super-duper-programmer's editor, don't count on folks getting all fired up. The first really usable version of Unix which could run on a PC without sending a month's pay to SCO got a lot of folks really excited.
Need a dictator (Score:3, Insightful)
Still, you can start with very little, version 0.0.0.1 may not need much more than statement of the goals, and just a bit of code. Version 0.0.0.2 might have some of the major internal APIs defined, a sketch of the user interface, and a detail or two in place. Even at this point it is pretty hard for anyone to contribute anything except opinions. About at version 0.0.0.20 you should have the main building blocks defined (in text), the interfaces between them, and dummy code that does not yet do anything, but does exist. That is the first moment people can see where to put their code, and how to write it. There can still be huge undefined areas ( a web interface goes here, a file system there, an intelligent player AI here, some robotics there, and somewhere we need a booster rocket to get off the ground...)
Even if you do not get many people involved in the beginning, optimize your project infrastructure during those early moments, make everything available and visible, cultivate relationships with the most promising participants, and gain their respect by showing some enlightened leadership. Listen to reasonable suggestions, but cut through with decisions.
Still, expect to do a lot of the work yourself, that's why it is *your* open source project. Linus may claim to take credit for lots of other people's work, but I bet he still works a lot on the kernel. It will take years of time, a great idea, and a great amount of respect and skill and hard work to get there.
Best of luck, anyway!