Improving Open Source Using Software Process Concepts? 34
icanoop asks: "I'm working on a project to help improve open source development using mature software process concepts. What process issues do open source developers think are most important and/or can be improved? If you are interested in seeing what is being considered read the problem statement at the project site. It's not final so feel free to suggest changes."
It would be nice (Score:4, Interesting)
It chases off professionals interested in real projects. 'Oh I don't want to get involved with that, there are 30 projects like it on Sourceforge.....'
Maybe my gripe it with how the opensource projects are handled.
Vaporware that sits for 2 years is not a project.
Re:My issue with closed source projects (Score:1)
The goal of this project is to provide a process and tools that any distributed collaborative development team can use. The interesting part is that it is built on/as free software and is specifically designed to help solve the problems that open source teams run into.
Considerations: (Score:5, Insightful)
1) Allowing the developers to dictate the initial design rules. Allow a focus group determine what it is that is required, then let the developers determine how feasible it will be to implement.
2) Fear of COTS product integration. That is, use the right tool for the job. Of course, if everyone's a whiz with CVS and Emacs, then the more power to them. But don't let anyone make a project a "perfect fit" for their tool of choice which no one else is willing to use. That will cause problems later.
3) Not using outside code / help. Often times, portions of what you want to do have already been beaten to death. Look hard.
Of course, you know all of this. It seems your problem statement and proposed solutions on the linked site are quite thorough; I don't see anything that looks like a sticking point.
Maybe you want to restate the question.
Focus Group can kiss my.... (Score:4, Informative)
Re:Focus Group can kiss my.... (Score:3, Insightful)
If it is on your own time after all, I don't see how anyone can force you to write according to what the focus groups say, not like you're getting paid for it or anything.
If I decided to start my own project, I sure as hell wouldn't want somebody to start commiting code to said project when the code benefits no one, apart from possibly its author.
If you want to do something with my code, fine, fork the project, just don't get your knickers in a knot when I decide your code doesn't belong in my project.
It is after all MY project that _I_ started and _I_ control.
Re:Focus Group can kiss my.... (Score:3, Insightful)
If you start a project "for fun" then focus groups are overkill. Now if someone uses your program and finds a missing feature and submit the patch then you have the "focus group" and coder in one person. That's the basic idea with OSS.
You as a maintainer don't have to include it, but most project maintainers have too little time already so I recon they'd appreciate some help.
Re:Focus Group can kiss my.... (Score:1)
I was going to clarify it earlier, but I decided not to (being sleepy clouds my judgement... whooo ^_^ )
Re:Considerations: (Score:1)
> portions of what you want to do have already
> been beaten to death. Look hard.
Very good point! I wonder, rather than telling people to look hard, if we could find a way to make it easier for them.
The solutions I proposed in the problem statement would certainly make it easier to judge whether or not a given piece of code can be reused in your project, but it doesn't address the problem of finding that peice of code among the many thousands of open source projects out there.
That's a great idea, thanks!
Step one: (Score:5, Funny)
Save man-years by not saying things like "mature software process concepts" when you mean things like "good plan."
Re:Step one: (Score:1)
The main goal here is to help those who aren't experienced in "good plans", not the ones who already use them. So using words everyone understands is important.
Bitkeeper, CVS, et al. (Score:4, Insightful)
- Lack of a plan.
- Lack of peer reviewing.
- Lack of predictability (both feature and time wise).
There are many points here, but one of the most important is the lack of a plan. It would greatly benefit most OSS projects if there was a plan of features to be implemented. This would not only tell users and project members where the project is heading, but also prevents eyecandy and other code bloating problems to enter the project too early. It would be good if a feature had to be on the TO-DO list to be included into the project source tree. This way each feature has to be discussed, specified and granted before being implemented. This helps building more consistent software.The second problem, peer reviewing, could be solved by including it in the code versioning system (hense the subject of this reply...). All code must be tested and reviewed by an independen peer before included in the source tree. By introducing automatic testing, such as a small test bench application showing that the submission works, modularity is encouraged. By introducing good modularity, new patches are more easily tested and included in the source tree.
The last point is mainly a project management issue. Someone has to say that these features will be available at this date in this release. This problem is simply the addition of time to the first problem (a plan). This is the thoughest challenge when working with spare-time programmers. Not many will be happy about commiting to a project, then being forced to keep a time plan. Anyway, this can be enforced in the big, with partially paid work-time, projects.
Re:Bitkeeper, CVS, et al. (Score:1)
You could even stat the codes/reviewers based upon total bugs found etc...
Code auditing system (Score:3, Interesting)
I've made this suggestion several times before; what we really need in OpenSource development is a reliable and powerful code auditing controller.
While direction and design are problems experienced by a lot of OpenSource software, it is quality and security issues which are of more concern to mature projects, which have the highest visibility and widest use.
A code audit system would allow a project to be viewed as a graph of procedures/methods, and force every procedure to be marked as audited by a number (variable threshold) of auditers of a predetermined "skill level". i.e. the designers and core audit team together decide that certain procedures and modules are sensitive, and require an audit by three senior/trusted auditors; other less sensitive code requires only to junior auditors.
The system could track the auditing in conjunction with source code control, and use the software call graph to invalidate audits on procedures when dependancies are modified (and automatically marked as unaudited).
e.g. Procedure A calls procedures B and C. They are all marked as completely audited. Any change to Procedure C will necessarily invalidate the audit on C, and by relationship invalidate the audit on A. Once C is re-audited, the audit of A will still have to be performed again (unless, say, a special "interface not changed" flag is used on the re-audit of C).
In this way everyone can be happy that a project release is secure and reliable, based on the extent of the audited code.
Re:Code auditing system (Score:2)
Re:Code auditing system (Score:3, Interesting)
I agree completely, although there should be no harm in having auditing available to new projects (project leaders just wouldn't consider it for the purposes of when to stablise and release).
Although I would like to contribute more to OSS development, I have very little time or opportunity. In a project of any maturity which has a well designed and documented code structure and is employing DBC (design by contract), it is very easy to audit a single function now and then. Coding the function is somewhat more time consuming - even with strict modularity it takes some time to understand the requirements and implement them.
I'm betting that there are many eyes out there willing to find the bugs, but not sufficient tools to track where the eyes are (and should be) looking.
Re:Code auditing system (Score:1)
> development, I have very little time or
> opportunity. In a project of any maturity which
> has a well designed and documented code > structure and is employing DBC (design by
> contract), it is very easy to audit a single
> function now and then.
Amen. Frustration with current OSS development is one of the main reasons I want to do this. It takes a lot longer than necessary to get up to speed on an OSS project and start doing positive work on it.
In some cases, I have contributed code to OSS projects in which I really didn't have a clue what the requirements were or whether or not my code was correct. I knew it worked, and that seemed to be enough for the maintainers because they just threw it in the release.
That made me wonder, should I really be going around and advocating that everyone use this stuff when I am not even confident in my own contributions?
So I decided this project would be the best way I could really help OSS.
Re:Code auditing system (Score:1)
> willing to find the bugs, but not sufficient
> tools to track where the eyes are (and should
> be) looking.
Good point.
Hoot could display source code that the developer would like to have audited, and there could be a central listing of code like this. If there is no one specifically that he wants to do the audit, it could be marked as open for people like you come and audit when you have time.
There would have to be some way to authenticate that the auditor can be trusted, just to make sure that enemies of OSS don't come along and screw with the process.
You raise a lot of good points. Please continue posting or email dev@hoot.tigris.org if you have any more.
Re:Code auditing system - false sense of security (Score:3, Interesting)
If an auditing process like this is used, people might take a function for granted as "working" just because it's been checked by three different people, two of which are "experts". Even experts make mistakes.
A far more reliable solution is to have unit tests (like with JUnit [sourceforge.net] or xUnit). An expert at writing good unit tests is far more useful to a team than someone that just scans code.
The unit tests also become important regression tests, so bugs introduced indirectly are found immediately. For more info, googlize yourself on "test driven design" [google.com].
Re:Code auditing system - false sense of security (Score:1)
> bugs become shallow", not "with an arbitrary number
> of eyes
But most projects have an "arbitrary" number of eyes, not "enough" eyes. Relying on thousands of individuals to review every projects code is very inefficient, and not realistic. It may work for the Linux kernel, but what about the rest of open source?
> A far more reliable solution is to have unit
> tests (like with JUnit [sourceforge.net] or
> xUnit).
The plan is to reuse projects such as JUnit and other testing facilities rather than write them from scratch.
Although no final design decisions have been made at this point.
> An expert at writing good unit tests is far more
> useful to a team than someone that just scans
> code.
This is very true. That's why the proposed solutions include testing as well as reviewing. But there are some defects that can be found in a reviewing process that can be missed by regression testing. Testing only finds the defects that are tested for. Reviewing can find others, as well as improve code.
Re:Code auditing system - false sense of security (Score:2)
I'm going to be a slight devil's advocate here, because I believe strongly in unit testing, but there is a significant amount of academic literature which demonstrates the falacies inherent in using tests to catch bugs.
Incidently, test driven design is concerned with demonstrating that the software can function as specified. It is not concerned with - and bad at picking up - software that performs those functions with side effects, or contains code which can be exploited in a security-sensitive context.
The simplest "proof" of the problem with testing is to consider a function with four arguments; let's say this is C, and two arguments are int while the others are char*. To rigerously exercise a single argument of this function, you need five to seven distinct tests: obviously good value, obviously bad value, inner- and outer-values for boundary conditions (may only apply to the int), and NULL. Unfortunately a black box function may have strange interactions between parameters which testing each individually cannot expose; a proper test set to ensure the function behaves correctly in all circumstances will require approximately 6 * 6 * 5 * 7 = 756 tests, being the permutation of all single-argument test cases.
No, I'm afraid this is not a suitable answer. Unit tests are best used for two purposes: testing that a function produces the correct and expected effect when used correctly; and testing that a function does not produce a bug that has already been discovered (i.e. regression testing).
For everything else, there's Mast^H^H^H^Hcode inspections. A code inspection (audit) will detect problems in the handling of parameters far more efficiently than building hundreds of tests. What is more, it will (when performed by an experienced developer) expose problems that tests cannot: tests are particularly bad at showing up race conditions, buffer overflows and other security risks.
In short: More bureaucracy (Score:3, Insightful)
Of course some of those "process" things can be valuable, and when a project meets a problem, they can use a solution. For some projects it even makes sense to talk of focus groups (KDE comes to mind), and regression testing is used in some (GnuGo).
I think it is fine to make some of these techniques available to those who see the need of them. Good introduction material about such would be welcome. But do accept that most OS projects are better off without a pointy-haired boss and his bureaucracy.
Re:In short: More bureaucracy (Score:3, Interesting)
I'd say that most projects would benefit from a general plan stating which features that are wanted and for whom the project is intended. As soon as a project grows from 5-10 developers a more controlled development process will be needed if to avoid bloat and general confusion about what the project is supposed to do.
One of the main problems when bringing GNU/Linux to the desktop is actually the huge number of options. If projects were better narrowed down, and more structured, most of these options could be hidden, or atleast bundled. Open source does need far better project management than is common today!
Re:In short: More bureaucracy (Score:1)
> write all these papers *before* you start coding,
> this outsider will most likely feel himself not at
> all welcome, no matter how fine theories he is
> trying to push.
Yes, this is a situation that I wish to avoid at all costs.
> I think it is fine to make some of these
> techniques available to those who see the need of
> them. Good introduction material about such would
> be welcome.
And that is exactly what I want to do. To make it available to those who can benefit from it, but in no way whatsoever even attempt to force it on anyone.
> But do accept that most OS projects are better
> off without a pointy-haired boss and his
> bureaucracy.
I don't think any open sourcers are fans of bureaucracy, I know I'm not. But if I'm working on a big project, it would be nice to have confidence that the other developers are working towards the same goals that I am.
It would also be nice to spend more developing new code and less time fire fighting.
Cathedral or bazaar ? (Score:5, Insightful)
Do processes make better software
I've been involved with a lot of software projects (though never contributed much to Open Source...), and I have never seen a single project that was succesful because it followed a process. Nevertheless, whenever a project runs into trouble, the first call is usually for "more/better process !!". So let's look at this in more detail.
Succesful projects seem to grow their own process. The process seems to be simple, and often appears to be way less than you would expect, and rely heavily on interpersonal communication rather than documents and frameworks. There's usual a small core of "gatekeepers" who set the technical and philosophical tone for the project. The Linux kernel is a good example.
I am very worried about people using phrases like "mature process", "industry standard" etc. - in my experience, this often refers to the Rational Unified Process or the Software Engineering Institute's Capability Maturity Model. Both are laudable and when I go on holiday, I really want the airplane's control systems to be written using such processes. However, for many projects, the burden of bureaucracy is inappropriate (yes, I know you can tailor the RUP to suit your needs, but it contains over 140 different deliverables, none of which appears to be code). The training required to bring developers up to speed with these processes is significant, and usually expensive.
Instead, I'd look at the Agile methodologies at Agile Alliance [agilealliance.org] website. The "Crystal" methodologies are especially interesting because they encourage you to actively choose the processes your project needs based on a variety of parameters - size, risk etc.
Having said that, I think a lot of the problems addressed are real - I think they get solved by people, not processes though.
Re:Cathedral or bazaar ? (Score:2, Interesting)
And what about Extreme Programming? I don't know too much about it, but I hear it's starting to gain popularity.
Although a friend of mine suggests that the reason process doesn't work is because in the end, you have to meld the software with the hardware. Usually since your processes don't address the quirks of hardware, you end up having to rewrite a lot of code to get it to work.
Re:Cathedral or bazaar ? (Score:2)
Succesful projects seem to grow their own process.
It is true that porojects are not successful because they followed a process, but successful projects are generally ones that have followed a process. A good process which is followed by the entire team certainly helps. Projects can fail for a variety of reasons, but most of the failed projects I have seen have either not had any processes at all or the processes have not been followed by everyone in the team.
Processes do no have to be heavy-weight affairs like the complete RUP or CMM, but there should be something in place. Lightweight methodologies and processes are perfectly suitable for a large number of projects.
The biggest problem I have found has been getting the project team to follow the process that has been chosen. Developers generally don't like documentation, yet documentation is crucial for projects of any reasonable size. Specifications, design documentation, interfaces, data models, comments in code - these are the things that I had the most trouble getting developers to write, yet when the developers were working on someone else's code they screamed if any of these things were missing.
Having some simple rules about code formatting and keeping documentation up to date (no documentation at all is better than documentation that does not match the finished code!) and rules governing who checks code in/out helps move the project along and keeps everything on track.
Having said that, I think a lot of the problems addressed are real - I think they get solved by people, not processes though.
Yet processes can help the people solve the problems! Succesful projects seem to grow their own process: usually in response to problems casued by a lack of process (or faulty process) initially.
Management systems (Score:2)
OSS can potentially attact a huge number of devlopers, most of which can only spend a few hours a week on OSS, this is amanagement nightmare.
I'd like to tie up all the code/bugs/designdocs/developers into a nice heiricical linked structure kept in a configuration management system. That way I can look through the open bugs
find out what modules it's thought to relate to
find out who's responsible for the various parts and look an creating a patch.
What do processes do ? (Score:1)
I am working for Philips CE as a software engineer, and we have reached in our department CCM Level 3. As I have been here some time, I had some training in CMM and I had to participate in the Level 3 Audit.
I read this post already yesterday evening but I had to think a night about it.
What these software processes are about is formalising the right steps needed to obtain a certain product (plans and planning, development templates), the necessary parts of the process (like setting up a process database, configuration management, bug tracking, etc...) and probably the most important part, enhance and promote good ways of working in the development process and trace and remove bad ways of working.
I think that for an organisation like ours, this is very positive. We have scheduled releases of all kinds of software and it sure helps to deliver working software on the right time.
To implement this however, you need a large organisation and people who are really interested in writing processes.
Now for FLOSS side of things. I suppose that KISS helps here, because not many people working on a FLOSS project will be interested in process development. This means that the people which start the project will somehow need to define their own (simple) process(es) and formalise them.
What do you really need?
I had a look at eXtreme Programming, and I must say that if it is compared with CMM, then it probably fits most goals of CCM from a practical point of view, not from a formal point of view (CMM requires you to keep all your processes written down somehow, for inspection by the auditors).
However, XP (like probably most of these methodologies) is also geared toward fast and correct release of deliverables.
For FLOSS projects, it thus seems more appropriate to concentrate on things which need to be in place, than on procedures to produce deliverables on time. If you want to have this, then mention this at the start of the project and ask people if they are sure that they kan keep their promises.
Re:What do processes do ? (Score:1)
The project is not locked to CMM/CMMI, they just happen to be what I am familiar with. As well as being an open source project, this is also a research project. The research is aimed at identifying processes that open source projects can get the most benefit from, without sacrificing a lot of time and effort to maintaining a process. Those processes may come from a number of sources, including XP.
P.S. Ideas can also go to dev@hoot.tigris.org.
Send further ideas here (Score:1)