Ask Slashdot: On Good Software Design Processes 244
Marko Rauhamaa asks: "I'm a software manager in a medium-size technology company. Today I ran into a professional argument with my superiors about our company's software design process, which, I suppose, is fairly standard: The software team is to write one or more MS Word documents that have predefined section headings. The documents should describe all aspects of the coding phase that is about to begin. Then the documents are peer-reviewed, polished and submitted under document control. Questions: What kind of design process do the successful free-software projects have (Linux, gcc, Apache, XFree86, etc)?
In your experience, how often are design documents revisited after
the project? Have design documents helped in technology transfers (that is, have they been more helpful than the source code alone)?
Are engineers good at writing design documents? Have you been able
to read design documents written by other engineers? Have old design documents been kept up to date with the changes in the implementation?
Has the quality of your products been better because of design
documentation?"
Well... (Score:1)
In conclusion, it could be a lot worse than at your company.
ps. Anyone looking to hire an experienced C/C++ Solaris developer?
My experience (Score:1)
Just my limited experience... All standard disclaimers apply. Also, the rules vary a lot depending on the personalities on the team, the type of project, etc. It's impossible to summarize a general methodology in a slashdot message. I'd recommend picking up a few good books on software engineering.
- pmitros at mit.edu
Extreme Programming (Score:1)
The basic philosophy of XP is to write code until you have something that works. Write a unit test for it. Listen to what the users think of it. Then write it again. Refactor mercilessly.
It's a good philosophy, partly because it cuts out all the crap about up front design and UML and allows the shape of the code to take shape naturally. At the same time, the refactoring ensures that the project doesn't become a mess of unfinished prototypes, and the tests ensure that everything works when you refactor (or at least tell you when you broke something.)
here [armaties.com] and here [c2.com].
McConnell, docs, and IEEE Software (Score:1)
Strong second on Steve McConnell's books, though my personal favorite is Code Complete. Steve continues to write on this topic, and is using his current position as editor of IEEE Software [computer.org] to continue spreading the gospel^H^H^H^H^H^Hmessage.
He's been getting curious about the open source movement and the Linux phenomenon -- note Software's Jan/Feb 1999 Linux edition, and the editor's column and response in the current (Jul/Aug 1999) issue. Access is limited to IEEE and ACM members, but editor's columns tend to show up after a month or so at the Construx [construx.com] website.
Among Steve's criticisms of OSS are that design and architecture documentation are sorely lacking. As others have noted here, there are many instances where free software has set sights on existing functionality -- implementing systems the way they should have been in the first place, often closer to the design documentation than the proprietary application.
I'm a victim (Score:1)
[just venting...]
Project Management Software (Score:1)
An open source package that does this would be nice so that I could add any features that are missing, but I could settle for something closed. It would also be nice if it were accessible from a web browser, but it would be OK if it were not web based as long as it runs on Linux.
Re:You'd better start thinking... (Score:1)
Fred also has said,
Re:My software design process (Score:1)
^~~^~^^~~^~^~^~^^~^^~^~^~~^^^~^^~~^~~~^~
Keep a notebook (Score:1)
Detailed design is only necessary for the most difficult problems. For real world problems, databases, you don't need to document in extreme detail. Since most open source projects are simple utilities and desk accessories, there isn't much designing to be done.
Development Processes and Open Source (Score:1)
That said, I think Open Source projects have different goals and objectives, and this can get in the way of following traditional development processes.
In general, the more successfull OS projects have followed a lowest common denominator approach. They've general lowered the bar of entry for a "developer" to get in on the project as low as reasonably possible. This is a good strategy to follow, as the key part of open source is to get as many people involved as possible.
XP (Score:1)
A few scattered points ... (Score:1)
I do Windows NT development, yet I run LaTeX (on NT) for documentation. Pepole want MS Word around here too, but they can stuff it. If I'm going to document, it will have to be with the right tools that save time and energy. LaTeX also allows me to easily carve up the document into multiple files which are checked into the version control system separately. Since our stupid version control system has strict locking (should be using CVS, doh!) this helps when two or more people need to modify the document. I can also use keyword expansion to label checked out copies of the document automatically so it's clear where they come from in the revision control system, and what version they are.
My second point is this: your superiors clearly don't know how to manage the risks of software development, and are trying to fall back on a naive ``waterfall model''. In this model, a carefully managed paper trail is supposed to lead up to the working product. One begins with the Exctracted Requirements, then moves on to the Requirements Specification, followed by the High Level Design, and the Detailed Design. CASE tools may be involved. Only at the end of the paper trail does the coding begin.
This model is unrealistic, difficult to follow to the letter, and increases your time to delivery by wasting everyone's time with paperwork. It's also unrealistic to assume that the requirements will never change, or that the design won't have to be modified after the coding has begun. It's a huge waste of time to polish documents that may turn out later to be wrong or obsolete.
The activities of designing, docuemnting and coding should be done in parallel with development. Often, it's acceptable to produce a design document after the coding is already done, as a way of gathering recording the intellectual work done by the developers. The design document can use elements from e-mails among developers, and their scrapbooks and whatnot. (It's not like nothing is recorded as the software is developed, it's just that it's not in a polished document!)
What you should do is settle on the requirements specification first, and then run with it. If your project is split up into teams whose software components have to communicate using tight interfaces, then design and document those interfaces, but don't worry about the internals too much. Write only those documents that are essential for everyone to be unblocked so they can go on with the business of writing the software. You need to know what the requirements are, and you need to know the interfaces to software components that other teams are working on, so it's good to have these captured early on. Worry about the design documentation later; wait until at least you are past the prototype stage and are into version 1.0.
Hope this makes some sort of sense.
My Experience (Score:1)
Needs differ. There should be some process, but it differs.
For example, I have just enough documents for my GPL'd game, that I can handle it. They are on my web site for others to look at, but I am the main consumer. When more work is done and others begin to use my code, for example as a library with which to create another game, those documentation needs will clearly outweigh what I have. I will have to produce more documentation.
Finally, search the book reviews for my review of "The Unified Software Development Process" by Jacobson, Booch, and Rumbaugh. It's probably too heavy weight for small/medium sized projects, but the concepts are valid and the ideas intriguing. Reading it didn't hurt me.
Engineers vs. CS (Score:1)
[I am CS.]
In my experience, the engineers suffer from a fatal flaw: they think computer science is easy. They think programming is just a trivial exercise, nothing compared to say building a bridge and putting your name on the blueprints for liability purposes.
Of course I believe differently. I believe we'd have had a bit more robustness in our designs if the engineers didn't just charge ahead into coding, but rather considered the theoretical aspects of what they were doing.
Of course this is a generalization, there were good and not so good individuals on both sides of the fence. But I found the generalization strange and peculiar.
Re:Free Software design (Score:1)
Who did you think /dev/null was? (Score:1)
--
Re:Waste of time - not always (Score:1)
If you're programming BY yourself for others they (and the guy who eventually has to maintain your wonderful creation) could use some clues as to how it works, the caveats to using it, why you did this or that, and more importantly why you DIDN'T do this or that. (A little note saying "I wasted three days investigating this other neat -looking alternate design that didn't pan out" can save you or someone else another three days later in life.)
You may even find that mapping it out a little can save you a ton of time and effort later, or assure that it'll also work with XYZ package/file format/etc. When you're working on a big project composed of hundreds of different programs, you'll be screaming for up-to-date doc on them. Given your attitude though, I don't think there's much chance you'll get hired for anything like that in the near future.
All bow before the Process (Score:2)
Unfortunately, ANY such process, while it may have it's benefits, isn't perfect. True wisdom is knowing when "the Process" doesn't apply.
A few thoughts:
1) You can't think of everything up front, no mater HOW GOOD you think you are. This only works for small systems and systems with few well-defined intercomponent interfaces. You'll struve for the latter, but you won't get it first time around (or even second), but you'll get closer every time.
2) Anything which divorces implementation from design runs the usual risks associated with multiple views of the same think. Code often gets out of sync with UML, for example, though this can be audited for, and incrementally corrected. I do this CONSTANTLY. It is a form of overhead, though, and must be taken into account.
3) While metrics are great for predicting project side and time, collecting them often brings up a Heisenberg Uncertantly Principle effect: the mere act of measurement distorts that being measured. In some cases this can slow down the productivity of your best people to that of the mean -- that's expensive because you're best are likely an order of magnitude more productive than your worst, and so generate far more metrics to collect, even though many of them are useless. This flies directly in the face of rapid prototyping where the goal is to make things with the intent of throwing them away.
4)Don't waste your time auditing for mistakes that you are very unlikely to make. Of course, you need to collect some metrics to find out what mistakes each individual DOES make, but, in the absense of these, most programmers and designers have a good idea of where the mine fields are and ALREADY take extra care.
5)Don't come up with a "one size fits all" audit procedure. Some things are hard to audit for, and not all reviewers will have the skill to do this.
6)Do try to get the machine to do most of the work. If there is a "coding style standard", invest in a prettyprinter program instead of having people count spaces. Reading other's code isn't the same as iteratively developing your own. For example, I like my code to be fairly dense, so more of it fits into an editor window. This way, I can review the last thing I coded for correctness while I code the next one.
7)If something is too error-prone for some to use, but a great productivity booster to others, do let those others use it with the caveat that with this freedom comnes responsibility.
8)Remember that programmers and designers are not interchangable - the difference between the best and worst will be on about one order of magnitude (i.e. a factor of 10). Do not fall into the trap of hindering your best because the average has difficulty understanding a particular technique - if it is well documented (say anything in "Design Patterns", or even "The Art of Computer Programming" (I'm dating myself here), it should be fair game, and the onus on the neophyte to learn from it.
9)Remember to not etch process in stone. Third-party code you license will likely NOT follow your coding standards, and woe unto him who changes it when it comes time to integrate a source upgrade. Neither is it perfect. Don't trust the documentation -- read the code. If I had a dollar for everytime I say code not match documentation, I'd be rich.
10)Document what matters, and keep it short. The basic idea on a design or sub-design should be expressable in no more than a page or two. Don't linger on unnecessary details unless it is a code API.
11)Be flexible. While your process may be best 95% of the time, you will be burned when you try to apply it the other 5% of the time.
12)Know when to violate process, and how to weigh the risks of doing so. There is certainly risk here, but it is not infinite, and it might be far less than the risk of not violating process. This usually happens when (a) schedules are tightened unexpectedly, and/or (b) you run into the 5% scenario that your process handles badly.
13)Don't get bogged down in overhead. If you make a one line change that corrects a clear oversight, don't fill in a three page form, unless the oversight looks like it may be a systemic problem.
14)Finally, never assume anything. You're always playing the odds. Learn to manage uncertainty.
Re:Design Documents. - bravo! (Score:1)
I've always hated uncommented code to the point that I over comment a bit. I always love the ability to pick up a random page of code and be able to understand it just by reading and not having to look up all the function calls and interpreting each line. And I can have a hard time remembering my logic in choosing a particular implementation, so I try to leave notes too.
One reason (if not the only reason) that people hate documenting is that it doesn't directly affect the final product - the compiler preprocesses documentation out of the code stream, docs only are the source code communication between programmers. In short, it feels like a waste of time because we like short term effectiveness and productivity over long term maintainability, programmers feel like they are making a difference in writing things that end up in the compiled program structer. It's all psychological.
I don't have the best 'root' document stating everything a code tree does, but it always has something to start from, and a generalized layout and plan of action.
One thing that might help is write everything in pseudocode, and go back to make real code, leaving the pseudo code as comments. It never hurts to think something through twice rather than coding on the fly and forgetting the details of why you did something, saving much debug and rewrite time.
I like to test and compile my code often, every time I finish even a function or a complex loop / conditional.
I suppose some of you might think I am anal, but I've never had porting problems with my code - not a line changes when compiling for 5 different platforms (NT, Solaris, SunOS, Linux Alpha and Intel/AMD).
While I am not against artistic coding, the scientific method can help a lot, as well as communicable clarity in code writing. The compiler can tell you wether your code is legal according to the given set of rules, it can't tell you anything about future maintainability or readability by you or other people.
Re:Waste of time (Score:1)
Here's some clues:
The size of project affects the software process you should use. You may be a good carpenter and can build a doghouse with your eyes closed, but your skills won't get you very far if you need to build a domed stadium.
The sooner that mistakes are caught, the cheaper it is to fix them. For example, it's much better to catch a missed requirement before the code is written than after, and much better than after the customer has bought and installed the software.
The initial code is a very small part of a "product". Users need to know how to use it. Tech writers need to know how to document it. Testers need to know what they are supposed to test. Product managers and sales people need to know what features there are and what features may be added to it in the future. Other programmers need to know how to enhance and maintain it. Etc. etc. And where do you suppose that all of these players are to get their information?
Explanation of an old maxim (Score:1)
And you can find bold programmers
But you can't find old, bold programmers."
That's because old programmers have too many scars from running boldly onto the battlefield shooting at anything that moves. Other formerly bold programmers are now dead.
Re:Been there, done that. (Score:1)
>This requirement was needed since most military at the lower ranks had no high school or dropped out of school. This is not the case so much anymore.
Actually, This is not the case at all. Entrance to the military (even the Marines!) requires a high school diploma or GED (most branches won't even take a GED).
FWIW, anyways.
Documentation stuff (Score:1)
as it eliminates problems with group communication
and lets me pursue a vision as to how things
should work. IMO, if you have 10 programmers
and 9 programs to be written, it'd be really
great if you assign 1 to making a library with
stuff that the other programmers think are likely
to be useful to many people, and assign the other
9 to one project each.
down troll.. (Score:1)
"Good" Software Design Process? (Score:1)
Usually documentation can only speak to 1 audience. If it's for the lusers.. I mean Users the techies say their isn't enough detail. If you include enough detail the users get bored to death, come back at the end and say I didn't know it said that I thought it said
Here are some of the thoughs I have on project management.. Not really a process though.
1. All users are morons. Untrained monkeys can understand the system but our users can't.
2. Users never say what they mean. They expect you to be psychic.
3. Talk to the Users NOT the managers of the users. Standard biz philosophy says "Managers don't need to know anything about what they are managing." And true to the philosophy they usually don't.
4. Make the involved parties sign off on what the project is. The document is probably wrong but at least you have something to change. (Verbal/Mental designs mutate with no controls.)
5. A project schedule that is highly detailed and inflexible is always wrong after a short number of days. Stick to a functional points in a sequence and an estimate of days. Be willing to change it when it hits the fan.
6. Break up the project into manageable bits.. 5 days for a person at most. Have a done, work in progress and to be started list. Watch your "critical path" for changes because of slippages/changes and adapt as needed. (if it's more than 5 days then you haven't broken it up enough.. I used to not believe that but it's true)
7. Project Management is kinda like when they had to fire the rocket on Apollo 13 with no guidance computer. Keep the earth (end of project) in the window and hang on.
As for the crux of the original question..
Any documentation is better than no documentation.
Find something that works and use it.
Source Control is an assumed. Those that do without it are just ignorant of it. Teach them and they will use it. (I had to do that once.. After a year they said I don't know how we ever got by without it)
Force all changes to documentation to go through a review and sign off. If someone can goto an individual (programmer/manager/etc) and change something with no review you are doomed.
-Jerry (and now back to our regularly scheduled interruptions)
Re:Documentation stuff (Score:1)
by the way, are there ever summaries compiled on the ask slashdots?
"The lie, Mr. Mulder, is most convincingly hidden between two truths."
Re:My advice (Score:1)
If you don't document things, you'll waste time when you have to pick it back up in five months to do phase II or a vital bug fix.
If you don't have a process, your hackish methods of "getting it done" won't scale beyond about 5 people in the company.
If you *do* have a process, your senior people can usually write specs, document interfaces, explore architectures, and then turn *all* of that over to the juniors to get 80% of the coding done. Then they can do the really hard stuff that they know how to do best (because they're seniors).
My company is still working on this, and we'll probably keep working it out until we feel we "get it". But we know that we want to go from 7 coders => 20, and we can't do it unless we get a process in place.
Differing styles of design process (Score:2)
That said, this question touches some deep issues that I'd like to try to comment on.
The main thing is that different projects have different needs in the design process. In most of the projects that I've worked on, learning how to do "X" is more of the thing than actually writing the code to do "X." This is probably because I tend to choose "fun" projects. If I were banging out yet another goddamn middleware transaction thingy, I'm sure that more formalized "processes" would make more sense.
As it is, the usual processes that get applied assume that it's already understood what is needed and how to accomplish that. One common thread to many of the horror stories I've read is the churn-mill of changing requirements and specifications. To me, this is a natural consequence of not knowing what you're doing to start with.
And not knowing what you're doing when you start is necessarily a problem, especially if learning is a part of the process. This is perhaps one of the greatest things about free software - it's a learning-centric process.
I think good design is just as important in the free software world as elsewhere, but it manifests itself in different ways. Usually, there is more than one competing project in a given space. Ideally, the one with the better design will win more developer hearts and minds. This is of course not always the case - often, a program will succeed in spite of its bad design, or perhaps because of it (a certain popular mail transfer agent certainly comes to mind :).
And the other thing about free software design is that it's often a whole lot simpler than comparable designs from the commercial world. Free software rarely comes with chrome-plated tail fins. In my opinion, the mark of a truly outstanding design is that you don't even notice it's there. Take the sockets API for example; compare it with any other networking API out there.
I guess my point is "different strokes for different folks." The fact that free software turns out such good work even though there's no formal design process certainly proves that the formal techniques are not necessary, and suggests that the formal people are missing a big piece of the puzzle. But for successfully executing needlessly complex projects by an army of more-or-less competent programmers, I'm sure they are an indispensable communications tool.
Maybe some day the software engineering crowd will work out tools and processes that work so well everyone will use them, even in free software. But I'm not holding my breath.
Software Design Methodology (Score:2)
In summary, while by no means am I suggesting that doing quality design work before the software is written is a bad idea - I'm just pointing out that in my experience, the only projects that can be designed to the last detail are projects you'd most likely find exceedingly uninteresting. Most software I've written that's worked out well has had a highly iterative quality to it - you design a bit, argue in front of a whiteboard a bit, code a bit, demo a bit then repeat. Certainly a good rule to remember is this: The longer development goes on for without the client/customer seeing the product (even as a demo/prototype) the larger the potential for the project to diverge significantly from customer expectations becomes.
You can't get there from here (Score:1)
Re:You'd better start thinking... (Score:1)
"you'll get about half way into the project, realize the way you are solving your problem is all wrong and you have to scrap your work"
There's nothing wrong with recoding things. In fact, you should always plan on throwing away a lot of what you start doing.
The way I usually design programs (which seems to work well) is by hacking together a half-working solution (very buggy, missing features, command-line, but actually doing something) and then looking at the code and deducing where the separate pieces are, what problems will I face, and then creating a design document which outlines a good design to the problem.
Until you start coding, you really don't know what you're messing with. The best idea is to plan to throw away your first prototype, but use it as a place to design from.
Also, a great book to read about the software design process is "the mythical man-month". It's old, but the ideas are still useful.
Re:Waste of time (Score:1)
Sure, it's no big deal when it's just you doing the coding, etc. Now try looking through some of the programs you wrote 5 years ago (in high school?) and figure out what you were doing. Even better, have someone look through your recent code and have THEM try to add a feature to it just from their looking at your code.
That's when good documentation becomes invaluable.
What your teachers are trying to do is have you get used to this process early in YOUR development as a good engineer, so that it's a mere force of habit once you start doing this professionally. It will not only help you, but also your co-workers and those who will eventually inherit your code.
Sounds like SEI requirements to me... (Score:1)
With open source projects, many of those controls are unnecessary because the pointy haired boss is removed from the picture. So what is left? All the good points from the engineer's view. We have a large number of peer-reviews (more than any company could produce). The main difference here is that unlike a formal peer-review, you don't generate a list of defects for the programmer to go back and address--the reviewer has the ability to make direct changes. You have open communication. In fact, a lot of corporate entities could learn from the openness of communication. The development mailing lists are excellent tools, and it helps newbies get up to speed quicker. Granted there is a fair amount of noise on some of the lists, but the gains far outway the annoyance. In fact, the only thing that open source software may be able to learn from corporate life is the pre-written design/requirements documents. Many projects start without a specific direction in mind, or don't effectively communicate that direction, and consequently run into problems. What helps them, and what replaces the written documentation, is the open communication. In fact, the development lists is, many times, the 'white board' so to speek where the design is discussed before implementation. So your dev list archives become the replacement for your peer-reviewed documentation. In some ways it is better, because you get all the reasoning and ideas behind why they are going in a specific direction. In other ways it is not as good, because you have to fish through all the reasons and justifications to arrive at what is going on. All in all, the average developer has a better idea of the design and why we didn't use algorithm xyz that you thought (in theory) was so great.
Re:My software design process (Score:1)
Re:It depends on what is good enough (Score:1)
Meanwhile, back in TRW...
I used to be a games developer, now I'm a website developer. If your game crashes on the MD (CEO in the States?) two days before delivery date, you'd better forget about seeing your bed anytime soon. And if your website loses customer orders for as little as a hundred quid (tr: bucks), you've got a lot of explaining to do.
The simple fact is that badly or undesigned code is a pain. Ever tried debugging 1000 lines of somebody else's uncommented 8086 assembler knowing that the man with the money for this month's wages is calling back in half an hour?
(And BTW, that's another downside to working for a small company where nobody is organised - it probably means the boss isn't either, and he's got to get the cash coming in.)
Learning the hard way. (Score:1)
> What kind of design process do the successful free-software projects
> have (Linux, gcc, Apache, XFree86, etc)?
None, I think.
> In your experience, how often are design documents revisited after
> the project?
Less than what they should.
> Have design documents helped in technology transfers (that is, have
> they been more helpful than the source code alone)?
Yes.
> Are engineers good at writing design documents?
No.
> Have you been able to read design documents written by other
> engineers?
Yes.
> Have old design documents been kept up to date with the changes in
> the implementation?
Yes.
> Has the quality of your products been better because of design
> documentation?"
Yes.
I think that I learned this "the hard way". At my first programming
job there was a "software supervisor" who forced all the
software-writing employees to submit design documents. Everybody
agreed that he was a pain in the ass. But everyone did his
duties. Design submitted, talked over with the guy, and software
written. 3 weeks, and that includes designing and building the
hardware, and learning the processor involved (8051).
Next they assigned me to a new project, in a different building. So
that software supervisor wasn't there to bug me. That project was
about of the same complexity, but it became a mess. Simply because we
didn't have a design document, we started adding features along the
road.
I think it pays to spend say 10% of the estimated development time
with your feet in the air thinking about the problem that is to be
solved. To give the bosses something tangible, you should write
something down in your design document.
Oh, another trap to watch out for: I've been reading some the specs
for the binary format for Excel sheets. Those are really well
designed. The specs that is. The actual implementation of what Excel
writes into a spreadsheet file is complete rubbish. So they have a
"design team" which did a reasonable job at designing the file format
and it turned out to be unusable for the guys actually implementing
the thing.
So you should not have a "design team" and then an "implementation
team". The implementors need a say in the design. Officially there is
a bell-curve in the number of developpers involved with a project. If
your project is small enough (say around 5 programmers), start the
design process with all programmers involved. Possibly you'll send
half the crew back to what they were doing during the next phase (say
if you have around 10 programmers), but have them in on the design.
If the project is even larger, you may have to design the project with
a small team, but then you should have a catching up session with the
new programmers once they get involved.
-- Roger.
Re:In all fairness..this is too big a question for (Score:1)
My job is to make my users happy!
They don't need to know about 3NF, relation databases, parent-child relationships, objects, inheritance or any other techno terms. I design what they want and do it using the best design techniques at my command. The only time I "counsel" them is when they propose logical contradictions. In fact, they have supplied some of the best ideas towards solving some problems.
OSS documentation, design, process (Score:1)
The projects i have worked with spend a great deal of energy on reviewing ideas. They are all quite egalitarian about the review process, and it mostly takes place on email lists. Large pieces of work tend to get adopted by small groups, who may work off-line and then publish the result. A good example of this is the evolution of apache's mod_jserv, housed at java.apache.org. Thus, requirements and high-level design are covered. The mail list bantering, change logs, and source comments are all that is needed to keep many OSS projects afloat. Of course, this is amongst some very experienced and dedicated code-grinders. It should, however, serve as a point of reference. Oh, and BTW: not all of these developers use English as their primary language, yet the projects are expressed in English.
Re:My favorite design process (Score:1)
I think it's best to make a rough sketch of your program first and only start filling in the details when you've coded the sketch (you can code a sketch you know, it won't work, but you'll learn from it anyway). This has the advantage that interfaces you code are usually pretty good...
just my 2 cents..
Re:Waste of time (Score:1)
1) Have you been programming for more than a few years?
2) You've programmed, but have you ever really designed and implemented software?
3) Have you ever had to work on a team?
4) Have you ever had to work on something someone else worked on over a year ago-- or better yet, something you did a year ago?
I'm assuming your answer to all of these is no, or maybe the answer to this question:
5) Are you insane, a masochist, or both?
...is yes.
If you would have LISTENED in some of those classes, if they were anything like mine, you might have heard somewhere along the line that 80% of the software dev cycle is MAINTENANCE.
That's important in at least 2 ways:
A) If you have a battle plan laid out before you even start laying down a single byte of code, it's much more likely that later, you won't have as many issues to resolve because you thought about it and kept yourself from screwing it up.
B) You don't remember as much as you think. You will look at things you've written not more than 6 months ago and not remember a damn thing about what you were thinking. Let alone trying to divine what someone else was thinking if you're inheriting someone else's work.
So even when I'm just 'coding for myself', I still load my code with comments and at least write a readme in the header of the program or in a text file somewhere.
Please, please, please... do us all a favor-- don't go work for Microsoft, go back your classes and LISTEN for goodness sake.
Design doesn't end (Score:1)
I'm a firm believer in the proof of concept prototype and the investigative coding but I also feel that you need to go back and make the design document reflect the final outcome. Remember, this won't be a release one project forever and, god forbid you should move on to a new project, they're still going to come to you for the sticky problem that they can't figure out and it's nice to have a quick refresher doc available when all eyes are on you. Besides, if the doc is complete enough, they won't have to!
P.S. Don't skip Step 2 8^)
It helps me think ! (Score:1)
However, I still insist on creating them - because they help me think before writing code. I happen to need all the help I can get when coding, and writing down what I intend to build before building it works best for me (peer reviews being a close second). If you and your team are all capable of working without these crutches - good for you ! But most of the code I have seen over the years would clearly have benefited from a bit of thought up-front - and I would assert that it is easier to deal with poor quality code than with a poor quality conceptual design if you are the poor sod who is trying to fix a bug long after the original developer left...
The purpose of design is not to keep management happy - most managers wouldn't know a good design from a hole in the ground, and the others haven't got time to read and understand a useful design document. The purpose is to make the developers think before they code.
Oh, and Steve McConnell has said all this far more clearly than I could...
Agreed (Score:1)
I hate it, personally. I'm a very linear thinker, and I don't like just throwing stuff at the wall and seeing what sticks. But the business reality is that for some people you don't really have the option to spend gobs of time on design.
I should say here that my client is quite reasonable when it comes time to fix things that might have been avoided had there been time to do proper planning up front -- so they're at least willing to admit that at least part of the problem is their own impatience.
A Perspective (Score:1)
Where I am now, there's almost no formal documentation. My current project is going much better than any of the 'documented' ones, even though it's approximately 40 times bigger. How? Let me tell you a tale...
Good design documentation is focused. It serves a particular purpose. The person writing it knows what they're writing, and the person reading it know what they're looking for. If the prescribed section headings don't entirely fit, then they'll be thrown out and better headings will be used. Nothing gets in the way of the outcome that the document is being written to satisfy. Nothing.
Most design documentation, however, is done by bored programmers who would rather be coding. It's done to a template they didn't design, with no knowledge of why. It's done too early, by the wrong people, for the wrong reasons.
Fundamentally, there's a difference between documentation as a Tool, and as a Chore.
If the people aren't excited about writing the spec, then don't do it. Hmmm. 'Excited' might not be the right word. Ah, I know. 'Frantic'. That's a much better word. You should be frantic to make sure the project's not going to fail because you missed something. That's the reason for a spec, and don't ever forget it.
Management likes specs because it gives them that 'warm inner glow' that (a) work is happenning, (that they can understand) and (b) they've got something to cover their asses with later.
Orinoco's laws of Software Specifications.
1. The best spec is 1 page long, and a high-school student can read it. This goal is impossible, but worth aiming for.
2. End users can't write specs. If they could, they'd be system architects, and not order-entry clerks. Compiling a wish list of every user is like fulfilling the combined christmas list of a room full of 5-year olds. What they really need is new socks and underpants, but takes a deep knowledge of human nature to understand this.
3. Moreover, End users can't READ technical specs either. Don't show it to them, it will just cause confusion and worry. This causes more documentation. An endless cycle.
4. Management can't read specs either, though they pretend to and like to have a copy. Pander to this need, otherwise they will make your life a living hell.
5. The only people, therefore, who read and will understand the spec are your co-workers and yourself. So keep them in mind. If there's nobody on the project but you, you're only writing for yourself.
6. The full functional spec is only ever used during a billing dispute. If you've hit that point, you've failed anyway.
7. If you haven't already done it at least once, it shouldn't go into the spec. This implies that the spec should be written after the prototype.
8. Once work starts, a spec should be constantly evolving, but not too much. New things should trickle in by everyone's agreement. Major changes are cause enough for stopping and asking "Why wasn't it in there in the first place!" because there's bound to be more. An unchanging spec is dead and useless.
9. Writing code is inherently creative. Any creative endeavor is unpredictable. Never forget that.
10. Specs are like Backups. No-one understands the value of a good spec. until a project has crashed and burned around them for lack of one.
Far better than specs are what I call 'principle documents'. Something nice to nail to the PM's door. Nothing specific, just all the important generalities like "Nothing should take more than 3 seconds to process" for a help-desk system, or "Easy learning curve" for a system used only rarely.
And lastly, all specs should contain some elements of process re-engineering. A modern buzzword which boils down to "Why the hell are you doing it THAT way?!?!"
A good project is just like a good hack. Half of it is social engineering.
Hmm. As usual I spouted off and this is far too long. Apologies.
Re:Documentation stuff (Score:1)
If the programs are completely independent and will never be changed, I'd agree.
Of course in the real world, I don't plan on writing the same code twice (so the programs would use a common library) and programs are constantly changing to adapt to the changing business around them (so someone else may need to maintain them later).
Writing code is easy, but doesn't pay so well. Solving business problems is hard, but is compensated accordingly.
Joe
Design docs (Score:1)
architect, configuration manager, and sysadmin, on mainframes, pcs, *and* UNIX for coming
up on 19 years. I've worked for companies from 6 people, including the boss and his wife, to Ameritech.
You wrote:
> The documents should describe all aspects of the coding phase that is about to begin.
Define "describe all aspects". I have worked from design docs that I think Boing did, in the
early eighties, that were pseudocode, and I have worked from vaguely-worded memos, and
I have worked from verbal descriptions. Personally, when I do it *right*, I prefer flow charts.
So, just how detailed will these documents *be*? If they're down to the pseudocode level,
they'll take as long as the coding phase.
> Then the documents are peer-reviewed, polished and submitted under document control.
Define peer review. Most places I've worked, *if* they had peer review, it was dumped on
someone already overworked, who then maybe skimmed it, and said, "looks fine to me".
> Questions:
> In your experience, how often are design documents revisited after the project?
Never, except for *possibly* some poor programmer trying to figure out what this thing *does*,
so they can fix what's broken, or enhance it, years after everyone else involved is long gone.
> Have design documents helped in technology transfers (that is, have they been more helpful than the
> source code alone)?
That depends on the level of the design documents. You might consider that they define what has to
go in, what has to come out, what *must* be stored (not what they *think* they want to store, for
some possibility that they might look at it in the future), and conditions *must* be looked for,
along with some style standards, and a *real* architecture, appropriate to the project (not
ridiculously overengineered, nor kludged together as they go), and leave the lower level details to
the folks actually doing the work.
> Are engineers good at writing design documents?
Few (I'm an exception). Unfortunately, competant tech writers are few and far between, and
even more rarely *hired* by management.
> Have you been able to read design documents written by other engineers?
Some.
> Have old design documents been kept up to date with the changes in the implementation?
Never. Ever. Not in nonprofits, not in colleges, not in mortgage co's....
> Has the quality of your products been better because of design documentation?
Depended on the documentation. Note that 80% of the time, btw, that documentation is the
*last* thing done.... (B-{)}
mark roth-whitworth
Use cases, iterations, top-down/bottom-up (Score:1)
Basically, you script out how a user interacts with the system and try to cover all standard and anomalous cases.
If the system is large, it is usually productive to phase the project in terms of iterations, as opposed to doing -all- the design up front a-la the waterfall model. This is because a tremendous amount of discovery happens during the implementation process, no matter how thorough you might have thought you were in the design process. Both the customer and the implementer will end up discovering new and changed requirements.
By doing a top-to-bottom iteration- a reusable prototype so to speak, you and the customer can get a better sense of what you will encounter during the rest of the project. You will also have a preliminary unit effort measurement with which you can guage the rest of the project by.
I recommend putting the use cases in a database that preserves revisions, and composing the design docs on the fly. This is because you will want to view them from multiple points of view, such the users point of view, a workflow point of view, or a system component point of view. Databases are much more amenable to this than documents are.
On the other side of the coin are infrastructure type projects where the users are higher level components. The primary design doc for this kind of project is an API spec. With languages like Java or C++, you can deliver this in code, that both the users and implementers can compile against. You can write stubs and drivers that permit both the implementer and the user to work independently. Code is the language of choice for 'design docs' targeted at technical people. If you cannot express a design clearly in a layer of code, the language and API set your are basing your project on may be either at too high or too low a level for what you are trying to do.
IMHO, a chunk of infrastructure is best designed by a single good architect and reviewed by peers. This ensures some degree of consistency in the outcome. If an infrastructure project is over-divided among several architects, the 'whole' may end up being a suboptimal compromise.
I have personally experienced such a situation. Because many designers wanted to have a say in the project, there was a lot of political horse trading over who would implement what capability in their part of the system. The end result was an execution path that had way too many interface layers and representation conversions to be efficient. There were good arguments for each individual design, but since there was never 'proof' of the superiority of any the approaches, management forced a compromise rather than a choice.
I think that a group needs to elect a leader for a project and then submit themselves to the choices that this leader makes. The leader should not compromise architecture in order to appease different opinions in the group.
Well, that's my 2 cents. I hope it helps.
Mike
Re:My favorite design process (Score:1)
good release. Always plan to throw one away.
Proprietary != Open Source (Score:1)
With free software and open-source projects, people coming on board the project have probably already reviewed the source code in detail. They've had it for awhile, and probably are joining your team because they've begun tinkering with it on their own.
With a proprietary project, new developers have never seen the code before and have no idea how it might work, because it's proprietary. Therefore you need to spend resources helping them learn about it, and documents are one way.
With an open-source project, all of the users can peer through the code and search out the bugs when something doesn't work. This in my opinion is the biggest "10x" advantage that open-source projects have. Also, a saying I like is that "every problem has an obvious solution, if enough eyes see it". A bug that just mystifies you will be totally obvious to one of your users.
With a proprietary project, your users cannot read the code, and only you and the people on your team can debug it. Therefore you must devote time and energy toward tracking down bugs--to make up for not having hundreds or thousands of eager users doing that for you. And the bugs may not be obvious to you at first (not enough eyes) so you have to have additional procedures to methodically track them down.
Thus a properietary project needs peer-reviews, code-walkthroughs, and so forth.
In a nutshell, all that "software engineering institute" advice really is valuable, and you really should do it. Though people here will tell you that they have almost no procedure and things seem to work--it's likely they're talking about either one or two person projects, or else about open-source projects.
In a proprietary software environment, you really do need to do a few things by the book.
Of course if those documents aren't reliably maintained, or there isn't a way to mark them as "out of date", or whatever, then they may not be very much good for knowledge transfer; they should still be of some "design review" benefit though.
My biggest problem with this is...
WHY ON EARTH ARE THEY MSWORD FILES?! For gods sake, put those things in HTML and post them on your internal web. They're much more accessible that way.
Re:My favorite design process (Score:1)
Rational Unified Process (Score:1)
Try the Rational Unified Process
It's about the best of the current processes.
(Even though it is a Rational product
Has a lot of recommendation for documentation and
process cycles
(BTW, I'm a Java consultant and I use RUP)
Asmo
Coding from design documents... (Score:1)
Having a plan in the form of some written specs is very valuable when embarking on a coding project. I was managing a decent sized project some years ago and we went through the process of a preliminary design followed by a review, a critical design, again, followed by a review, etc. If this sounds too rigid, well, we did it to satisfy the people who were paying the bills (we were contractors to the Army). Since our system had to interact with flight critical equipment, we pretty much had to do it their way. We actually got off pretty easy. If we had had to adopt their methodologies in toto we would have had to hire a boatload of subcontractors.
For many of the people on the team, it was the first time they'd had to sit down and actually plan what they were going to do before they started doing it. It was painful for these folks but, you know what, the hardware and software that we produced worked the first time. When additional software modules were introduced for testing... they worked the first time. I suppose that much of that might have happened without the highly structured design work that was done up front. But I don't think that the coding and testing phases would have gone anywhere nearly as smoothly if we hadn't done it this way. We would have spent way too much time rewriting code that didn't allow additional modules to be added later. Our data structures would have been modified too many times and adding each additional function would have been a separate nightmare.
Maybe I was lucky and had great programmers (I'd like to think that I was able to put together a crack team.)
As another example, take a look at the way the basic building blocks of the internet came to be. I don't think the IETF documents for the various protocols were written after the fact.
An interesting study was done some years ago. I think it was at McGill Univ. Teams of engineering students were assembled to build the same thing. One team built the project and wrote the documentation and the user manual afterward; the other team did all the documentation first. The team that did the documentation beforehand produced a product that worked better and more closely met the design goals. Not proof that this is the best way; just another example showing that it seems to work better when you do it that way.
Let's face it, programmers hate doing documentation. Just look at the spotty documentation for some important areas of Linux. The manpages and the info pages are often out of sync (why must this be?). The LDP project needs to be better supported and, dammit, the developers need to be major contributors. Maybe they shouldn't write the prose (they're often much, much better at C/C++ than English) but the technical input needs to be there. Pass the documentation back and forth a couple of times until it's correct and is readable by people of different levels of UNIX experience. This is the stuff that the new users are told to read in order to get Linux to work. If the documentation sucks, then that's the impression they're going to have of Linux as a whole. If Linux works but they can't figure out how then it's all going to look like Black Magic and its adoption will slow or decline. Do any developers out there look for assistance from the user community to help write the documentation, manpages, etc.
I realize that this got slightly off the original topic but I feel that the two go hand-in-hand. Now that I got that off my chest...
Re:That was very helpful (Score:1)
Ah, youth!
I can remember having that attitude when I started programming.
BZZZZZZTTT! Wrong!
I have many old floppies with code from those days on them that I would just as soon rewrite from scratch than attempt to figure how they worked. I'm sure some of these old programs might be worth modifying to be useful again or contain some code snippets that could be re-used. But without the comments and accompanying documentation, it just isn't worth the time anymore to have to put on the ``code archeologist's'' hat in order to find the useful stuff.
Go ahead and take that attitude now; you'll come to regret it when you get older. Or, perhaps, when you take that attitude into a job interview.
no easy answers, sorry (Score:1)
btw, your questions seem to indicate that you already expect a certain answer. why do you do that?
Re:My software design process (Score:1)
That process will give you... unpredictable... results. Are you prepared to cope with the following?
1). Do you really want to write a library function to see if your source code is infested with the minions of Nyogtha?
2). Are your users sufficienly trained to cope with the non-Euclidean madness that will appear to form somewhere above their keyboards?
3). You won't be able to stop gazing into the Shining Trapezohedron.
4). Thursday morning staff meetings will now be used to contemplate the enactments of ruin.
You should weigh all of the pros and cons involved before taking a step this drastic.
->Dan
Re:In all fairness..this is too big a question for (Score:1)
Absolutely correct. This boils down to the difference between being an engineer and being a cowboy.
The products of engineering are design documents. ChemicaL engineers produce designs for chemical plants, and then construct, or supervise the construction of, those plants according to the designs. Software engineers produce designs for software, and then construct, or supervise the construction of, that software according to the designs. If that is not what you are doing then you are not a software engineer.
Re:Docs to read (Score:1)
Anyone who cannot write a good design document is not an engineer. A programmer, perhaps, but never an engineer.
Re:Hmmm (Score:1)
One important thing that I got from the book is that there is no "one true way" of software development. Each project has its own nuances that may or may not warrant a specific approach. As a contractor, I have now gotten to use almost all of the major software life cycles.
Something to also consider is that the team itself will work differently under different life cycles. I personally have started to favor the "jelly roll" model. It's directed chaos can be adapted to many situations and its high output and use of numerous prototypes keeps both management/users and developers interested. However, there are some teams where it will break down rapidly or become too much work to keep it from breaking down.
Poor communication and/or poor team dynamic can be overcome, but it's often easier to choose a more classic life cycle that requires more stringent design and documentation. Some programmers really can't seem to produce quality stuff unless you've micromanaged their portion down into bite-sized chunks.
P.S.: In documentating a design, never underestimate the power of use-cases. They are both useful in making sure the design is complete/robust and imparting the ideas onto the other developers in the team. When you are presenting a "well understood" idea to a team of competent developers and every one of them goes, "Oh, I get it now." when they turn to the use-cases then you'll know what I'm talking about.
P.P.S.: Never overestimate the value of whiteboarding. If someone doesn't document it for review it's as good as useless. As an experiment, doodle a design on the whiteboard for you team. Then, have each one of them go back to their offices and put together a design document. They will be extremely different in ways that will surprise and possibly shock you.
Start small, grow fast (Score:1)
I have not seen any of such documents accompanying any of the most popular open source applications. I wonder if there is something like that for stuff like GCC, Linux, Emacs, WM, XWindows, Apache and the like. As I said, I have yet to see it. Umm, at most I could mention the GNU Standards document, but that does not document an architecture or anything like that.
While some applications (GNOME comes to mind) do have some sort of Manifesto describing the architecture behind them, I believe that is not really important in the development of open source applications.
Most open source applications begin because the original author has a problem and needs to get it fixed. They usually start with something small, not very ambitious... just a little set of hacks to solve the problems the authors have. Over the course of time others will play with the code and begin to improve it and improve it and improve it. And suddenly the set of patches and hacks become something like Apache or Linux.
I believe `Start small, grow fast' would be the best way to go in open source and, in some situations, closed source applications.
The original author usually does not have a clear idea on how he'll implement all of the things he knows he will have to. As ESR points out in C&B, he will usually realize of new solutions to the problem only after he has implemented things.
The time spent documenting the architecture behind the applications is time not spent actually coding them. The architecture is constantly changing as other persons are always suggesting new ideas and sending code back to the original author. The latter has to be willing to throw lots of code away and begin again continuously. And I don't think stoping to write documents explaining the framework will really help.
I'm not saying the design process should be taken lightly; it is very important and can really save time. But once the author knows how to do the thing, he will just go and code it, not document it.
On the other hand, most open source applications have a developers mailing list behind them. It is not uncommon to see discussion about the framework on the lists. I believe the messages passing through it can play the same role of the design documentation you mention.
What is often important to provide is documentation on how the application *currently* works, documentation on the latest implementation. For example, you can go and find documents stating how to create Apache modules: explaining the structs and the functions you can use to get things done. But their purpose is not to document how Apache should be built; they are just provided to help persons interested in using Apache (creating a module). And they are written after the application is written, not before. They follow the implementation, not otherwise.
Alejo.
Re:Start small, grow fast (Score:1)
Alejo.
Re:Well... (Score:1)
Ironically, my company sent me to a software design class two months earlier where I learned about specifications, design documents, etc. When I came back from the class I told my manager we should implement something similar to what I learned in the class THEY SENT ME TOO. He promptly stated that all of that stuff was a waste of time. So instead I'm wasting time trying to figure out how in the hell this damn program is supposed to work!
Oh yeah, I'm also a Java & C/C++ Solaris developer and looking to get out of here.
Re:In all fairness..this is too big a question for (Score:1)
implementation details be documented by the individual coders as comments within the actual code!
The only thing I would add to this is that, at the design level, I find way too many people get WAY too into the details of a project. As the post to which I am responding said, algorithms first, details later. You never know exactly how you're going to need to fit parts X, Y, and Z together once part AA rears its ugly head. It's much more efficient to expect and plan for an evolution of model ahead of time than to expect to be able to plan for every contingency.
--
blue
Re:don't overdo it... (Score:2)
Design takes about 50%.
And implementation takes about 60%.
These are not performed concurrently, which is why software is always late. Except when you leave out the design phase... But we all know what kind of software that spawns.
Hmmm (Score:3)
I've personally not worked on open source projects (yet) but I imagine that they are vastly different than any commercial effort. Seems to me that managing gratis developers is like herding cats - if you try to control them, they'll simply leave.
But, I would strongly recommend Steve McConnell's book on Rapid Development, and Code Complete while you're at it.
The RD book - well, eat it. Read it cover to cover twice, and with that knowledge in your head, use what fits your project and developers.
Your people may like to do thorough design up front, and follow the traditional 'waterfall' process, but that doesn't stand up well to changing specs.
Incremental development seems to work well where I work. We have a small team, and in-house users, so feedback and even design changes can happen pretty quickly..
You'll need to look at the risks your project is facing as well as a number of other factors - i.e. do you subcontract, buy COTS stuff, use strict CM and are you subject to stringent V&V?? Who are your users, how skilled are your people? Look where you fall on the Capability Maturity Model (1.1 release) hierarchy and how you rank per ISO 9000-3. If nothing else, you'll get some ideas.
As you can guess, there's a huge number of variables that go into defining a successful process. The Software Engineering Institute at Carnegie-Mellon University [cmu.edu] may prove helpful, but I would recommend the aforementioned book (RD) first.
The lesson they need to learn.... (Score:2)
Maybe I just need to get some sleep
Re:In all fairness..this is too big a question for (Score:1)
In general, the answer I like to use in bracked in these types of situations is "whatever works for you." How you and your team approach a problem will decide how you solve it. Forcing youself to an artifical system that no one understands WHY you are doing it will get you nowhere fast.
I know programmers that like to just hack stuff out just to get it out, even if they have to redo it all later. Some like everything pre-modeled and conceptualized to the nth degree. Most do something in between.
My personal preferences includes something I call the "metaphor" system. Whatever I do, whatever I try to come up with an accurate metaphorial equivalent. During the lifetime of the project I create whatever documents are required to demostrate the functioning of the software. The advantages to this are:
= Both technical and non-technical people can read your documentation and get a clear understanding what is going on based on their respective levels of understanding.
= The documentation and code should if correct make clear rashional sense.
= The metaphoral system can be used to give you a clear understanding of what tools you use in order to accomplish your goals most effectively, for example if you metaphors include references to finite physical objects, then a OO design might make sense.
= In my experience, the metaphoral system can lead to more effective and innovative designs, and eases troubleshooting.
With a large group of people with different skill levels, a metaphoral system is always the way I go.
designs documents? (Score:1)
Most of the open source has man pages, info on a web site, a README, and then the source code. Being someone who has developed a few products both corporate and open source, design documents are not always done first. Sometimes the application comes first and then the design.
In the open source apps I have developed, there were 'idea' documents. Basically a few sheets of paper that had what I wanted it to do, and what I wanted it to look like on it. Other than that there was nothing that I'd call an actual document.
In the workplace however there have been cases where there have been design documents, and there have not been. In the case that there is a design, it always seems to be a changing design. Either the system cannot do what the design says, or after an initial implementation the design is changed. Sometimes the design document is done as the code is written. This is a working design.
On a few occasions there was no design. My notes on a whiteboard, and the the code. The white board gets errased, and so does the design.
This is not to say that designes are useless, they are good to have, and theu do make writing programs easier, if they are well thought out.
The thing to remember is that just because you have a design does not mean that you should stick to the design like it was carved in stone. The design should be flexable and changeing, until the applicatin is complete. Also a good design makes redesigning a system later a bit easier. You may write it in Java this year, but in 2 years there may be a new language. You will want your design in those 2 or 3 years, especially if your application does Financial Calculations, like an account package.
In the system I am currently working on, we do not have the design documents, so we are looking at the code and reverse engineering it. It does ledger updates, and number tables updates. There are literally thousands of programs that make up this system. It is a real pain. Right aboutnow I wish I had the designes from 20 years ago.
Re:Documentation stuff (Score:1)
Re:Waste of time (Score:1)
Re:Design Documents. (Score:1)
I am currently having a small debate with my boss over whether to have any design documentation or peer review at all. It's small because she just tells me what she thinks, and won't listen to anything else. She even publically humiliated me last week because I wrote requirements and design documentation for a tool I wrote some months back. She is of the belief that getting a prototype working and shipped is more important than getting something which works. (Of course, we also had a big shouting match about how to round numbers up to the next highest multiple of eight. She got some expression from some Microsoft sample program which didn't work, but we had to do the calculation for all 8 possible remainders before she believed me.) Of course this all means I will be looking for a new job in the next few weeks.
So, I in my spare time I am working on a program to do the very prosaic job of backing up a file system to a CDROM burner, using mkisofs and cdrecord. Since it's my program I did it my way. I started with a requirements document, did a detailed design, and only then started coding. I started with RCS, but now I'm using CVS. It seems to have worked pretty well, but I've had to do some backtracking to fix some technical details. Some of the stuff I wanted to implement turns out to be too time consuming to do with the 5 hours/wk I have available. I believe the process helped me a great deal.
I also believe that requirements documentation is even more important than the design document. The design document tells you how to solve a problem, but the requirements documentation tells you what the problem is. Of course, both of them are in some sense conjectures until you get to the end of the project. You don't often know what the problem is until you have tried to state and then solve it a few times. But if you don't try to state the problem in natural language terms, you will never succeed in solving it. Just my 0.02 Euros.
(Anybody need a CS Ph.D. with 15 years of experience who both designs and codes? Boston area, or my DSL? No Win, W2K or NT please. Resume on request.)
Re:In all fairness..this is too big a question for (Score:1)
To try to draw out the EE analogy, a CPU designer undoubtedly wastes many transistors in order to get some other sort of efficiency, say regularity, or to use something like a standard register definition. These other efficiencies might be manufacturing or design efficiencies which don't occur in the actual device, but which represent a cost savings overall. (I don't know if this is true, and I would be happy to be told I am wrong.)
Now that large scale computational resources are almost freely available, the environment has all changed. It's now possible and probably necessary to design large systems, each of whose components are perhaps not the most efficient possible, but which fit together nicely, and which work, whatever "work" means.
Re:Design Documents. (Score:1)
My 2c (Score:1)
I have a couple of things to clarify/add:
- Assume you will throw your first one away. I think I got that from the Camel book, but it has proven very correct in the real world. If you are doing something that has not been done before or is not substantially similar, you will get it wrong the first time. This is a Good Thing. The first try (sometimes called a prototype, or the alpha version, or whatever; pick a label) will teach you a lot about what it is you're trying to do. You will discover holes in the requirements. You will discover that what seemed easy is actually hard, and vice versa. You can then take this newly gained experience and start again. A caveat: Don't try to hack bits together to make the first try work. Scavenge the good bits and write the thing again, or you'll end up with an unmaintainable kludge that doesn't work properly and is difficult to debug. Clean code is good code.
- Don't adhere to one design methodology all the time. Just as sometimes it's good to use Perl and other times you need C or assembler or whatever, you need to choose the correct tools for the job. Informal methods may work well for you, or you may prefer a structured approach. Project size will affect your decision. I generally use an informal approach on small to medium projects with 1 or 2 system architects who design
Ultimately, experience is the best teacher. Learn how to run a project by participating in one. Note what works well and what doesn't. Constantly review what you've learned (albeit subconsciously). Everyone has their own style and their own preferences. Learn from others and develop your own. You'll know you've got it right when it works well.
Justin
I didn't use formal documents (Score:2)
You know this, but I'll just go ahead and say it anyway: Unless you're writing "Hello world" you ALWAYS do a thorough design first. With my company, that meant everyone standing around a white board armed with markers, arguing about how to design it. Once we had our design up on the white board we would leave it there and divide it up into pieces and hand out the pieces.
Things were always informal with a small group, and we never had to worry about miscommunication. With a big team the plan you described sounds like a good one to me.
Of course, I think they should always start with a couple hours around a white board.
If you're serious (Score:1)
There are a multitude of books written on the subject of software design including a whole field of software engineering. The fact that the documents are written in Word but Word is not available on Linux seemed to be a major focus of your question. As many will tell you, the document format you use is of course irrelevant, your goal should be to insure that those documents have the required information so that not only will any future employees be able to understand what the author has done but the author himself will also be able to review them to re-focus himself. As anyone who has ever worked on a large software project will tell you, you WILL lose track of your original goal, you WILL get distracted by relatively minor issues. Hell it even happens on small projects.
I personally lean towards Object Oriented Analysis and Design (probably because a certain professor pounded it into my head). There is a whole series of books cropping up around the Unified Modeling Language which you may want to read if you are serious. It provides a way for you to represent you system because let's face it, words are not sufficient and MS Clip Art while better than some OOAD packages *cough* Paradigm Minus *cough* just won't do.
I'd learn the concepts before worrying about what packages are out there and what platforms they happen to run on. In the end it won't be nearly as important as you think.
Re:In all fairness..this is too big a question for (Score:1)
This is not an appropriate forum for this question. The idea that someone could convey how to implement good software design in a 100 word post is ludicrous. This is something that someone who is serious would have to invest a great deal of time into.
Oh and as far as algorithmic proofs, they may not be right for you but there are some projects where a simple "Uhhh geee well it looks like it should werk" won't cut it. Sorry that plane just crashed, I guess the logic in that algorithm was flawed....
Re:Well... (Score:1)
Unfortunately the problem faced when trying to impose these extreme restrictions is that the SQA people work with the coders. We all know based upon human nature that reports will get fudged and the idea that the SQA people can remain at all independent is laughable. In working closely with the coders, friendships are formed and exceptions are made.
Also if you actually wanted contact info it might help to provide a private email address. I doubt anyone would want to provide it on a publically accessible and very well known bulletin board.
Re:My favorite design process (Score:1)
Documentation (Score:1)
a) agree on the need for good documentation
b)...provided we don't have to spend too much time writing it.
From looking at KDE, it seems the requirements for a successful open source project are:
a) Version control of all software (CVS/PVCS/Clearcase whatever)
b) A good centralised mailing list/ UseNet group where all can comment and criticise
c) A bug progress tracking system
d) Documentation. In KDE, I don't think there are very tight design documents written at the start of the project. Standards have been agreed on and documented, but this appears to have risen organically during the project.
The design process appears to consist of a number of people making a sales pitch for what a program should do, and whether that feature appears or not depends on how much support it gets in the development team, or the single minded dedication of the proposer. A successful Open Source project seems to allow both engineers and programmers to contribute [ I'm using the term engineer in the sense of taking a more formal approach to software design than a programmer]
My software design process (Score:4)
This isn't the best way to design software, but it seems to be a common method.
--Shoeboy
Re:Well... (Score:1)
Reply with your contact information and I'll get it into the right hands.
This might help (Score:1)
Re:My favorite design process (Score:1)
Good engineers write good design docs (Score:1)
I don't know about open-source development, but what I've found -- even on 1-man projects -- is that the process of writing out a functional spec of design is valuable because it forces me (or my team) to think about the design issues early. That's the real value.
> In your experience, how often are > design documents revisited after the
> project?
Rarely. The main value is in the thinking you have to do in order to produce the document.
> Have design documents helped in > technology transfers (that
> is, have they been more helpful than the > source code alone)?
You mean to other engineers? Definitely. Assuming the document is written well, and with care, and is read carefully.
> Are engineers good at writing design documents?
The absolute best engineers I've worked with *all* produce excellent design documents.
> Have you been able to read design > documents written by other
> engineers?
Yes. I think the best approach is to do brainstorming and then record the important points. Everyone gets to do some writing, and everyone reads what everyone else writes. The brainstorming and documentation can be achieved simultaneously in email, (if the engineers are comfortable with this); or you can brainstorm in front of a whiteboard and write stuff up later.
> Have old design documents been kept up to > date > with the changes in the implementation?
Usually not. But on the other hand, if the original design is good, the distance between it and the implementation will be small.
> Has the quality of your products been better > because of design documentation?
Yes. If I skip design and start hacking, (always tempting), I find that I suffer for discovering important design issues too late.
All that said, the documentation effort has to come from the engineers. If imposed from above it won't work.
Re:Design Documents. (Score:1)
(I was a C/C++, mostly UNIX, developer for 9 years, before switching to UNIX Administration).
-- Keith Moore
But there's only one Jim Carey (Score:1)
One of the biggest problems is people solving non-existant problems 'cause they're interesting. Usually writing their own middleware, reinventing STL or some other junk. Control is necessary.
From the SQA side of things... (Score:2)
Developers don't like doing design documents. I don't blame them for not wanting to write the specs until after they're coded in. But it's important. They keep everyone on the same page, and prevent developers from putting in the app what they think should be in it.
Design documents allow more than one person to code an application at the same time. The smaller the project, the less "formal" the docs need to be, but they need to be there. Even if the app is just a simple text editor, if one developer is thinking "vi", and another "wordpad", you're going to have problems.
A story: Two days after a major application is released, customers are screaming that the autosave feature is overwriting unrelated files. Someone traces the cause down to a combination of autosave and the autospell. PHB calls the tester into his office and demands to know why the bug wasn't found previously. Tester explains that the autospell was never in the specs, and had no idea it existed, and so could not test it. The PHB, the idiot that he is, fires the tester, and gives a bonus to the developer that wrote the autospell.
Re:You'd better start thinking... (Score:1)
Re:A "must read" for this subject area.... (Score:1)
Most importantly, because it's so old, it talks about monolithic projects in which the developers and managers have no contact with the end-users. People expect much more these days.
However, TMMM is useful for saying to does-not-compute conservative managers 'Look, we're not doing this, and it says in TMMM that we should.'.
GIFMO (Score:1)
Beyond that, you're right: in most cases following such an anal retentive method is likely to prove to be an exercise in GIFMO (Garbage In, Flying Monkeys Out (or "Gerbals In", if you prefer)).
For most software projects, the best approach is to throw down some code that lets the users get some benefit now, and then fall into a cycle of revisions that follows the constraint that each revision sucks less than the previous.
Admittedly, that scheme isn't the best for special cases such as launch control or life support software -- the revision cycle in such cases would be prohibitively expensive. But notice that even the most rigorous software engineering discipline is no guarantee here either -- think Arianne V. I'll concede that SE can hedge your bets, but overall the result is only as reliable as its weakest link, and the cost of attempting "no weak links" is prohibitive for most development projects.
Also, the throw-down-and-revise strategy is probably not apropos for commercial hypeware, since the goals there are not what most of us would put under the heading of "Good Software".
But for the workaday world, I think the throw-down-and-revise strategy is best. Here are some reasons -
1) I've worked at corporations where most of the day-to-day software that employees actually used to get their jobs done was unauthorized, throw-down solutions provided by some engineer or low-level IS techie at the departmental level.
2) Similarly, I've seen where the most effective big software solutions were simple accretions and evolutions of things arising as in (1). These often attained official status and accrued a support staff, simply because they were so darn useful.
3) But some very-high-level PHB always has a better idea, so ground-up custom solutions tend to be mandated from above. These may be purchased or developed in-house using "good methodology", but the infallible result is that they make the grunts less productive by being difficult to use and generally imposing an electronic bureaucracy (e.g., requires input of information that the intended users do not have, or refuses to divulge information that they desparately need).
All that to the side, let's look at what typically(?) happens when organizations do try the "documentation approach".
4) After some unbounded time in committee, the documents are released and the programmers go to work. Usually the software is already late by this point, at least from the p.o.v. of the people who need to use it.
5) While the programmers are busy at their work, the whole hierarchy of PHBs are off attending meetings of some Revision of the Week club that they've joined, so that the developers are flooded with a steady stream of non-negotiable, undocumented revisions to the official requirements.
6) The program is finally rolled out, but even ignoring (5) it is full of bugs (as all software is), and thus differs from the specs by yet another increment.
7) The program hits the real world as if it were a fan, and customer requests start flowing in. At this point, one of two things can happen: (a) the requests are ignored and the company looses personal productivity and goes broke, in which case no benefit was obtained from working from the design documents, or else (b) the customer requests are satisfied, which means that the software drifts even further from the design documents.
8) The changes in (5..7) are never systematically integrated into the documentation.
9) You end up with a wheelbarrow full of spaghetti code that does not match the design documents and may or may not match the users' requirements.
Which is exactly what you would have had if you had skipped the paperwork and followed an informal release-and-revise strategy to begin with, except that you spent more money and took longer to get there.
YMMV.
AMMCFOOMB.
...which is why I call it a GIFMO system.
Docs to read (Score:1)
Re:In all fairness..this is too big a question for (Score:1)
Re:In all fairness..this is too big a question for (Score:1)
> Im more than serious about designing things properly...
>as for the original poster, If you dont design you deserve what you get (and if you worked for me it would be the sack)
I didn't see that he said anything about not doing design; I interpreted the question as "Is it worth it to spend so much time writing it out in prose and polishing the form of the document?".
In my experience, the answer is "no". The documents are frequently copied-and-pasted to fit the form and meet the [the letter of] the document requirement, but rarely get updated.
And just because I don't write it in prose doesn't mean that I don't do design.
I agree with an earlier comment that diagrams on a whiteboard are likely to be far more useful than a formal design document.
Re:In all fairness..this is too big a question for (Score:1)
Users *do* know what they want - they're just too used to being *told* what they want, and then having to *ask* if they can do something. Computing and software have reached a point where it is almost unreasonable to say "sorry, we can't do that".
ask users what they want. get a clear picture. make sure everyone understands. !!!!Get it in writing!!!! commit only to what's in writing, and then do the work.
and never, ever take the attitude that "i'm the coder, you're the user. i know everything, you know diddly"
my $0.02
Re:In all fairness..this is too big a question for (Score:1)
as for the original poster, If you dont design
you deserve what you get (and if you worked for me it would be the sack)
Don't overdo the design docs (Score:2)
I've worked on dozens of s/w projects ranging from solo assignments to enormous multi-headed monsters involving hundreds of people doing coding and validation. I've noticed a method which seems to work fairly well for a variety of project sizes. Of course, YMMV.
1. Don't get tied up in writing a bunch of specs up-front. The spec-it-all-first method almost always results in sub-optimal implementations and reams of out-of-date documentation.
2. Get a handful of senior coders (and one or two customers of the product, if possible) in a room and have them architect a skeleton of the project on the whiteboard. Get a junior coder to take notes. These minutes are your first "design document".
3. Figure out how to partition all of the pieces from #2 into nice libraries and data structure definitions. Define the basic functionality of the libraries without specifying the exact interfaces (which will be wrong for the first two iterations).
4. Prototype and re-prototype the implementation, allowing the developers to work out the interfaces between themselves. Get customers to try out the prototypes.
5. Schedule a "get well" phase of the project where code is cleaned up and interfaces become mostly frozen. This is the ideal time to document the "final" interfaces, data structures, and algorithms. Hold peer reviews of code and documentation. Discipline is vital here if you want a documented design.
6. Productize.
7. Maintenance mode. If you can find a way to keep the documentation up to date from this point on, please let me know how you do it.
While any document which isn't auto-generated directly from the code will become out-of-date, getting most implementation details figured out by trial and error before writing an official document will make it more accurate with less maintenance overhead.
I've seen a lot of projects fail miserably when management spent too much time trying to spec everything up front. One group in my company has an offical flow which goes something like this:
1. Write an Market Requirements Document (MRD)
2. Review the MRD with a lot of people, many of whom have no idea what the market really needs but have a lot of opinions about how the MRD should look.
3. Revise the MRD based on the review.
4. Write an External Product Specification (EPS). This defines every feature which is to go into the product that is visible to the user (as well as how it is visible to the user).
5. Review the EPS with a lot of people.
6. Revise the EPS and get "buy-in" from all of the
"stakeholders".
7. Write an Internal Product Specification (IPS). This defines all internal modules and most of the data structures and algorithms. Note that usually this is published before any code has been written.
8. Review the IPS with a lot of people.
9. Revise the IPS until everybody is happy.
10. Start writing code (usually about 4 months after the process has started).
11. Find out that the customer has already started using another product from somebody who has followed the first method above.
12. If the other product is from another group in the same company, try to kill that product via political means.
13. Finish coding up something which is grossly inefficient and doesn't meet the customers' real needs, but that meets most or all of the items in the MRD, EPS, and IPS.
Ok, maybe some of that isn't in the offical methodology...
Finally, there are many factors besides design documentation which help to make or break a project (having talented people seems to be most important); so pick your battles if you're going up against management edict.
Design Documents + Software Engineering (Score:2)
Design Documents are just part of the whole theme of a well engineered software system. Much depends on the system being constructed and what it needs to interface with. I've written and used extremely detailed design documents (this BIT goes here, that BIT goes there) for a project where all the pieces were being implemented in different geographic locations. When the thing was assembled it even worked! On the other hand, I've been gratefull for a very brief, high-level 'how this fits into the big picture' design statement that described a report system that just sucked from zillions of different data sets that were previously defined elsewhere. So it depends on what the goals of the project are.
In 'open' software, I'd much rather have sections of documentation inline with the code that say things like, "At this point we know XXX and we're now going to YYYY." Tell me the high level story as it is less apt to rot over time compared to the minutia.
Finally, Design Documents that are used will be updated. DD that are just worthless trash from the outset, no one will care if they further get bit rot.
If you have not read Fred Brook's book, "The Mythical Man Month", now would be a good time.
Your question certainly is bigger than a ask
P.S. Revision control and MS Word documents? Doesn't the constant change of format of MS doc's cause you to worry that your documents will not be readable later in the products life? I'd be using "Plain Old Text" as ISO-Latin1 is not apt to go out of style anytime soon.
Requirements hell (Score:2)
The requirements document of the project I just spent 80 hours a week working on for the entire summer was a source of so much grief that I get queasy just thinking about it.
Basically, the customer handed us over 1000 pages of documentation and said, "This is the best requirement document that has ever been written. You shouldn't need to ask us anything about it, because it's so damn PERFECT!"
Needless to say, it was NOT perfect.
It looked all fine and dandy on paper or on design flowcharts, but actually trying to write compliant code was a nightmare and a half.
But wait! Here comes the frustrating part!
Every time we tried to talk to the customer about something we needed to implement that wasn't in the requirements document, the response would be...
"Read the requirements document!"
My favorite design process (Score:2)
Step 2: Type "rm *"
Step 3: Write the design document.
Seriously, I've been amazed what you can find out by simply trying to code something up. I find that I can avoid so many horrible errors by throwing up some code that I know will go to
Documentation value / Extreme Programming (Score:2)
Historical documents are useless eg those produced by various project phases, except for butt-covering in bureaucracies.
At the program level, document the interfaces to the programs and the expectations and assumptions, again quite concisely.
Some great reading on development is at c2.com/cgi/wiki?ExtremeProgrammingRoadmap.