Beginning Project Documentation? 314
mirthe_v writes "Hi, I'm working for a small webcompany (about 20 people), with ColdFusion programmers and designing staff. We all work on a bunch of projects (Internet, intranet, cd-roms, etc.) on the same time, with different people and different or no methodologies. There is an ever growing need for documentation, but we have no idea where to start."
mirthe_v continues: "I was just wondering how other people/companies keep track of their current and older projects.
Do you put stuff in a database, if so, what about all those diagrams and handwritten notes.
If not, do you store things in a folder per project, and how do you then stop documentation from getting lost and making sure people store things where they should.
"As I said, I don't know where to start, especially since the staff varies greatly in the need for documentation, technical background, experience with writing documentation and even different languages (English and Dutch).
"Please share all your thoughts and experiences. Cheers, Mirthe""
I've got the answer... (Score:3, Funny)
-Ben
Re:I've got the answer... (Score:2, Insightful)
http://www.twiki.org/
Comments lie. Code never lies. (Score:2, Interesting)
Write good code and pray.
Re:Comments lie. Code never lies. (Score:2, Insightful)
I'm new at my current job (3 months now), and am in the lucky situation that the guy who wrote all code I have to work with is sitting right next to me to explain it. But then still... it's a madhouse around here so he doesn't always have the time for it.
The least you can do, is document the reasoning behind what you coded (needs to be no more than 5% of the code you wrote), and let the code itself tell the rest of the story. Without that small fraction of documentation you will get lost, unless you've had the exact same training as the others who work on the project.
Re:If documentation is short, it's useless (Score:2)
Re:I've got the answer... (Score:2, Interesting)
total development time for this was 3 months and had 1 guy dedicated to developing it. We then made it so that everyone was "excused" from projects for one week where he/she went thru the code of a project that she worked on (or more) adding the comments to functions.
We have now over 1 GB in documentation and new employees grasp the coding principles VERY fast.
Anyway thought i put my 2 Cents
PS: we have 60 people working in our Company and only 20 coders. But the documentation applies to all including us in the IT Department.. and with the scripts its VERY easy...
//Vic--- from Finland
Re:I've got the answer... (Score:2, Insightful)
CowboyNeal-Flow (Score:1, Offtopic)
KDE (Score:5, Informative)
for which there's a great guide (crash course) [caldera.de]
that they encourage their writers to use.
m o n o l i n u x
java idea is really good... (Score:3, Informative)
maybe you could design something similar, and then use your new program to document itself!
Re:java idea is really good... (Score:2)
One teaches someone how to use a product, and is frequently written after it is done.
The other teaches a programming group how to code a project. (Interface definitions, conventions, hierarchy, etc...) This is supposed to be done before the first line of code is written. Most programmers don't write these, unless their manager makes them, and most managers don't see the use anymore than the typical programmer does.
And when the manager does make them write one, they usually do it as they go, writing the document, and coding side by side, rendering the document useless. Sigh.
Re:java idea is really good... (Score:2)
it defines all the function, pre-conditions, post-conditions, input, output, what it does... all in a highly readable and understandable form.
i'm not sure if that is what they want...
Re:java idea is really good... (Score:2)
100 pages of javadocs do little more to help one understand a project from a high level than no documentation at all. What most projects really, really need is a good high-level explanation, then some mid-level documentation, and *then* finally the nitty-gritty stuff that can be generated automatically. If you do XP (that's Extreme Programming, not the OS), then the testcases you use to test high-level functionality are pretty darned good documentation -- usually better than the code itself, because they only discuss the visible API, and less about how each object actually does its thing internally.
That all said, when it isn't asked to be something it ain't, javadoc kicks ass -- and its C++-based cousin, doxygen, is perhaps even sweeter (except, of course, for having to deal with C++).
doxygen (Score:2, Informative)
I like doxygen, myself.
Re:called javadoc (Score:2, Insightful)
Perl has it, too (Score:2)
http://twiki.org/ (Score:5, Informative)
http://twiki.org/
Re:http://twiki.org/ (Score:4, Insightful)
I'd give a working example of a wiki, but I'm worried about the Slashdot effect on it -- the guy might run up bandwidth costs (and he's not my enemy), and it might become too popular (which means too expensive to run, and it means that it would be edited too quickly and by too many people so that it would lose its small-community feel that Slashdot has also lost).
Re:http://twiki.org/ (Score:5, Funny)
Re:http://twiki.org/ (Score:3, Funny)
Does it have an audible announcement for notifications? Maybe something along the lines of, "BeeDeeBeeDeeBeeDee. Hey, Buck!"
Old Fashioned Notebooks (Score:4, Insightful)
I'm always suspect of documentation and record keeping that is 100% digital. If the notebooks are witnessed and microfilmed, there will never be a question on when the work was done.
If notes about how things are done are kept on a notebook like that, it is a lot easier to go back and figure out what was done and how it was implemented when it comes time to writing real documentation. This does not address the inprogress documentation, but you should not forget about the tracking aspects. (CVS is good, but it could be falsified easier than a notebook I would think).
Don't get me wrong, I hate doing this... but it is a good idea
DrArt
More is better than less! (Score:2, Interesting)
Re:More is better than less! (Score:2, Funny)
Document repositories... (Score:5, Interesting)
As far as organizational structure, base it on the development process you use. So in each phase of development, you have the documents needed for that phase seperate from everything else. If you have deliverable documents, put those in a directory and then have subdirectories for any supporting material.
That's a suggestion. I would highly recommend sitting down and formally structuring this. If everybody knows where the documents are, and where to put new documents as they generate them you'll have a lot easier time of things.
Re:Document repositories... (Score:2, Funny)
What is important with whatever documentation you keep is that you have it easily available for everyone. Everyone also needs to be able to easily get the most current copy and be able to submit new information into the current copy.
Sound like version control to anyone? Yup, just use whatever version control software you have on your project. Something that the developers already know how to use. Put the docs under the same project as the source code. When I cvs update (or whatever) to get the current code, I should also get the current doc.
Using CVS will not be the most efficient if you are using Word for your docs, since you have to store the whole binary each time. This isn't usually a problem, but the storage can get big quick. Also you won't get what changed each time since it is a binary. Text files clears this up.
Getting your business guys to use CVS is another whole can of worms, but at least all the developers can get the docs easily.
Re:Document repositories... (Score:2, Funny)
(yeah, its a stretch, I know.)
Start like writing started: Oral Tradition (Score:2)
Then refine this.
It's a fairly painless process, and you can do it in teams.
Of course, the big step will come when you need to get someone to update and maintain what you've created.
Sig: What Happened To The Censorware Project (censorware.org) [sethf.com]
Re:Start like writing started: Oral Tradition (Score:5, Funny)
Who wouldn't buy the support contract if it included a yearly visit from the master storyteller? By jove I believe I've just solved the "how to make money from GPL softare" problem...
:)
Re:Start like writing started: Oral Tradition (Score:2)
2. ????
3. Profit!
Re:Start like writing started: Oral Tradition (Score:3, Funny)
(He runneth for cover)
Re:Start like writing started: Oral Tradition (Score:2)
lab notebooks (Score:2, Interesting)
Documentation Consultants (Score:4, Informative)
Far too many companies approach tech writing as the "music department" of the company: Nice to have, but not really necessary. The problem is that the lack of attention given to quality documentation winds up costing the company later. If you don't know where to find such a consultant, you could always shoot off an e-mail to a highly qualified individual such as myself [mailto]. *smirk*
Welcome to the music department (Score:3, Insightful)
It is very hard to make development people see this. Part of it is a real antipathy to the whole chore of producing documentation, even if your role is just to being interviewed by a tech writer and answering a lot of "stupid" questions I put "stupid" in quotes because it's the really stupid questions that your customers most need answering.
To be fair, the technical communication profession itself is a big part of the problem. There are a lot of "technical writers" out there who think an English degree, a little computer experience, and maybe a quickie certification course is all they need. Or recycled engineers whose prose style would make Bulwer-Lytton blush. There are companies where most or all of the writers are like this. Engineers find them a pain to deal with, and the docs they produce are mostly useless. Small wonder so many engineers think of technical documentation as a lost cause.
Cynicism... (Score:2)
Re:Documentation Consultants (Score:3, Insightful)
This is true and it can cost them dearly. My wife's company recently did a review of their products and that included feedback from customers. It turns out that in the past six months three big customers are considering moving to a competitors products because of lack of documentation. The company previously had decided to outsource all documentation to India as they figured there had to be something written up but they did not see the benefit of quality documentation. Additionally, a good technical writer may have experience in other arenas such as interface design. No offense to the full time programmers here, but some of you folks create the most attrocious interfaces I have ever seen and could use a little direction when it comes to interface. It may make sense from a technical standpoint, but put a lousy interface in front of an end user and they can be totally lost.
The other issue is when companies try and cheap out on documentation and hire "writers" that have no experience or training in the documentation arena. What results is poorly managed and poorly documented projects that can cause more harm than good. My reccomendation is to find someone that has several years experience and preferably a masters degree in technical communication or better. Like most things, you generally get what you pay for, so don't cheap out.
Focus on making money (Score:2, Interesting)
Re:Focus on making money (Score:2)
You pay the piper now, and it takes less time and money, or you pay him later, and it takes a lot longer and costs a whole lot more money.
It seems to me that the "right people" (aka - process-oriented people) in the company you worked for before were likely more focused on process than on results. But there is a happy medium where the process leads to better results.
the only choices aren't quality or speed (Score:2)
The problem with your approach is that the bare minimum quality works in the short run, but it's very difficult to get out from under that hole once you create it.
Also, you assume that moving as quickly as possible is the polar opposite of moving forward with a process in place. My counter to that is that a good process can actually save you time, even in the short run.
I've been at places where everyone is in such a hurry to move fast, to get product out the door, that they had to do the same things over and over again because of duplication of effort, mistaken assumptions, and so on.
Hell, look inside 90% of the burnt-out hulks of once world-beating dot-com companies in the Valley and you'll hear the same tale over and over again - "we were in such a rush, nobody knew what was going on, when the senior developer left, we had to redo everything from scratch.." etc. This isn't just hearsay - I've seen it first-hand.
I'm not trying to argue with you, but just ponder that the choices might be less binary than you present them. There is a middle ground between complete failure and moving as quickly as possible with the bare minimum of quality.
A little extreme (Score:2)
Documentation wasn't invented by some lunatic who liked to file things. I have seen many projects and organizations who were failing because they lacked documentation. The key, I think, is to keep the ultimate goal in mind. Don't generate documentation simply because that's the way it is "supposed to be done". If it will help you, then do it. If it is interfering with actually building the product, then don't do it.
Re:Focus on making money (Score:2)
I work at a company that started small (dozen or so people) and grew to many hundreds over just a few years.
I can't tell you how much time we (as a company) have now wasted trying to go back and re-document everything that we have done. Sure the code we have is readable, but there is so much more than clean code to making a company work well.
So, document as you go. Keep it simple and straightforward in the beginning and grow into more advanced techniques/tools/etc.
And by the way, if anyone ever says "You may get people who want do things "the right way". Fire them QUICKLY!! "
Re:Focus on making money (Score:2)
I'm more than willing to make my code readable and documented inline with a quick README explaining usage and initial intent. It's been enough information for the hundred page books to be written later in about the same time as it would have taken initially (NOTE: don't know of anyone whos ever actually read the books, they always go for the readme and code -- but they exist now).
No documentation is bad, but so is a 'make work' project where the only benefit was an employee had work to do.
A
Larger companies get the project manager(s), a proposal, business case, technical writeup, lead contact, lead developer, and projects which never complete or don't actually meet the proposal.
My experience is heavily limited, but fly by the seat of you pants has done a much much better growing the company I work for than the plan everything and document when people sneezed method shrunk it shortly after. (35 people to 160 to ~200k [buyout by large company who turned over most staff with their methods in mind] back to 70 and another turn over to get those people out.
As a final note, if you code an interface between 2 components that isn't to the spec initally declared (but it has equal or greator functionality and both components work as advertised) just make the coded version the new spec and toss out the old one. This even works with intercompany projects till the 'do it the right way' person comes along.
Re:Focus on making money (Score:2)
What happens is that we never get the projects that would take us to the next level because we can't show the the skill set needed to handle the large project to a customer.
We are constantly bogged down maintaining poorly or undocumented web sites that work only by accident and fire fighting on servers that have not properly been maintained. Every server that we have has been hacked or infected by viruses at least once, some times multiple times. Our main web server is also where we keep credit card numbers and is also where we do development.
None of the code we have can be reused on the next project because it is crap.
Projects take 2-3 times longer than needed because there was no written specification of what the customer wanted - so when we show him something he says, nope - it's all wrong and has to be redone.
Our proposals to the customer are so vague that they come back and say that you were supposed to supply feature xyz as part of the project, and there is no way to argue the point.
The result is a company that doesn't learn from it's mistakes, and hasn't grown over the years, and can only handle one major customer at a time. Employees are hired with a minimal level of experience because that is all the company can afford - and leave as soon as they realize that they can do better elsewhere.
Management is always underbidding projects because they have no record of how long it actually takes to build such and such a feature. The result is many projects that lose money.
The fact is that best practices are known, and unless you make a real effort to follow them you will never be able to get to a point where you will really be able to prosper as both an individual and as a company.
Users or developers? (Score:4, Insightful)
A web application by nature should almost always be self explanatory. A help link or button should be available prominently on every page. The better you do this part, the less it costs to support your app.
Developer documentation for a web app also works well with HTML. Not only can you use comments extensively, you can link variables and functions from where they are used to their actual definition. A common way to structure HTML documentation is to have a frame with the left frame containing a tree of links, an index, and a search. I would use something like ht://dig [htdig.org] rather than a database to index your docs and allow searches.
I hate to say it but... (Score:3, Informative)
When I worked for Intel Software Testing (hence the nick) we used Visual Source Safe. It allowed us to store any kind of document in it (source, jpg, vsd, mpp, etc). With the check-in/check-out feature and version history it allowed us great flexibility and reduced wasted time figuring out who changed what.
I know it's a M$ product, but it did the job for us.
SoureForge (Score:2, Interesting)
Documentation does not live in a vaccuum. For every project, you want a certain set of communication tools: Revision Control, Mail lists, Documentation, Releases, etc. For my old company, I set up an example site of sourceforge running of my workstation. The idea was to set up a quick and easy project website. SoureForge (the project) had everything we needed. One of the nice things is the document erepositor portion. Since SF is by project, documentation is already per project. Etc, etc.
Php groupware also looks to be promising, as does phprojekt. I am not sure if there is an open source Cold Fusion based solution, but that may suit your needs better, since it sounds like you have some in house exp. with it.
Re:SoureForge (Score:3, Funny)
My documentation lives in a vacuum. My tech manuals live in a Hoover, and my user manuals live in a Sears Craftsman Shop Vac. I guess you could say my documentation system sucks
Thank you, I'll be here all week.
Documenation is not your only problem... (Score:4, Informative)
From the what you said with different people and different or no methodologies there is a lot that needs to be done. First off, standards:
- Literate Programing: What you write, should make sence. Variables such as a, b, c, thing, object, stuff, crap etc should not be allowed. A new programmer should be able to come into your shop, read the code and it should make sense.
- Commenting: All source should be commented. That doesn't mean that hlaf your source files should be comments, but anything that doesns't make sense by simply reading the code needs to be commented. At a bare minimum, the person who wrote the code should have there name in there
;)
- Source Control: Please tell me you have CVS or something like that setup. If not, set it up
- Testing: Test early and test often. When your done with that, test again.
;) You might want to investigate Extreme Programming. This will help quite a bit to launch good solid projects, on time.
- Standard Design Methods: It will help you out quite a bit to have standard design methodologies. Standard patterns to follow and ways of doing things will help you quite a bit.
Secondly, make sure you have a forum for you developers to work together. Setup an IRC channel or something, just make sure you have a place to chat and share experiances / bugsThose few things should get you started. They don't totally address the commenting issues, but I would say that is the least of your problems... I could be reading way to much between the lines. Just my two cents.
-ryanRe:Documenation is not your only problem... (Score:4, Insightful)
Maybe you mean something different here, but Literate Programming is a field, invented by Donald Knuth, that has little to do with what you are saying here.
There are many good reference, try this one [literateprogramming.com], for example.
AFAICT, the tradition of Literate Programming comes out of Mathematics where proofs are given as narrative, but the equations (programs by analogy) are terse.
For example, this example [literateprogramming.com], written by Knuth himself, contains many example of terse variable names like rx, h, t, l, etc.
thisIsAStupidVariableNameLikeJavaCodersUse (Score:2)
for (itemIterator=0; itemIterator < itemMax; itemIterator++)
itemPointer[itemIterator].isFlagged = 1;
... then I really don't want anything to do with you. There are good and valid places to use i, j, retval and friends. There's a rule of thumb about the length of a variable's name being proportional to the logarithm of the scope (measured, for instance, by the number of lines of code it can be accessed from).
--grendel drago
Re:thisIsAStupidVariableNameLikeJavaCodersUse (Score:2)
(define (next x) (+ x 1))
(define (prev x) (- x 1))
This, in my opinion, makes things like recursive functions easy to read and rather elegant. Such as:
(define (factorial n)
(if (= n 1)
1
(* n (factorial (prev n)))))
A Perl aside (Score:2)
Documentation is only useful as part of a process (Score:2, Insightful)
I poked around - weeding out the huge screenshots and photoshop files the designers had created - and tried to find out how the system worked; I found a folder called "architecture", and started to read. Funny thing was, most of the documents were more than 6 months old; several were more than a year old. Hardly any were written by current project members. There was no road map, no versioning. In short - the documentation - though plentiful - was badly out of date, often actively misleading, and totally pointless without the writer to explain the context and intent.
This is not an uncommon occurence; there are people with (parts of) the answer.
Check out "Agile Development" by Alastair Cockburn; he has very strong ideas on the topic of documentation and how to keep it relevant.
Next, buy "Code complete" by Steve McConnell, and make sure everyone in your development team reads it and understands it.
You might also want to check out the ideas behind Xtreme programming [xtremeprogramming.com] - it sounds to me like you're starting to find it impossible to maintain code written by other people with different coding styles and are hoping that documentation will save your bacon....it won't.
Most long-term projects rely on coding standards, a common vision and understanding of the architecture, effective communication and well-written code. Documentation is usually done because the customer insists on it, or the management team believe it's going to solve hand over problems. It rarely helps the development team to get to grips with what is really going on...
Where's the documentation for the linux kernel ?
Budget Yourself Time (Score:2)
Having a record of what you did to get to the point you're at is INVALUABLE in saving time later on: a process that takes 4 hours (including documentation) may take only 1 hour the second time, IF you have proper documentation, 3 hours if you don't.
Document what you are doing (Score:2, Insightful)
I suggest having templates for all the major areas you would need to have documented - or at least identify the things you want documented and make sure everyone does their part. Things like proposals, project charter, agendas, minutes, functional specs, design documents, project plans, change requests, test cases, test plans, etc.
Obviously, this all ties in with project management and your methodology of choice. If you don't go out and get a consultant to guide you, at least read a few books and get some ideas.
WikiWebs and/or Source Control (Score:3, Informative)
I've used, successfully, two systems. SourceControl is standard way of doing it. Basically, treat it like a shared filesystem. You have to be very careful about the structure or things tend to get lost. Also, it becomes very easy to make duplicate docs when one gets lost.
I found it easier to use Twiki [twiki.org] (which is a WikiWeb-like project). Twiki has built-in version tracking, can store any kind of information, including meta-data, and has pretty advanced search features. We've just started using it at my new company, but I really like it so far.
The only thing Twiki is lacking is proper authorization. Anyone can go in and write over my docs. Of course, it logs that change wit the user's name, so there are decent forensics. Still, I'd rather not have these hassles than be able to track the perpertrator down.
Re:WikiWebs and/or Source Control (Score:2)
Can you send me a link to the docs where they describe how to use fine grained access control?
Thanks,
Dave
Source code managment/Documentation managment (Score:2)
Documentation can be written in whatever format you want, but a standard which is compatible with your management systems change log tracker is a good idea.
doxygen (Score:3, Interesting)
In fact this doxygen was only part of the solution, our research projects had other documentation which required addition, however we simply had the developers get into using LyX and had the doxygen script merge in the static docs with the code documentation. Made for a nice doc tree which was easy to update (and if you really want to get fancy every once in a while, import into LyX and whip up a tasty DocBook). Of course the developers has to conform to the commenting style required, however if this is a problem for your development team, then the whole 'team' concept isn't going so well.
If you don't use CVS and/or doxygen, and those tools don't fit into your workplace (although CVS should be used, IMHO). You could easily whip up a php or perl script to merge TODO and Changelog files, this is of course assuming that all the developers use these files to track their work, which they should do anyhow (at least in some form). It would be trivial to whip up a parser in perl and merge those files into some sort of report. Thus the whole Practical Extraction and Reporting Lanuage thing...
Plain old HTML (Score:2)
Depending on what type of software your company writes or is producing documentation for (I'm guessing web applications), the structure of the document would be tailored for this, with appropriate entries for functions, global variables, and document constructs.
What's good about this is you can turn HTML into anything any language, and if you write it right you can show it through just about any device. Later on, you might want to package windows help files with your shipping product for your client -- HTML would work well for that. Well, HTML would be much better than any custom or home-brew solution you could come up with.
I would start by looking at what other projects have done. Which project's docs are easy for you to understand and progress through? Copy them. Start with a good synopsis for your own, introduction, and begin planning your document structure. What you will probably find is that your documentation is going to look like (or should look like) your source tree.
You mentioned Cold Fusion (Score:5, Interesting)
Part of FuseBox is FuseDoc which is a XML based spec for putting docuementation inside your CF code. By using Fusebox and FuseDoc you can break your web apps out into separate modules that work together much like different objects in C++ or Java. This allows you to have multiple people working on an app at the same time, while also separating your content from programming logic. I have used this approach in several web apps and it has worked well. Couple these techniques with something like CVS and some organizational programming standards (make standards that make sense!) you should be able to improve your work enviroment.
Like the old quote says (Score:3, Funny)
A simple system that we use (Score:5, Informative)
1. Code level documentation. Having good code-level comments is vey important for figuring out the nuts and bolts of how things work. Well commented and well structured code can make or break a long term project. The important thing is to keep the comments up to date so that you aren't providing misleading information. This is easy to do, however. All it takes is a little dicipline.
2. Process and Design Documents. There are many ways to handle these, but one simple way to do it is to create a development "Intranet" and keep all of your developer docs in HTML format. HTML documents are easy to create, easy to organize with hyperlinks, and easy to view. Keep a copy of your HTML documents in CVS or some other version control tool. That way you can have a record of your changes.
Code level docs are pretty easy to get started. If you have a good development team, you probably already have your code well organized. If not, assign pieces of your project to different programmers and have them document their assigned code.
Design and Process docs can be handled the same way. Make a list of things that you need to document (ex: build instructions, class hierarchies, etc) and assign these out to programmers.
The key to any documentation system is to keep things up to date. The best protocol is to have people make changes to the docs as they change the system, or as they discover bugs in the docs. Treat your docs just like you would source code. Strive for "bug free" docs. If you can't make a change to a document for whatever reason, log it as a bug so the change doesn't fall through the cracks.
Once people get used to treating docs as code, they will keep them up to date. If people have the attitude of "I'll document it when I have the chance" your system is doomed to fail.
Good luck!
A good place to start... (Score:2)
RIGHT NOW
Or suffer our fate...
There will be a time were you will absolutely have to start documenting your products, how-tos, specifications, deliverables, whatever. Finally when you realize you have to do it NOW... you'll look at all the work you have to do, and nobody likes to write. The best thing to do right away is, if you have the money, is to hire a technical writer. Then have him or her write, that's what they do all day.
Then you can worry about problems like versioning or storing the documents, whether storing them in the DB or in your CVS (I guess that is considered some form of database).
One thing to bring up is there is a lot of great ways to document these days within your code. Javadoc is a great utility if you want your coders to start documenting their own stuff, learning javadoc is easy, but doing it is a pain in the ass. I think Microsoft in their Visual Studio.NET is introducing the
/**
* @author Me!
* It's 2:00 in the morning, I'm tired and drunk
* the following ain't pretty but it works for some reason
*/
Multitple word processors is the key (Score:4, Funny)
This way you will know exactly who maintains the document - store all the files on the users hard drive , dont worry about backups - hard drives are very, very reliable these days and the damn server is usually always on the blink.
If someone needs to modify another document, they simply need to write notes on the old printed version and the document maintainer can update them later.
At least that's how it was in my last job
Document what? (Score:2)
First you must decide what you are documenting.
Are you documenting:
These three items have radically different approaches.
The first, item, "The Process" has its own sub-itmes. Is it to get someone acquainted with the program. Is it used to help manage its growth?
The second item too (pun intended) has a couple of sub-items. Is it to keep functions from overlapping? Is it to show what each one does once the user already knows that this is the function to use.
The third item also can be divided into sub-items reflecting on who needs to read it.
The approach for documentation is first to decide what *exactly* you are documenting, and then, who is it being documented for. Between these two details, you can decided on the approach, and the tools.
Deciding where to store the documentation would be where everyone could get to it. Storing it ina database and displaying on an intranet sounds like an excellent idea.
Start with commenting the code! (Score:3, Insightful)
Start with documenting the code first! Since the code is what must be
maintained in the long run, this is the critical place to make sure you have
decent documentation so any programmer can pick up where the last one left off.
Well documented code should contain as much comments (if not more) as
code.
Once the code is well commented, pull the comments out and use that as a
start for technical documentation, a reference to the design, functionality, and
interaction with other programs, modules, etc.
Once this is complete, then, and only then, should you start documenting the
programming project as a whole, mapping out database layouts, etc. (note: this
is from the standpoint that you did not do it correctly the first time and are
basically working from scratch). No matter what you use to document the system
in, it should be a common format between all projects and systems. Databases
should be documented using a decent data modeler, and don't forget to document
the stored procedures (in the procedures as well as external
documentation).
Once all the technical documentation is complete, then you should start
thinking about documenting the project from a user's point of view. If a system
works well and is easy to navigate, the users will rarely reference a help file.
I have walked into too many projects that were completed in haste, and looked
at way too much code that was not commented at all to know that had the previous
programmer done his/her job well, it would have saved me half the time trying to
understand why they had done something the way they did. When writing code it's
easy to assume that the code you write is blatantly obvious, but to someone not
knowing the system from the start, it won't be, keep that in mind and all your
code will end up being well commented, and writing documentation will be easier
later for anyone looking at the system.
In the future, remember that a well organized programming project will
involve much more planning and pre-documentation then actual coding.
too many methodologies out there (Score:2)
Some more popular ones:
First is the one coming out of MicroSoft called SOLID. MicroSoft software development was pretty disorganized in the 1980s, but improved considerably in the 1990s. Love them or hate them, they manage to get a fair number of winners out now. You should know their methodology.
A very popular "counter culture" slashdot-type methodology is Xtreme Programming. Emphasizes small groups, pair-programming, incremental project goals, e.g. shippable software every 2-4 weeks.
And then there are the tools, for handling pieces of the projects. UML for capturing the objectness of a system. Source control for software versions. Nightly builds and automatic tests. Project Timeline Planners. VERY, VERY USEFUL is the bug/enhancement multi-user database. It ties together managers, developers, testers, and customers. It gives a measurment as to whether more bugs or fixes are being generated at given pahse of a project.
Well... (Score:2)
We've got about 15 people here, and generally, the way we do it, it we have project specific production folders on the network, and those folder get backed up at important points in time (delivery, every month, depends on the project and it's timeline). Thos CD's are stored in categorized drawer, which has an accompying spreadsheet available to everyone on the network. This spreadsheet is (or should) be updated every time a cd is put in the drawer. Most workstations have a cdr built in, plus we have a machine for just burning cd's, plus a USB drive for those unfortunate who don't have an internal, and too lazy to go into the other room.
We also have a paper system, where all relevant materials, such as proposals, vital emails, as well as hand written stuff, goes into hanging file folders in a file cabinet. 3 months after final work is done on a project, the folder gets archived to the basement where we do all our file storage.
And of course, the gratuitous entire nightly netowrk backup, which is kept off site, in case any of those cd's ended up kaput.
It works for us, like I said 15 (which is generally our max, last summer we were down to 7) production employees, including programmers, designers, QA, and project managers (that list isn't exclusive, most of us where a few hats).
Project boxes (Score:2, Interesting)
If your company is affected by much turnover, then it's critical also to organize the boxes. For example, the first project of 1999 is numbered 99-1, the second 99-2, and so on. The archive shelves are ordered by project number. Put accounting in charge of keeping track of project numbers, as they already do...
What to avoid:
- each person responsible for keeping their own archives (no one remembers a year later if the guy who left did that....)
- lack of centralized storage (no one remembers three months later where the cd with the source code is or who had it last)
- depending on the client keeping the copy you send them. This is just stupid. They hired you for a reason, after all.
- depending on the weekly server backups. Those aren't project oriented, may or may not be around for a given week a year later, and can be hopeless to use if you're not sure what you're looking for and where it was.
Our company made some attempts to leverage work across projects. That usually didn't work so well. Each project, and each client, was just different enough. So you probably don't need everything from every project at your fingertips on the server all the time. The main goals for our archives was to be able to:- Prove it was client changes causing problems, as what we sent them worked, and we're happy to send it again.
- Pick up where we left off when a client comes back wanting changes in six months.
- Deal with a project that gets cut or goes on hold, and then comes back the next quarter with more money and wanting to "add it all back".
- Provide documentation for any change orders or contract disputes that may arise, during or after a project.
A good PM will archive at various stages, external and internal: every client delivery (alpha, beta, RC1, final), at end of design phase, when you've got a working prototype, etc.Good luck!
modules, etc (Score:2)
Make sure that everthing you code is in modules/objects/etc.
Then, as people code they enter data about how the modules and interfaces behave into a database.
Everyone gets to add info about the modules, no one can remove it. You are building a database of knowledge about your work.
Modules in the database have extra fields to identify functionality, connections, general areas.
You can then run reports which become the basis for chapters, etc.
Of course there are plenty of decent programs for this sort of thing, complete with version control, etc. In fact you could use any common coding version control program, storing basic text instead of code. Run it in side by side with the code. then export to a text file for reformatting, etc. just keep style, etc consistant.
Once and only once (Score:3, Informative)
Write things Once And Only Once
This means that whatever documentation you produce it should not duplicate information that can be inferred from reading the code. Documentation should only compliment your source. Describing algorithms in a Word document is a terrible idea as any duplicate information will get out of date. It's only a matter of time. Usually documentation should be more detailed around the user requirements and sparse around how the code works. Documentation is not an asset. It's a liability. Any new document introduced during the development process is an additional maintenance headache and a delay in project's completion. Always think twice before adding every single type of document into your development process.
Write it down! (Score:2, Interesting)
I like HTML for documentation because...
- It's easy to work with and cross platform
- Access it from any machine on your intranet (or internet if your brave)
- Set up a standard documentation template to help keep everything uniform and to save time in creation
- Use a scripting language to automate creation
- Easy to convert to any other format
- Easy backups at one location
I would suggest treating your documentation situation like any other important project. Give it some attention, milestones, priority, manpower, etc. Start by documenting your documentation process!
No idea? I can help! (Score:2, Funny)
we have no idea where to start.
"The "
Recommended reading: Zope Fishbowl (Score:2, Interesting)
See Zope fishbowl [zope.org]. It's a nice summary of a very lightweight process with some well-written thoughts on why such/any process is necessary.
As far as where to start, start with what you have. Most (not all) developers are happy to put their meeting notes, design docs, and such in a common/standard location and format (I recommend a text-based format in a revision control system) if one is simply designated. Someone whose technical ability they respect may have to remind some of the developers from time to time to put their docs in the designated place or to create a document from some important conversation.
Be careful not to attempt to switch from what you have now (nothing) to something unrealistic (full UML class and system diagrams, workflows, and state transition diagrams for every project). Be realistic and simply start to collect what you've got. Designate a spot for hard copy documentation and electronic documentation. Another easy first step is to designate a common format (ASCII, HTML, PDF, StarOffice, whatever).
Give that a few months to sink in. Then start to list common documents that should normally be associated with a project. Several separate "artifacts" of a project may include requirements, functional (user-visible functionality) design, detailed (class hierarchy and component interaction) design, implementation notes, decision log (bullet items of decisions made with date and those involved).
Remember: something is better than nothing. If each project simply has a decision log (Greg and Sheila decided not to support wizbang with a froznobit because it would cause all the tulips to wilt), then you have a start. Six months later, when someone asks, "Why didn't we use froznobits to implement wizbang? Maybe we should fix that," they'll have somewhere to look for the answer. The same can be said of requirements and functional design. Developers can usually figure out detailed design by long hours spent pouring over code. There's no way to divine what a program is *supposed* to do if there are no comments and no requirements and no functional design.
XML+XSL--HTML, man pages, headers, the works (Score:3)
Project artifacts (Score:2, Interesting)
Underneath the Squishdot app I have a projects folder. Under it, 2 more, software and firmware. Under those, a folder for each project. In them I use a plugin called LocalFS which exposes a system forlder to the Zope environment. We put all our docs, spread sheets, gantt charts and any other project artifact. Another page does an SQL query against our bug database and displays that kind of stuff.
I can also pull in headlines via RSS to make the site more interesting and usefull. I even use Radio Userland at home to create a custom RSS feed tailored to our specifics needs.
The last thing is to instill the culture, this takes time and patience.
Documentation should be part of the process (Score:3, Insightful)
The first thing you should do is create coding standards in which documentation is part of the coding process. The documentation standards should be such that you can automatically generate documentation as the code base changes.
For example, on every Perl project I work on these days, I introduce a script (about 30 lines of code) which auto-generates documentation from the PODs. The script can be run as a cron job that auto-updates the online documentation every week. That way, coders document as they go, and central documentation resource automatically regenerates itself on a regular basis.
The second thing you should do is set up an intranet site (behind the firewall!) for developer documentation. Include the auto-generated documentation, documents describing your coding standards, project management documentation for developers, use cases, class diagrams (does ColdFusion even have objects?), activity diagrams, sequence diagrams, etc., etc. If you have a team of 20 people, undoubtedly you have someone who manages code revisioning. Likewise, managing the developer intranet should be included in someone's job description, and, what's more, they should be given the time to do it properly.
Hope this helps
The way it works for us... (Score:2)
We used to have the same problem when I joined my current project in 1999 (yep, 4 versions later, we're still doing the same product). We are using Lotus Notes (no choice), which has serious drawbacks (no Linux client, buggy SMTP, weird shortcomings, very bad help, crashes at odd times) but happens to work very well as a collaborative environment. I suggest you get a good collaborative environment that allows its users to:
Then create one or several databases containing "how-to" documents. For instance, I see a few popular documents here: how to reload our DB tables, how to set debugging options of a certain product, how to configure SSL for our web server...
We have one of these little document for each procedure that has to be done several times. One-off tasks are not documented. There is a strong incentive to document a procedure when you've been asked to perform it several times.
Random papers and drawings are scanned, but their use is discouraged in favor of editable files.
Mostly, whatever collaborative system you use, train people to use it and provide incentive for its use.
And don't think it will be free. It will start saving money after several months, not next week.
PLEASE! (Score:4, Insightful)
Please, go out and buy it and read it - or check it out from your local library. It is a must for all programmers, even if you're not the one in charge.
Don't. (Score:2)
I came into a codebase of about 5000 cf pages. Only two of the original developers are still with my company. There is no documentation and these were not good programmers -- my portion of the application was by and large coded offsite by amateurs. And at the same time, I have never really been halted in my work by the unavailability of documentation. Hindered, maybe -- slowed a bit, especially during a set of endless includes. But never halted.
ColdFusion is a mess to program complex things in. Cfscript is apocryphal and best replaced by simple COM and Java modules that do the same thing faster and in a much more understandable fashion. What you're left with is a base of very simple, readable code that's actually spoiled by documentation. (those <!--- tags are murder)
Oh yeah: name your variables wisely. That's the key to ColdFusion
Re:Don't. (Score:2, Interesting)
Re:Don't. (Score:2)
We wrote the same simple i/o algorithm -- extract rows from a database and loop across them to output a table -- in practically a dozen different ways in three different languages beneath IIS, and the CF implementations (CF using a CFX tag, CF using cfquery/Cfloop, CF using a cfscript to loop and call writeoutput()) were all slower than even ASP. A servlet with a pooled connection and a persistant bean was fastest, but this is almost cheating.
We did extensive testing and have on numerous occasions sworn that we wouldn't accept ColdFusion anymore now that all the developers here know ASP and COM (all the CF only people are gone). And you know what? Despite the inefficiency and poor scalability we've noticed (which, I'll concede, is due in part to that fact that some parts of the application were not planned or coded correctly), we still do all new development in CF. It's just faster to get done because the syntax is easier and working with display is more like HTML than JSP or ASP. It's more natural.
In summary: I disagree, the lanugage does suck, but I love it. Hell, it's paying for my house!
Documentation can't be pulled from nowhere... (Score:3, Informative)
What documentation would I consider garbage? A lot of these replies suggest you use "literate programming" and "comments in the code", and while these are very useful to maintenance programmers they are nowhere near as important the specification, analysis, and design documents that should have been generated before implementation had begun. For example, I'm sure none of the
What you should be asking yourself:
1) For each of the projects we are currently working on, do we have the design documentation?
2) If 1, is the code we are writing completely consistent with the original design documentation?
if (1 == false), you are in BIG trouble. While development on a project such as that may go okay, maintenance will be a minor disaster area. It's unreasonable to expect maintenance programmers to maintain a project that's so poorly documented, and for such a small organization spending gobs of time hunting regression faults because the idiot down the hall made a change to a module that he didn't understand and that wasn't documented could be fatal.
if (1 == true) but (2 == false), you are still in trouble. Wrong documentation is orders of magnitude worse than no documentation, because what happens is that maintenance programmers don't realize that the code doesn't fit the specification and end up trashing half the project until someone finally catches the mistake.
My suggestion? If you have projects like #1 or #2, consider all the work you've done so far as a rapid prototype, trash all the code, and start from scratch with a reanalysis of the specifications. It might sound brutal, but you'll spend less cash in the long run doing a total overhaul now rather than waiting until later and letting it all catch up to you on the bill for maintenance.
If your projects are well-specified, analyzed, and designed, your employees already have the competence and the CASE tools they need to document their code well. Remember, the emphasis is on COMPLETE, COHERENT, CONSISTENT documentation; hell, even resort to printing your documentation on dead tree if you have to, as long as it's complete, coherent, and consistent it won't matter that much. Accuracy of documentation over efficiency of access.
-inq
Re:Documentation can't be pulled from nowhere... (Score:3, Insightful)
Also, tools that generate documentation from code give you the design documentation just when design documentation is the least helpful. At best, documentation of this kind will only serve to show you how brain-dead your initial design was (because by the time the tool is useful to "reverse-engineer" your code it's already far, far too late to correct your boneheaded design gaffs).
Finally, the title made sense to me but yes, it is a double negative; consider it revised to "Documentation can't be pulled from the ether...". Creating the documentation from nothingness is a classic case of selling the horse to buy the cart.
-inq
Hire a secretary (Score:2, Interesting)
Try hiring a secretary. Organizing and keeping track of documents is centuries-old technology already.
And good luck. You've just admitted that your organization doesn't have even the most basic organizational skill between the lot of you.
Re:Hire a secretary (Score:2)
Re:Hire a secretary (Score:2)
I've done both done consulting and development jobs with various sized development teams, and headed development teams of various sizes for several years, and I've seen large teams fail miserably because of lack of methodologies and small teams fail miserably because they where spending time on formal design and documentation phases where their competitors were not (guess who got to market first....), and the other way around.
It is all about setting the requirements based on the projects size and complexity, not about insisting on using formal methodologies whether the project requires it or not.
You may keep your illusions if you want, but if you truly believe that an organization spending time using formal methodologies for 2-3 month projects involving 2-4 developers or thereabouts (face it, the majority of software development projects are hacks, not major pieces of software) would be competitive, then I know I wouldn't even consider hiring you for my team.
Once the team grows, formal methodologies isn't only a good thing, it is an absolute requirement for success, as it gets to difficult for even the good developers to keep track of all the interdependencies.
Somewhere along the line you reach a point from where formal methodologies isn't overhead anymore, but a vital requirement.
But don't kid yourself, if the project is small enough it isn't about structure or methodologies, but the qualities of individual developers, and imposing stringent rules on them will often have the opposite effet of what you intended.
Actually, don't write any documents at all (Score:4, Interesting)
Let them hash out all the details of what you believe the customer wants. If you have a real customer available, chain him to a table in that room and don't let him go until you're satisfied.
Write down user stories on post-it notes, stick them to the walls, sketch UI designs and user interactions on the whiteboards. Don't worry about documentation! What matters is that you all have the same vision and understanding of what to do.
Documentation on paper is just about the worst medium possible for transferring information from one person to another. It is one way, it is low bandwidth, it is not visual (unless you have lots of pictures and such). It basically sucks. Two or more persons by a whiteboard is the best possible technology for information transafer. The ultimate solution is if you have a printing whiteboard, which can give you a hard copy of what you just jotted down.
The real documentation, on paper or on the web, can come much later, when things have stabilized a bit. In the end it may never be needed, in case your project is cancelled.
All these tips comes straight from the book Agile Software Development, by Alistair Cockburn. The best SW related book I've read in years!
Mats
Project documentation tool. (Score:2, Informative)
So, being a software company, we wrote a tool! It's called Outreach Project Tool, and we've GPL'ed it. You can check it out at http://outreach.sourceforge.net [sourceforge.net], and download the source from http://sourceforge.net/projects/outreach/ [sourceforge.net]. Note it's dependent on LAMP.
Congratulations to Lanifex (Score:2)
Phillip.
Video Tape (Score:2)
The main problems are:
To solve this, i propose a novel approach:
Video Tape everything
Record:
- Brainstorming sessions
- Developers describing their code (highlighting it, showing it on screen, explaining what it does)
- Building and integrating activites
- Everything else you can remember
Seems like a weird thing, but try it. When you document, having the tapes around is a precious resource.Re:Ease of use? (Score:2)
Doc writers will use the tapes in order to assist them in writing the documents for the project.
Ask youself: what's the purpose for docs? (Score:2)
All documentation on software development projects falls into a fairly small number of categories.
It's quite important that, in some form or other, you have records of your project's current state and future intentions in each of these areas. It's also worth noting that those records can serve two very different, but complementary, purposes:
From a certain point of view, how you actually generate and record these documents is only a detail. It certainly helps to have a good formal version control system in place, with each formal document carrying an issue number you can reference and a date of issue, for example. Tools like CVS can help with this.
<Asbestos> I question the worth of most tools that generate documentation "automatically" from code, particularly from specially formatted comments. </Asbestos> If you want to know what a particular function does, or what the parameters to a particular call are, or which web site your site links to -- all perfectly sensible questions -- these things should generally be examined by looking directly at your code/web page source/whatever. There is no point, IMNSHO, in duplicating this knowledge and wasting whole forrests in order to print this out as a 200 page document that's immediately out of date and no-one will ever read anyway.
Once you understand the basic types of documentation you're likely to need, you'll want to work out the specifics in some detail, preferably directed connected to whatever development processes you use. It's worth checking to see if you can get tools to help with things (e.g., a tool to draw UML diagrams for your design docs, a tool to parse web sources and graph which pages link where). Once that's all in place, you're on your way. It sounds like a lot, but actually, most of it should just be common sense, and it's well worth investing the time early to sort out what types of document you want to have, how to store them, what tools you can use to help write them, and training your team in how to use the system you've created.
Start today, organize tommorow. (Score:2)
You need to start documenting today. when you make a change, document the area you change, how it works, why it works that way, gotchas that you had to watch out for...
I started doing that 3 years ago for code that I maintain, and I'm getting close to done. Often I go back to something I documented before and update it with knowledge I discovered latter that makes things clearer. I also discovered I maintain one module that is as close to perfect code as I've ever seen. I know that because I have not done any documentation on it which means nobody has found a bug, or a situation it doesn't handle already.
Unfortunatly I've also discovered some things that I don't know how to document in a way that others will know what to do when it happens again. In my case hardware designed a new board that required different registers. We had to compile some things differently for that board, and to make our makefiles do that is really easy once you understand how they are generated, but difficult if you do it the obvious way. A simple change once I found it, but I don't know how to tell others so they will figgure out where to make the change.
In any case, once you get some documentation it can be orginized. Thats not something I do well. I don't write documentation well either, but at least I write it, and that has helped.
CYA (Score:2)
Small company here (~20 ppl), and we keep notes on various projects in a custom application that we wrote with a database backend that allows us to write notes on projects, track time spent on projects, and run reports.
Another major part of our arsenal is a CYA box. (Cover Your Ass) This is a box or other container into which all notes, drawings, printouts, scratchings, post-its, code scraps, etc....EVERYTHING goes. Why?
Two reasons:
- It allows us to show a paper trail of our development. If you ever get sued for copyright or patent infringement, you'll be glad you had this. It shows, in order, how the application was developed. Since everything in the box is layered in chronological order, this works well (First item in is oldest, last item in is newest). We seal boxes up when they get full, date them with the start and end date and mark a destroy date on them (7 years hence, usually).
- Ever delete something you wished you hadn't? Had a HD crash and wipe out some of a program? Then a CYA box is a handy thing. I haven't had to use it very often, but trust me, restoring programming code from paper is better than nothing. (tape or CD is better, but it's a last-resort backup)
Whatever you do, document EVERYTHING. If you sneeze on a project, document it. Documentation is one of those evils...if you don't do it, you'll be fast but you'll kill yourself later, and if you do it too much you'll slow down product. But it's worth the extra effort.Re:WIKI (Score:4, Informative)
Since you seem to be a cold fusion shop, try this out http://www.cdsi-solutions.com/cfwiki/ we had it running in about 2 minutes.
Re:Poor journalism (Score:3, Insightful)
You're just jealous that someone using ColdFusion can do something in half the time it would take using PHP.
Seriously, because it's "closed source" it's bad?
A good analogy would be Apple. Yes, Macs cost money, and OS X is not open source. But a Mac with OS X is far more elegant and easier to work with than a Windows PC.
Similarly, ColdFusion costs money and is closed source, but is easy for new people to understand, and incredibly powerful at the same time. It has a plain-english syntax that actually makes sense, and has tons of built in functionality that let you get work done, instead of chasing down some PHP component that you read on a message board might work if you can track down the guy who wrote it in some foreign country. It's backed by Macromedia, a real, permanent, profitable company that has tech support and a phone number.
Perhaps you're just scared that someone without a CS degree could take your job using ColdFusion?
Re:Poor journalism (Score:2)
You're just jealous that someone using ColdFusion can do something in half the time it would take using PHP.
In my opinion, someone that knows his language well and has developed a decent library will do things faster no matter what the language.
Seriously, because it's "closed source" it's bad?
I never said it was bad, it's just not as applicable to the majority of the readers.
A good analogy would be Apple. Yes, Macs cost money, and OS X is not open source. But a Mac with OS X is far more elegant and easier to work with than a Windows PC.
Not a good analogy. You are comparing one closed source system that costs money with another.
[snip sales pitch for Cold Fusion]
Perhaps you're just scared that someone without a CS degree could take your job using ColdFusion?
Someone without a CS degree will never be in demand as someone with one. There are many basic algorithms that apply to any language you choose. Where businesses use self-taught programmers, we tend to get paid even more to clean up afterwards.
Phillip.
Re:"... NO methodologies"???? (Score:3, Interesting)
In my experience, peer review is more important here than methodology. In fact, methodology can afford to be rather sparse (and it should be for most circumstances) when peer review is relied upon.
Peer review is good because it helps good programmers become better programmers and at the same time, stops bad programmers from doing really stupid things. I think documentation is something that should be done, for the most part, after a project is close to completion.
For instance, we followed the standard procedure of design, then implementation, etc., etc.. Problem was that the people who did the design and implementation, weren't a part of the maintenience phase and all that design stuff was no where near how things were actually implemented.
It's really something that can't be avoided. Code tends to have a mind of it's own once it's written. There's a bit of a chaotic factor in it to where the slightest unforeseen bug in the most obscure case senario leads into a huge design problem.
Besides, it's alot easier to manage with a working product with poor documentation that it is to manage with a broken product, and acceptable documentation (or even good for that matter).