Getting Accurate Specifications for Software?

spiffcow asks: "I design internal software for users that are largely computer-illiterate, and obtaining accurate specs for these programs has become a huge challenge. In the most recent instance, I asked for detailed specs on what an accounting program should do (i.e. accounting rules, calculation methods, and so forth), and received a Word document mock-up of an input screen, complete with useless stickers. This seems to be the norm around here. When I asked my boss (the head Sales manager) for specs, he responded saying that it was my responsibility to determine what was needed. How do I convey to the users that, in order to develop the software they want, I need detailed, accurate specs?"

Something I struggle with as well

[myurr]

This is something that I also struggle with, so I'm very interested in any tips 'n' tricks that others can supply. The only useful trick that I've found helpful in the past is to take an iterative approach to the documentation, repeatedly sending drafts to the interested parties and encouraging feedback. Often their problem is that they don't know what they want, only what they don't want - so starting to lay out some options before them helps them make decisions on what they would like to see. Start at a high level, and slowly drill down on the detail - making assumptions where necessary to keep the process moving, but always verifying those assumptions with the interested parties.

Impossible

[synx]

What you think your job is, and what your actual job is are two quite different things. Traditional software 'methodology' is bunk and doesn't work - this is why you are confused.

You think it works like this:
- User knows what they want
- They write it down
- You...?
- Programmers implement it (probably wrongly)

If you consider your job more like an architect, then you will see the flow is really more like:

- Users think they know what they want (maybe)
- They can tell you what they DONT want
- You interpret their needs/desires in to a design and spec
- Programmers implement it (probably wrongly, but nothing is perfect)

If you think about what architects do for their clients, they figure out roughly what the client wants (house, building, garden, etc) and various parameters specified and unspecified in fuzzy things (building code, safety margins, design principles, aesthetics, etc). They then produce a number of different designs and design ideas to run past the client. Iterate a few times and then once they have sign off, build it.

If you were required to write some 300 page doc about the house you want, you'd be finding a new architect. Likewise, make life easy on your customers. I'm sure they have pre-existing documents and references regarding the accounting rules they need implemented (I assume you are familiar with accounting - if not, why the hell are you building it?!). But as for the UI and other software design features, most people just want something that (a) works (b) well (c) usable (d) does what they need. Meaning, don't ask for label or window placement.

If you have a RAD tool such as interface builder on OS X then you can create semi-functional mocks easily. I'm sure .NET has something similar.

Yes. This is hard.

[WasterDave]

obtaining accurate specs for these programs has become a huge challenge ... When I asked my boss (the head Sales manager) for specs, he responded saying that it was my responsibility to determine what was needed."

You're trolling, right? I hope so.

Yes, it is hard. Much harder than actually writing the code. Yes, it is your problem. Software Engineering is a profession. That's why you and I get paid the big (in theory) bucks ... to make hard things your problem. And solve that problem.

Without going into too much depth the process you have described (accurate specs, make software, test software against spec) is known as the waterfall model and is famously difficult to do for non-trivial projects. Can be done, don't get me wrong, but very very hard. Better, probably, would be to take an iterative approach: Take the word doc and bash together a prototype (RealBasic, Ruby on Rails, whatever); drop the prototype in front of the users and make notes as they say "nooo! not like that, it needs to do X, Y and Z"; feed back into the prototype and try again. Finally use this prototype as a "living" requirements document. The hard part is persuading the pointy haired types that that prototype is, in fact, not the completed piece of software. Yeah, good luck with that.

Not wishing to sound offensive but it sounds like your company needs to hire someone with more experience to act as a project manager. There's nothing wrong with writing code to spec (no matter how it's translated) and letting it be someone else's job to keep the project on track and ensure the users get what they want. And, in case you hadn't noticed, this job is hard f'kin work.

Dave

Re:Yes. This is hard.

[ivano]

It's not up to him to make all the decisions since it's not him that is taking the risk if the thing doesn't give the right answer or says the transaction is done when it hasn't. You want him to make the decisions that people higher up should be. Decisions need to be made by the person who has the risk. Sure testing, documentation, hell, even code-writing is his job, but then to insult him about his abilities and then talk about how fucken difficult the thing is is a bit back-handed.

You even mention him needing a good project manager (PM). Well if he hasn't got a PM then wouldn't the advice be "get a good PM" (not just calling him a troll). I've seen too many projects die because the software engineer is a "yes" man. "yes, I can do that", "sure, i can fit that in". Whenever I see a developer do that I worry. Because if he's saying that to me he's probably saying that to all the other PMs and bosses and so wasting time on projects he's not allocated to. So the guy works day and nigh and on weekends and when the big roll-out comes he's so crashed and burnt he can't even think straight let alone fix the last few bugs.

Your job as a software engineer is to also stand your ground, say, "*You* need to prioritize the tasks you want me to do", "*You* need to give me the financial algorithms in a way I can implement them" etc etc. That's what's fucken hard. The other shit is easy :)

Requirements Solicitation

[s31523]

Your problem is not unique. I attend an Extreme Programming workshop near where I live and a guy by the name of Richard Sheridan came to do a presentation on his companies technique called High Tech Anthropology [menloinnovations.com]. It was a great presentation and it is something you might try. Basically, you camp out in the users "Den" and observe them, taking notes and trying to understand how they work, what buttons they push, which user interfaces frustrate them, which things they like, etc. You then take this back and use it to publish your requirements specs. Some XP enthusiasts talk about bringing the customer in and having them work with them team, but Richard Sheridan makes a great point, that this can sometime lead to the users becoming more like engineers rather than the other way around (like the book The Inmates Are Running The Asylum). [uidesign.net]

Re:an unrealistic ideal

[Diomedes01]

What I found is that when they are in front of the app, after a bit of usage they think "could you add feature X?" "how can I do Y?" and so on. I implement X and Y, and only then they ask "oh, you did Y? So why not Z?" etc. So the spec becomes dynamic, in the sense that only when they see a milestone accomplished new possibilities come to their (and my) mind. It's a climbing process. I don't know if it's the same also for pro developers.

If you are lucky enough to live and work in an environment that allows this, then it is, IMHO, the absolute best method for developing software. Now unfortunately, in much of the world, and especially at larger companies, very rigid software development practices are followed that make this sort of agile, iterative development difficult or impossible. I am lucky; I work at such a company,and work directly with a group of developers who use a very rigid, unflexible system; we don't see the product until it's been completed based on the spec - any iterative feedback I or my colleagues has is worthless, and would have to be done to fit into the next quarterly release cycle. Luckily, I also do my own development for some internal departments, and am given the freedom to work in a more agile manner.

Re:he's right

[qwijibo]

Are there actually companies that write proper specs? I've only been doing IT for 19 years and I have yet to find any place where it actually happens. It's something I've always heard of, but never ran into anyone who had actually seen it happen. Generally, I've found the organizations least capable of writing specs to be the ones most likely to outsource, not the other way around. I think the idea is that if you don't know what you're doing, you may as well pay as little as possible since you already know you're going to fail. I agree with that philosophy, which is why I expect to be paid more for projects that people want to succeed. =)

The real goal is to ensure that the developers and users/customers are trying to address the same problem. The specs/requirements/design phases are just ways to document everything so that when it doesn't happen, someone can point to a document and said "this is what you said you wanted, pay us". It's a legal CYA. This is why it's more important to have these documents when the users and developers aren't part of the same small group of employees.

Re:Requirements Solicitation

[s31523]

I deployed this strategy and it worked quite well. I was developing requirements for a UI on a flight system for the function of flight planning. I traveled a lot with our company plane and road "shotgun" a lot. I started bringing my notepad and jotting down what the pilot was doing with respect to flight planning. I also made several notes on the most common buttons and scenarios based upon what Air Traffic Control was telling the pilot. I flew in good weather, bad weather, heavy traffic, and light traffic. After about one or two months I went back to our spec and said, this is all wrong. The widgets here and here are "cool" from a programmers perspective but the pilots will be annoyed by this. I also made several suggestions to the radio team because one of the most common tasks the pilot did was change communication frequencies for radio navaids and ATC operators. I got a lot of flac for my suggestions, to the tune of mind your own UI, this is what the customer was shown and this is what they are going to get.

Anyway, my point is, it can work, and probably for more applications than you would initially think. It might just take a little, I hate to say it, "out of the box" thinking.

Re:he's right

[Richard Steiner]

Are there actually companies that write proper specs? I've only been doing IT for 19 years and I have yet to find any place where it actually happens.

At most of the places I've worked, the creation of specs is an interactive iterative process between us (the developers) and the users. Sometimes it starts with an idea, sometimes with a detail write-up, but most of the time the gory details take a little while to mail down and will usually get implemented in code first and then changed over time based on user feedback.

When I worked at the Unisys Airline Center, they had a fairly good process in place to meet with the customers and draw up a series of general and detailed specification documents based on those meetings, but that was a place that released software on a roughly 18-month cycle between versions, not a live shop with a constantly shifting set of applications.

I think it's harder to write project specs for a living system -- it's a lot easier to document the system in detail after the fact. If one has the time. :-)