Making the Case for Better Bugtracking Tools? 33
WeNeedBetterTools asks: "Help! I am trying to convince the leaders of my development organization that the quality of our bugtracking tool has a meaningful impact on the quality of products we produce. Right now, the tool we have is difficult to use because there is a high overhead to creating bugs, and they are very difficult to find in the system once they're reported. My main argument is that the easier the tool is to use, the more people will be motivated to enter bugs. My managers argue that if people aren't entering bugs, it's their own damn problem, and we ought to crack the whip until they always file every bug they possibly can. I need some solid references for my position. JoelOnSoftware has a great argument for making it simple to enter bugs, but he lacks credibility with my managers by virtue of the fact that he is also selling a bugtracking tool. Can anyone point me to any solid research on the relationship between the quality of bugtracking tool and the impact on product quality?"
so true (Score:4, Funny)
I can confirm this. I introduced some bugs in my code today, but boy, am I tired right now. But hey, every bug counts.
Been there... (Score:3, Interesting)
Assign someone in QA as the "bugmaster". Possibly rotate the responsability between team members so you don't get burnout.
Have people email bug reports to the bugmaster and have that person enter the bugs into the system the correct way. Yeah, it can be a lot of work for the bugmaster depending on the size of the system, but the QUALITY of the data will be vastly improved.
Trust me on this. A tool will not fix a process problem, no matter HOW easy a tool is to use. If you ARE choosing a new tool, make sure that ANY tool you use is web-browser usable for ALL functionality (avoid IE only, active-x required tools)
Re:Been there... (Score:2, Insightful)
Re:Been there... (Score:2)
Hmmm... My experience is different.
A good working environment is designed to make good things easy and bad things hard. For example, I write a lot more unit tests when I have a good unit testing framework. I run them a lot more when it's part of my build process. And I do a lot more refactoring when I use a tool that takes care of the boring parts of it.
But I'd agree that no tools can substitute for a responsible human. A bugmaster should be a big help in introducing the tool, especially if the bugmaster is somebody respected by all.
But better still is the team coming together and deciding, "Damnit, we need to track bugs better! We want an easier tool!" Anything imposed above from the managers, especially ones who say things as painfully dumb as, "we ought to crack the whip" is much less likely to succeed than something adopted by consensus.
Show some initiative (Score:4, Insightful)
Most new technologies, Linux included, are installed in the enterprise by real people doing real work. Real Engineers(tm) don't wait around for the PHBs to realize a trend that's been going on for 6 years.
A Good Boss(tm) is one who sees that things work better now than before, and nothing more. If you expect more from a boss, you're delusional and you may as well start applying for management positions yourself.
And always remember: it's easier to get forgiveness than approval.
Re:Show some initiative (Score:2)
Um, what about the rest of the team? How's he going to get the rest of the developers to use Bugzilla if the managers aren't on his side?
Re:Show some initiative (Score:2)
Re:Show some initiative (Score:2)
>give them the bugzilla URL and show them how easy it is to add and
>track bugs. You don't even have to tell management until it's well
>entrenched.
You must work with smarter and/or nicer people then I do
Re:Show some initiative (Score:2)
If you are not using Debian, the installation process is complicated, cumbersome, and time consuming.
Hopefully someday Bugzilla will run on something other than Linux/MySql.
Re:Show some initiative (Score:2)
I tried out a lot of bug trackers, and I like Mantis [sourceforge.net] the best. It is simple, yet has the features needed to get the job done. It doesn't take hours to setup like Bugzilla. The user interface is much simpler, so filing bugs and using the system isn't a huge chore.
Re:Show some initiative (Score:4, Informative)
May I humbly suggest that you take a look at RT: Request Tracker [bestpractical.com] instead. RT is a general purpose ticketing system [bestpractical.com], suitable not only for bug tracking, for for all kinds of organized message exchange within an organization (i.e. help desk, sales force tracking, some aspects of inventory management, etc). RT allows users to collaborate via a web interface, email, or the command line. By providing multiple interaction interfaces, RT encourages users to work with the system by communicating the way they would already, rather than working against them by forcing them to adapt to a wholly new system. If you don't like the web interface, feel free to change it. If it's still not enough, people can just use email instead -- just cc: your RT account on ticket related mails, and include the ticket number in the subject line. Hey presto, people can do almost what they were doing in the first place.
RT is written in clean, OO Perl [bestpractical.com] making wise use of CPAN libraries instead of implementing everything from scratch. It will run on a variety of operating systems & databases (MySQL, PostgreSQL, Oracle). The system is well documented [fsck.com], easily extensible [fsck.com], and comes with a vibrant & supportive user community [bestpractical.com]. It can even be integrated with things like pagers, so that the creation of critical tickets can send out a pager message to key personnel.
All in all, RT is a very nice, very well engineered system that IMO is far more suitable for most users than Bugzilla, for which the suitable scope is much more restricted. That's why RT is now being used in, among other places, Perl's bug tracker [bestpractical.com] at rt.perl.org [perl.org].
Disclaimer: My company uses RT, and I have met Jesse Vincent, RT maintainer, a handful of times, and even though I think it would be pretty cool if people switched to RT and bought support contracts [bestpractical.com] from Jesse, I have nothing to gain if any of this happens. I just sincerely think that RT is better software than Bugzilla for almost all users, and would like to see development of the software continue to flourish and become accepted more widely. Spend a week messing around with RT and IMO you'll never want to go back to Bugzilla...
not enough info (Score:4, Informative)
Also, why is there so much overhead? Entering a bug should be as simple as filling out a single HTML form. Is your UI too big and complicated? You can probably strip it down to the essentials, without discarding it entirely. Is it on more than one page? You can probably condense them into one form. Are the bugs stored in a database? You can write your own search page.
And if you said which bug software you're currently using, you could probably have gotten more specific and relevant tips on how to make the best use of it (change the configuration, prepopulate certain fields, roll your own modified templates, etc.).
Just do it (Score:2, Informative)
Bugzilla is by far easiest and cheapest to implement in any environment. It is free and webbased and moreover, it is very easy to customize it. If your management doesn't want to spend too much money on quality it is their problem, but keeping your work fun and somewhat easy is your own.
So just implement a solution and after a couple of weeks show the improved results to management.
Another thing to improve stuff is by implementing UnitTesting. Have developers write UnitTests before writing any code and run your unittests with every build (Contiuous Integration is Heaven for testers!!)
If Unit tests fail, it is no use testing software by hand and you can start kicking developers around.
Re:Just do it (Score:1)
Re:Just do it (Score:2)
Amen! Any development shop should have a box that just runs 24x7 checking out the latest code, doing a build, and running all the automated unit tests (written by the programmers) and functional tests (written by QA). Whenever a test fails, the responsible developer should be immediately e-mailed.
Then if you get your developers in the habit of doing test-first development, which gets you circa 100% coverage, your bug counts will drop significantly. I worked intermittently on a high-traffic dynamic website that has had a total of zero downtime and two or three shipped bugs, all minor, after about six man-months of development and nine months in deployment.
Re:Just do it (Score:2)
I have yet to see a single bugzilla that even came close to end-user ease of use of Request Tracker. [bestpractical.com]. Bugzilla might have a million knobs and dials to twist, but that's precisely what I don't want to expose submitters to.
No research, but experience (Score:1)
Re:No research, but experience (Score:3, Insightful)
The first two steps, practices and their benefits, are something I've done a little research on, and put together a set of Defect Tracking Patterns [advogato.org] that discuss how to work and the benefits of those practices in context.
Maybe if you can convince enough of the folks to want to adopt good practices, and that the current tool does not work well for those, you'll have some leverage with THe Boss.
Fight hard (Score:4, Insightful)
While there was some resistance at first among developers and managers that had never done this before, over a few short months our code quality has improved and our production increased. In fact, the final case was made when comparing two teams of roughly equal size, one using the tracking system during development, one not. The one that was using it not only had fewer defects, but had written about twice as much code in the same time period. And the real benefit will come a year or two from now when we need to go back and triage a decision or issue long after the original engineers have moved around.
We happen to be using bugzilla and cvs as they are free, easy to modify to meet our needs, and familiar, but you could do this with just about any tool that allows you to track arbitrary issues.
Good luck -- and don't give up.
Corrective Action Process (Score:3, Informative)
Bugzilla is cheap (free) and good, but even more importantly, you can see how other groups build a cradle to grave corrective action process around it. My advice to you is to give your boss a tour of the Mozilla bug tracking **process** with their copy of bugzilla as your window. Set up bugzilla on your own machine and play-test it by re-entering a few of your own bugs into it and mock-working them.
use talkback (Score:3, Interesting)
Don't kvetch endlessly about specifics in the interface, just say "it needs to be easy and fast to report bugs. average time to file a bug report (now): xxx minutes. goal: xxx seconds."
Re:use talkback (Score:2)
If you make it easy for users to file bug reports they'll do a lot of the work for you. Yes, someone needs to remove duplicates and separate out the feature requests but the bugs that really count are the ones that your users are seeing.
If you're looking to exterminate crash-type bugs, you might check into BugToaster [bugtoaster.com]. The basic data is free for the taking on their site, but detailed reports will cost you.
bake the error capturing right into the app (Score:2)
While this is no silver bullet, if it's a web-based app, it should be trivial to capture the variables and pass them into your form. If it's not an error than can be caught by the system, how about a function on every "page" in your app that just grabs the relevant variables, making it easy for users to cut and paste. It will be easy to filter them if they are all somewhat standardized.
Pitfalls of The System (Score:2)
No one disputes that tracking bugs and fixing them isn't an important thing to do.
Where there is contention and differing points of view is when it comes to specific bug tracking tools.
Many folks, like me, resent heavy, overly bureaucratic tools. You know the kind: everyone has "roles" and there are authorized bug managers, assistant deputy bug fixers, project manager, etc., and every single keystroke about fixing the bug in the code requires 10 corresponding keystrokes of entry into The System.
The System sells well to managers because they get shown screen shots displaying exactly who is doing what, when, and how the bugs are being managed so incredibly well. Wow! What detail! How impressive!
The managers buying the system don't actually use it, though. Their staff do. And the staff suffers and learns to avoid The System at all costs so they can get some real work done - like fixing bugs for example.
Make sure that you evaluate potential bug tracking systems with the actual people who will be using it.
Other plusses to look for are a simple linear learning curve. Avoid the ones where you have to upload your brain with 4 days of training classes in order to use The System.
Tell the managers (Score:1)
Re:Tell the managers (Score:2)
For instance, most consumers aren't worried about buggyness, as long as its not terribly buggy.
However, aerospace kinda likes bugless code.
more details (Score:2, Informative)
In no particular order:
The system in use today is an internally developed tool. It is a web form, but it so completely fubar that most of the constructive suggestions about automated submission do not apply. We have tried. Thanks for sharing your ideas.
To expand on the fubar-ness of the tool -- as you fill out the form to create a bug, several fields will submit a request back to the server each time you change their value. There are at least 8 such fields on the entry form. Each roundtrip lasts ~3-5 seconds. When you save a bug, you're waiting ~15-20 seconds for it to save.
the quality of bugs entered is a problem, but not the primary problem. A bigger issue is that bugs don't capture information that people have about the bug, because opening, editing, and saving a bug is a major hassle. Thanks for input on this matter, but I feel it's mostly under control.
Bugzilla... I only wish we could be lucky enough to use it! When arguing to my managers for bugzilla, I have found huge resistance to the idea of using an open source tool. Which is disappointing, because we use plenty of other such tools internally (ant, JUnit, log4j, xerces, etc, etc)
putting up my own tool seems to be of limited benefit. There are too many people entering bugs (QA, developers, customer support) to generate a groundswell of support for anything other than the tool in use today.
testing is not a major problem. We have a great set of tests to cover almost every bug we've encountered.
What I'm trying to do here is convince someone that people need good tools to do good work. I expect that to be a truism, but it is not being accepted as such. Does anyone know of any kind of study showing a change in bugs filed following the introduction of an easier-to-use bugtracking system?
Re:more details (Score:1)
Just a suggestion, based on similar experience with bug tracking software and getting management approval..
Part of the problem might be that its open-source software, part of the problem might be that it (if you consider Bugzilla) written in Perl, a language that your managment may not be comfortable with..
I'd suggest you test the waters with Scarab [tigris.org], implemented using servlets.. that might go down better..since you obviously seem to develop using Java (based on the other open source tools you cited)
If you are sure the choice of language isnt the issue, I would HIGHLY recommend Mantis [sourceforge.net], a simple, VERY useful bug tracker written in PHP.. I use this myself, and its very small and fast to configure (only problem I have with it right now is that it can only use MySQL as a source database)..
Sometimes, the easiest way to get bug tracking software in a large org. is to install it and start using it personally (ie: for bugs related to modules that you develop).. People will gradually come to appreciate exactly how much easier it is on your system as opposed to the other thing, and there will be a creep towards an easier to use system that cant be ignored..
Sometimes, the persons in charge of decisions like this simply dont know how much easier bug tracking software can be than your present software, which is what holds them back :D
Hope this helps and good luck
Try looking at Roundup? (Score:1)
When i started working as a developer at ILM [ilm.com], there wasn't really any bug-tracking system in use. So i threw together a quick hack, on a weekend when i finally couldn't stand it any more (isn't that how so many software projects get started?).
Over the two years i was there, it grew, but it stuck to three core ideals:
To meet the first two goals, the system was based on e-mail: anyone could just send a free-form e-mail message to the roundup address, and a new bug would be created; then anyone who replied to messages about the bug would be automatically added to a mini-mailing-list. Every bug got its own automatically managed mini-mailing-list.
Most other bug trackers are really bad at the third goal: they take up all kinds of space with management details that people don't really care about. What a developer really wants to see is the descriptions of the bugs, so Roundup maximizes the screen space for that.
It grew far beyond my original plans, in terms of the number of users and bugs logged. The back-end implementation was terrible (no database, just lots of little text files; performance was awful but it did have the redeeming factor that you could just use "tar" to archive them).
As far as i know, ILM R&D is still using it today, and they're running multiple instances of it to support different applications and teams.
You can get the source code and see some screenshots on a page about Roundup [zesty.ca], though i'm not developing it any more. A new group of people has picked up the torch and carried it on in a Sourceforge project [sf.net] that is alive and well. Their project is a complete rewrite of Roundup, originally based on a design document I wrote, but now much extended. I encourage you to check it out.