When Should You Go Back To The Drawing Board? 177
Prozzaks asks: "As junior developers, one of the task we will likely be called to do in a company is system maintenance (updating, bugfixing, adding functions to a existing system). I've already had a situation where it was necessary to do some maintenance, and after working a few hours on the system, I realized I was redesigning it from the ground up. The problem is that companies don't want to allocate the necessary resources to redesign a system. What are we to do when we discover code in the companies products that -really- needs a rewrite and management staunchly refuses to realize this fact?" There are times when I can understand the desire to go in and rip out code that you feel isn't operating as well as it could, and there are times when you have to sit back and weigh the benefits of a rewrite against any possible gains. The adage "If it ain't broke, don't fix it," applies here, but there are times when badly designed code that works now, will break later. Under what situations are rewrites necessary, and how can you get management to understand the need if it arises?
Um, Slash. (Score:1)
Remember Netscape?? (I use IE now) (Score:1)
Example: NETSCAPE
From almost 100% to almost 0% or less market share in the windows browser market?? Sounds like death to me.
Assuming your Senior Programmer are Still There (Score:1)
Assuming you have one of those.
If you don't, it means that eventually that huge pile of cruft you are working on will enter either the overbloat or bit rot stage and you need to get the hell out of there.
Good luck,
Jim Burnes Sr. Software Engineer (20 yrs experience)
Re:Immediately! (Score:1)
Don't forget to write a JVM in MIPS assembler then run the JVM on a MIPS emulator written in COBOL.
This design is called the "phux0red" pattered by the Gang of Four.
Almost. (Score:1)
People always say, "If it ain't broke, don't fix it", but this is not right. The saying should be, "If it ain't broke, and isn't breaking, don't fix it."
That only half-applies here though. The question isn't about fixing (if you interpret this as incremental), it's about throwing it out and redoing things. But if something is breaking---that is, not going to fulfill future needs, or not going to scale properly, or going to present problems---it needs fixed, and that's where this question comes into play.
Sometimes, an existing system is so poorly designed or half-implemented such that doing any sort of maintenance at all would take more time than redoing things. I've seen a couple of these, and the choice seems obvious. You reimplement. Right? Maybe. You need to determine whether you have the ability and resources to reimplement the system. Junior developers can easily underestimate a task, and this can end up wasting a lot of time and effort, which translates into money. Before you even think about suggesting this, you need to be sure you can pull it off, or you'll cause problems for everyone in the future (including yourself).
If you can do it, then talk. Management understands dollars-and-cents. That's what they're there to get. That's what the company is there for. Translate maintenance on the existing system into costing more than reimplementation. This is what they care about.
Re:The problem with this approach (Score:1)
Indeed, often when I'm doing the incremental rework. I know very well where I want to end up in the end. It's just that ending up there after restarting from scratch is not always feasible within the constraints imposed by real life. What follows may be new to some of us out here, but software that is actually worth a rewrite tends to have active users who want bug fixes and new features now, not 2 years down the line. Maintaining the current version may also be what keeps the money coming in that is supposed to pay for all the rewriting.
I agree that clean code instantiating a good architecture is the max. But almost clean code instantiating an architecture that is actively on its way to become good is still a lot better than one pile of obfuscated dung that only pretends to work, and a pile of paperwork documenting a new and shiny system that didn't get more than half implemented before real life intervened. Of course, some systems are way beyond rescue. When I run into one of those, I too call for a rewrite and otherwise refuse to work on them.
--
Re:The problem with this approach (Score:1)
Anyway, if you're good enough a programmer to recognise the badness of the old design and come up with something better, the risk of ending up with nice and shiny code trapped in a bad old design isn't all that large, IMHO. If you're not good enough, you're doomed anyway. Even if you restart from scratch.
Also, if you have a working program to start from, or at least one that does at least some things right, gradual changes can be quite easy to apply and debug, even if they're design changes. This has the nice effect of providing a stream of small but nice successes that both keep you going and can help to convince management that it's worth investing some more and that you are capable of dealing with that. You do have to be willing to compromise and eat the cake in somewhat smallish bites, though. Don't shoot for the big ultimate design from the start. Instead, go for something that is 1) better than what you've got; 2) reachable; and 3) a suitable starting position to move closer to your ultimate goal at some later stage. Extra advantage: if, for whatever reason, you have to abandon the rewrite before it's done, your legacy is much more likely to still be usable.
One somewhat nasty but potentially also very big advantage is that the gradual approach can help to get the work done even if management is all PHB and doesn't recognise (let alone have) a 10 foot clue if it hits them in the face. Basically, you don't have to tell them all the gory details of what you're doing (which is not the same as lying about it). Of course, not all company cultures will allow for this.
I've done all this several times, both at work and in my spare time. So far it always worked.
--
Re:Tis easier to get forgiveness than permission (Score:1)
Re:If management refuses... (Score:1)
I wrote a rather sizable bunch of mod_perl modules, interfacing with a mysql database to keep track of a partnership system (pay-for-clicks/signups/whatever). Now, the company that runs it is quite small, and has limited resources (one db server, one web server, that's it). The first week, everything worked fine. The second week, everything still worked fine but it got slow. The third week, it took so long to load a page that you could get yourself a frappucino at the nearest Starbucks.
Was it a design mistake? Not quite, it had more to do with MySQL not being able to deal with 50+ queries per second. (AFAIK
That's a pretty clear-cut case where code that isn't broken is in definite need of rewriting.
If you encounter code like that, and you are confident that you know what it's supposed to do (learn the code first, then fuck with it - don't fuck with it and then learn about major fuckups.) then you go and fix. If you don't know what it does, or what it's supposed to do, it's best to leave it alone for a while.
Now can someone pass me my caffeine IV? My 48 hours-of-being-awake are catching up to me...
Re:Rework = failure (Score:1)
Bah
Re:Even senior developers face managerial headache (Score:1)
I got that way after too many bad experiences in $VBC. I guess on an experience level I'd be considered a junior programmer still; since there is a lot of things I have yet to learn, but dammit, I do know when something's broken or not, or is in need of some work to either make it work better, faster, or just make it more readable so the maintenance bit is easier.
(Ok so this topic is something I can get all fired up about.. sue me
refactoring (Score:1)
Will a rewrite actually help? (Score:1)
Even if the code is bad, if the problems are deeper than that, any rewrite will be a waste of time.
Completely disagree. (Score:1)
Now when I say Refactoring, I'm not talking about "heads down, that's all I'm doing". I'm talking about Martin Fowler's view of it -- every day, a little bit at a time, to clean up the code and make it more readable / maintainable, while you're adding features or fixing bugs.
I think your exposure to development teams might be rather limited to think that there are always "maintainers" and "designers", and that maintainers are lower on the rung. I've seen two situations to counter that: one where the maintainers were *deliberately* the more talented bunch of programmers, since they have to deal with tougher situations than those with a "blank sheet". The other situation (far more common) is that the core coders also do the maintainence until they have a replacement.
Management does not hate refactoring, if it is done properly. They hate *rewrites*. A *rewrite* requires management decisions because it is in the realm of business - it is about throwing out the existing system. And it rarely works the way it's supposed to.
Re:Almost. (Score:1)
"Electric Relaxation" - ATCQ
- Bwana
Even senior developers face managerial headaches (Score:1)
Within days of being Hired I was asked to work on a project I will call 'NOX3' (changed to protect the parties involved from litigation - Seriously!) - this project had been ongoing for almost 2 years by the time I was brought on. All of the original developers had left the company - and just two months before my arrival a 'needed' type of join had facilitated the change from utilizing mysql to Oracle as the backend DB.
As most of you may or may-not know, Oracle can perform very well - when the DB is archetected correctly with its relationships and indexes. As it turns out, we had seen a 260% DECREASE in performance switching to Oracle: this was because the developers involved (by the time the switchover took place, an outside development company was also involved) did not want to rewrite the hundreds of queires used by the system and so copied the table structure over to oracle with no schema changes, no added indexes, or anything. Due to the 'flat' nature of the schema, table-scans were common-place, leading to horrible performance with the full 800k+ rows in the main table. Linux's max-file size (2GB at the time, thank god for 2.4!!) actualy held us back by forcing indexes and tables to be spread over multiple files, which also affected performance...
Anyway, I'm getting off the point. As I said, within days of being hired I was put on the team of developers working on the project and told to 'see what could be done, a silver bullet would be nice'. Well, the deeper I dug, the more I realized a silver bullet didn't exist - the app was horribly archetected, badly (= none) documented, uncommented (180k lines of code, only 1k lines of comments!!!!!), hampered by the DB schema and even by the choice of technologies used to implement the system. To add insult to injury, the team of developers (now 'mine') inside the ISP were just running support/bugfixing, because the customer was doing field tests of the 'beta' (not my choice of term) copy of the app live, and the external development company brought in to consult was working on adding features (to the live beta app, mind you).
I told my managers that the project was a write-off at best, and that if they ever expected to get near the performance the client wanted ( we were two orders of magantude two slow), and support the number of simultaneous users they wanted (5000 active users, with peaks of 15k+), we needed to start a re-design imediatly before things were even further behind schedule. (the app was 13 months late... I'm not kidding)...
I wasn't listened to - the manager nodded his head and passed on the message, but nothing happened. So I started giving him reports with all of the horrible things I was finding, things he could show his managers - they didn't listen to the reports. So for two months I fought just to get permission to spend actual company time running benchmarks/tests of the system to prove my point... I ran the tests and boom - the system Crashed (not slowed down, actual crash) at 48 simultaneous users - crashed so bad it took down the machine the app was on (but not the DB machines). Finally I had my boss's attention, my findings were vindicated...
We never got to do a rewrite, instead the project degenerated into legal wrangling complicated enough to confuse OJ's lawyers. The consulting company claimed it wasn't thier responsability to comment on the quality/capability of the app - just to add features - meanwhile my company claimed that they should of performed 'due dilligence' in making sure the app could perform as required, and they should of opened thier moths way before I blew the whistle, meanwhile the customer went after both my company and the cunsulting company, and my company brought in a second consulting company to verify/audit my findings and gather evidence (this took 4 months, meanwhile we sat almost idle).
My team was dismantled - all of the developers on the team (except for myself) were hired with specific skill sets to work with the technologies the app required, and thus weren't needed any more (this was 2 months after the 4 month audit was completed). The original manager who started the project was fired for 'other reasons', and my manager, who came on board just 4 months before myself, was also let go. All of this firing and such took place 8 months after my hiring. A week after my hiring I had determined (and was already telling my manager) that the system needed a complete redesign, and it would take 6 months with the right design and current team, to do it right (no contact with the customer would of been allowed, along with no testing, we needed six-months of heads-down coding time). Instead of listinging to the person they had hired as thier 'problem solver' and senior developer they drug thier feet and then dismantled the project, but paid everyone for 8 months work along the way.
So which makes more sense, 8 months of paying everyone involved just to end up cancling the project and then get into costly legal manuvers, or six months of paying them to deliver a working app to the customer, late (which, surprisingly, the customer was fine with, as it turns out - time wasn't the issue, functionality was)? I don't claim to understand it, and I still don't know why they did what they did - my point is, asking for a re-write can be like wishing to win the lottery - the powers that be can work against you.
Re:I'll bite. (Score:1)
Well, how about a hard, real-world example. Granted, this is a "toy" by many standards, but it was definitely a case where I applied the 50-50 rule. On the plus side, we do value documentation in our shop.
Some of the code I was inheriting contained this gem:
short *arr_ptr[8];
for( j = 0; j < 8; j++)
{
arr_ptr[j] = (short *) malloc(1 * sizeof(short *));
if (!arr_ptr[j]) printf("\n Unable to allocate space for ln_buffer\n");
if (!arr_ptr[j]) exit(2);
}
for ( i = 0; i < ish_y_dim; i+=2)
{
iptr += 2*ish_x_dim;
if (iptr > xend) iptr = xstart;
ptr = iptr;
for( j = 0; j < 8; j++)
{
arr_ptr[j] = ptr;
ptr += ish_x_dim;
if (ptr > xend) ptr =ish_image;
}
}
Now tell me, why's that malloc() there and where's the memory going? That entire first loop has no reason to exist! As it was, the rest of the code was also overly complex and none of it was doing what it really should be doing. The stuff it was doing it was doing in a rather tortured way.
So, I threw out that function entirely and wrote a much cleaner new one. Right now, I've got some other code I've inherited that's a bit larger than this, and I really, really want to rewrite it. Fortunately, though, I'm not being asked to maintain it -- just run it. Since I don't have the time to rewrite it, and I'm not charged with maintaining it, I think I'll just let sleeping dogs lie.
--Joe--
Re:Don't change it immediately... clean up in part (Score:1)
The 50/50 rule (Score:1)
If it looks like its gonna take me at least as long to understand whats there as it is to rewrite it, I rewrite it.
Re:Refactoring and Maintenance (Score:1)
HOWEVER, that is subject to being able to *test* the results to make sure the software produces the same results as before. If you can't test it, you have no way of know that you didn't break something - in which case it does make sense to make the minimum possible change to fix a bug/etc.
Pretentious Coward Embarrasses Self (Score:1)
I don't value a person's opinion until he or she learns the basics of grammar, spelling, and other elements of good writing.
Your condescending tone makes your little error stick out like a wart on a bald head, pal. (Otherwise, I'd be too polite to mention it.)
GAY HOMO LOVE (Score:1)
TEH SALSHDOT SUKX0R-B0T0RZ!!!!!
Evolution not revolution (Score:1)
Even if a program looks revolting, in the main it is better to live with it rather than fully rewriting it. Remember if you rewrite it, then in theory you have to redesign, recode, TEST!!!, run pilot trials etc, all without interrupting the end users who are probably earning the company the money that you want to spend on the redesign.
Code for consumer products is even more fraught with difficulties; if you recode something and get it slightly wrong then you risk a product recall. Also consumer products generally only have a short lifecycle; if the product you want to redesign is out there its probably obselete, and therefore spending money on recoding is a bit pointless; you should wait till you get promoted to senior developer and work on the next generation, where a cleanup of the code can be absorbed more easily.
Run with it. (Score:1)
As soon as I saw this happening, I just said fuck it and started recoding everything that had been written. I didn't talk to management, I just did it. Granted your project is probably a lot larger than what i'm working on, but if it's really needed you have to go ahead and do it. I moved from a shell/perl/textfile mess to a database backend and i'm going to be adding support for realtime stats. I work on this in my own time and without management's approval. Sometimes you just have to go balls out and drive things to respectability.
Don't touch the interface, little one.... (Score:1)
Case in point: any Microsoft product.
If you are given a task of optimizing a routine or fixing an array over run - do it. That is your job.
I am betting Defect #12332 in the bug queue doesn't say "redesign entire system (by person with 2 years professional experience)."
Senior programmers have the job of revamping the design.... managers have the job of deciding to do it.
Remember, there are people at your company who are buying baby food with the pay check that this "horrible" software is generating.
This isn't to say "keep your mouth shut and toe the line" - definately point out where improvements could be made - but when given a task stick to it.
Most junior programmers start such rewrites b/c they haven't taken the time to understand the code.
Re:Um, Slash. (Score:1)
Nope. I think he was asking Slash from Guns 'N Roses.
Software Lifecycle (Score:2)
Do what I did (Score:2)
Bill - aka taniwha
--
If management refuses... (Score:2)
My view on it; if you think that code is going to break in new and interesting ways in a few weeks/months - fix it anyway. Do it on your own time if you have to, but keep track of how much time you do spend on it. Fix it on the side, but don't actually implement the fix. yet. Wait for the original code to break, watch management panic and basically walk in and say "i can fix it in (2*the-time-it-really-took) [days|weeks] -if- i get time, resources, and oh.. some days off".
first of all you already have the code waiting so you can now legitimately read slashdot all day (doing research), and you got your name in the good books - not to mention you're on your way to having a reputation as a miracle worker.
okay so some of you might not agree with this approach, but it's worked well for me, and it's a good way to pretty much tell your manager "you suck" without getting your ass kicked. (besides i like watching people squirm and having to explain to -their- bosses why stuff broke in the first place).
Fred Brooks and other random wisdom. (Score:2)
The real trick is distinguishing between code 'that is bad', and code 'that I think is bad', usually because I'm not used to how it goes about solving the problem. Learning to figure out what someone else's code is doing is the best skill you can aquire early in your working life. You'll be doing it a lot.
Re:Linus himself stated it (Score:2)
I believe this means he wrote a really simple multitasking kernel that switched between two processes, compiling it on Minix but not using Minix code. Then he wanted to turn that simple 2-process kernel into a terminal emulator, which raised the question of drivers, and it was on its way to being a real kernel. (If you look at the Linux-activists archive [195.92.249.252] (warning: 600K archive), early Linux seems much cruder than Minix, something which wouldn't have been the case if Linux == Minix being rewritten piece-by-piece.
One acronym: CBA (Score:2)
I have found (at least in my situation in a large corp.) that in order for management to approve anything done, you need to give concrete dollar amount savings for anything you might feel is the "right thing to do".
I know this is hard sometimes to quantify, but in my experience, you can go on for hours with reasonable/logical arguements about why something should be done.And after all of this you get one question, "That's nice...how much $$$ will this save us over what we have now?"It's not so much the savings, but the immediate costs that are taken into account when making the decision whether to pursue something like this.
A way I got around this:
I recently had an idea for my immediate team that would save a few thousand dollars a year in production support. Instead of presenting it to my manager and listening to them saying that we can't afford to spend time on developing it now (usual B.S. from clueless managers), I just went ahead and starting working on my idea in those spare moments during the day, 15 minutes during lunch, etc.I presented it to them saying that this cost them next to nothing (since it was 90% done, and I wrapped the time into other tasks during the day), and it would be ready to implement in a matter of a few days.
You can try to take the credit early on for an idea, or you can work in your spare time to prove it can be done, and that you are willing to make it happen.The latter is what (whether unfortunately, or not) is what the suits will buy into.
Just one person's experience...
Rework = failure (Score:2)
"If you code monkeys knew what the fuck you were doing in the first place you wouldn't be whining at me for more time. Go back and fix the shit you built as all fuck to begin with. Move!"
Whereas if you fix something, even if what you do it a piece of shit drive-by chunk of crap, then you're a fucking hero !
The customer won't notice or care about the difference and if it runs like shit blame it on the hardware or the sysadmins or the crappy db design. Or the H1B that touched it last.
Plus if you patch vs. rebuild you never have to hear from someone else that what you did was a horrible design.
That's what I would do if I were cynical but of course I'm not.
I'll bite. (Score:2)
You can understand its function without understanding its implementation. For instance, if you program C, you know pretty well how "printf()" is supposed to work. If you then go open the hood on the printf implementation with intention of adding something to it (say, a new % operator) and discover it to be the biggest pile of rotten bits you've ever seen, then the 50/50 rule applies.
Often, it's much easier to figure out what the interface to a black box is than it is to understand the contents of the box when you open it.
--Joe--
Re:You have three options: (Score:2)
-- Seriously, though - it's very hard to rewrite a program from the ground up; somehow, it's never as good. I'd clean up the code, but not rewrite.
- This statement is wrong, just plain wrong. The axiom in computer science is "Don't fix bad code, rewrite it.
May I ask where you studied computer science? Rest assured, that is not what I've learnt. I didn't think computer science had an answer to this at all, and if it had, I would guess it would be something like "Interesting problem, let's conduct a study by doing both...".
The fact of the matter is, the field of computer science doesn't really care, but the field of software engineering would. (If the algorithm was put into code form, that's generally good enough for computer scientists)
Woz
It's a question of priorities (Score:2)
You also have to look at time to market. If your competitors will get to market in two months, then maybe it's better to get there at the same time with ugly code, rather than be a month later, even tho the later code could be enhanced and maintained much better than the competition.
Best you can do is work out the estimates, throw in priorities and market considerations, and take your best shot.
Try various degrees of cleanup. Maybe a moderate cleanup will take 2 weeks and double your efficiency. Maybe a thorough cleanuop will take two months but only quadruple your efficiency. Maybe you do the simple cleanup now, the fancy one six months later.
Compare it to freeway vs city streets. For short trips, the extra overhead of getting to and from the freeway doesn't make up for the longer travel time. Sometimes fast city streets are a good compromise between freeway and slow streets.
--
Check the metrics (Score:2)
It is definitely worth retooling code that is awful if
--G
Refactoring and Maintenance (Score:2)
1- If you are maintaining code, you should not be refactoring code. There should be 'specific' bugs that you are fixing. Dont do more than this. If you do more than this, the additional changes must be tested, documented, etc. This is not the purpose of maintenance.
2- If the code is that bad, then propose refactoring to management and let them consider and delegate. It is highly unlikely that the developers doing maintenance will be the ones who design the refactored implementation.
Management hates refactoring because they have to justify and fund a project to provide the same functionality which currently exists (although in a much less glorius form)
Development entails not just coding, but testing, integration, documentation, etc, etc. The code part of the project is the smaller of the whole, which is why management may be relunctant, even if you say you can code the changes in a few months.
Re:One Tip (Score:2)
The "particular function" in Netscape's case was the whole renderer. CSS had to work (not JSS and a bridge as in the case of NS4), DOM had to work (W3C, not layers as in the case of NS4). Even plain ol Netscape inventions such as tables had overhauled to work right. And. once they had committed to rewriting the renderer, it's understandable that they decided to shuck the other things that weren't making people happy (such as the Motif UI under Linux). What's unfortuante is that they didn't start the rewrite back in 1995, like Microsoft did with their Mosaic fork.
--
testing bottleneck (Score:2)
The upper management doesn't believe in hiring
lots of testers, and people don't like being
full-time testers for long.
Code redesign has to be aligned with testing resources.
Our company has a ratio of 1:4 testers:developers.
MicroSoft advises 1:1. However they have a
hundred as many customers as we do.
Called "refactoring" in literature (Score:2)
In our yearly release cycles we try to get
in a months worth at the beginning of cycle.
Sometimes deceive the project manager and do
it under the guise of development.
A design cycle might be:
(1) Specification: one month
(2) Development: four months
(3) Testing & bug fixing: four months
(4) cleanup, shipping and vacations: one month
Sneak in refactoring between stages 1 & 2.
Re:Don't change it immediately... clean up in part (Score:2)
Also I often write my own versions in my spare work time or in my own time and make my own version. Then you can compare the two versions and if yours is better offer it to your boss to try. If he doesn't want it then keep it for later, he may want it eventually or you can reuse parts. If you wrote it in your spare time then you can make it opensource and/or sell it yourself.
It's a management decision (Score:2)
It's _their_ software, not yours and _they_ have to decide what to do with it.
_____
Rewrite it. (Score:2)
One of the first things it tells you to do is to come to work every day ready to be fired. I believe in this. If you're fired for differing with your boss on the grounds of a technical topic that you are the expert on and he is not, then you're doing yourself a favor, and it will not reflect negatively on your career.
Another is that there's a difference between saying and doing, and doing should be done underground. Rewrite it without telling your boss. He can't appreciate the importance anyway, so there's no point in making a soap opera out of it.
We can all advise you on the importance of not mucking with code because so many newbies have the unquenchable desire to rewrite, but we're stereotyping you and don't know the actual score. You're the one who has to to deal with that hairball, so you know better than any of us what needs doing. Follow your instinct. Whether you're right or wrong, I gaurantee you that it will be a valuable learning experience.
Show them what it costs. (Score:2)
Project both costs against the short term and the long term. What short and long-term constitute depends a lot on your company. I would say 3 months vs 18 months.
Carmack does it... (Score:2)
I'm a big fan of keeping parts of a system small and isolated, so they can be completely rewritten when needed without bringing the product down.
-m
Here's a good rule of thumb (Score:2)
Focus your efforts (Score:2)
However, this doesn't mean you can't do anything at all. I consider a couple things fairly safe:
eliminate all compiler warnings. Turn on all compiler warning options and either eliminate them or document (in the code) why they're not a problem. I can usually clean up over 2kloc/day and while 80% of the warnings are trivial (e.g., missing prototypes), a good 5-10% are latent bugs. Variables initialized with incompatible data, functions called with the wrong types of parameters, etc. This sounds incredibly stupid, yet I've repeatedly found "fragile" code becomes much more stable once compiler warnings are eliminated.
close memory leaks. Run the software under an instrumented "malloc" package that detects memory leaks -- and ensure that your code always releases all of the memory it allocates. (You might not be able to release code allocated by libraries you use.) There may not seem to be much value in this, but my experience has shown that the effort required to do this always results in improved code. You should also practice "data hygiene" if you do this - clear the data structures before deallocating them, so it's clear if you subsequently attempt to access a freed object.
An alternative approach is to profile the code and verify that there are equal numbers of fopen() and fclose() calls, equal numbers of malloc() and free() calls, etc.
profile the code and find the hot spots. Profile the code and look for the routines that are called most often. Does it make sense, or is this an indication of a particularly inefficient algorithm? Sometimes the code is written one way because of the limitations of legacy hardware, and now it's just a pointless waste of time. (E.g., I've seen routines that opened a file, read each line until the Nth line, the closed the file and incremented the counter! That might have been necessary 20 years ago, but I replaced it with a bit of code that mmap'd the file into memory and then did some pointer magic and shaved 25 _seconds_ off the startup time!)
prune dead code. Run the code through a call-tree routine and comment out all procedures and global variables which are never called. (I find it best to bracket the code with #if 0/#endif and tag each line with /**/ in the first few columns.) N.B., call-tree software can't find functions called indirectly through data structures! Repeat the process until you've found all procedures only called by procedures which are themselves uncalled, etc. Consider moving routines to other files, if it will allow you to make it static. I've found up to 2/3 of legacy code is now dead, and understanding the bit that remainds is *much* easier. Do not delete the "unused code" until a formal rewrite, however!
After all of this is done, the code may still be a undocumented ball of spaghetti, but it should be sufficiently organized for an intelligent decision whether it makes sense to rewrite subsections.
Been there (Score:2)
The initial design was done by a moron, who is just out of Uni and who read too many UML design books, but never seen/written a real program. The code was
- over engineered
- threaded unnecessarily (java threads are easy to program)
- with layers and layers of abstraction.
- and ofcourse BUGGY.
He left and I got to be the poor bastard to support / maintain it. I hated his guts while doing this. But I have become quite an stealth programmer. I have learnt
- a lot of Java thread issues
- how to work around design flaws
- how to do a 'transparent' hack
I have lobbied manager for rewrittnig it. But never got the permission b/c of 'it is too big already / we cant justify the time to rewrite something within 6 months'.
FInally one of our customer who was using this program threatened to pull out of the deal if that component didn't do what it supposed. It took me a week alnighters to get that done. And every day every one from my manager to VP of Eng were on their toes!
After that I had permission to rewrite/design 'parts' of the program. So it is kind of a background process. During my design
- KISS (Keep It Short & Simple)
- avoiding all the pitfalls I have learnt
- unix philosophy (doing one thing & doing it well).
I also have a second programmer assigned, so design/coding decisions are analyzed well aswell.
In my opinion version 1.0 of a program should be simple and v2.0 can be a redesign considering all the user feedbacks.
And the best thing I have learnt is 'you get a design right, after you did that wrong'
LinuxLover
Tis easier to get forgiveness than permission (Score:2)
Give up (Score:2)
Simple. Do what I did. Give up. Spend all your time reading slashdot, working on pet projects, and learning cool new technologies that you might actually be able to use if and when you find a new, better job. I have literally refactored and rewritten this project I've been participating in several times. But I never have the guts to actually show my work, since it would obviously reveal my opinions of the current codebase. In contrast to the recent Ask Slashdot about finding beautiful code, I suggest you learn to live with unconscionable quagmirical morasses of code. Just put it out of your mind, or you will never get any sleep.
Compare the code to the US water & sewage systems (Score:2)
Just earlier today Reuters reported on incresingly shoddy water and sewer pipes [yahoo.com] across the country. Basically, some of the major cities are running on piping that's a century old. St. Louis' pipes predate the Civil War. Nobody wants to spend the money to fix them.
This quote from the article relates to code as well as water pipes: "Is the sky falling? No, the sky is not falling today but unless people start taking this more seriously, within 10 to 15 years the cost of addressing these problems is going to be even higher. It's not an issue you can ignore for long."
Read McConnell and SEI (Score:2)
read bout it (Score:2)
Understand it first (Score:2)
You mentioned that you are Junior developer.
Many re-design efforts start from not being
able to understand how system works.
Re:You have three options: (Score:2)
True, but on one serious condition: The rewrite is done by the same person(s) as the original. When rewriting someone else's code, you are will have to map the other guys mental model from his source, form your own map of how things should be, and transform that into code. Much less effective than rewriting your own code, where you can reuse the mental map you built the first time, especially the parts you realized when it was too late to change, and had to kludge your code to work anyway.
Re:don't do it. (Score:2)
At the very least be aware that "here there be dragons". More often than not when I run across some piece of code that makes me exclaim "What brain-damage is this?!?" I later discover that some other bletcherous mess is depending on that particular flavor of brain-damage.
If a piece of code has a tight enough interface specification that you can actually get away with the 'school-taught software-engineering, it's a black box we change the insides at will' mentality -- then chances are that code isn't crying out for a rewrite in the first place.
Re:The 50/50 rule (Score:2)
And if you don't know what the code's supposed to do, how can you recreate it?
--
How much do you really care about the project (Score:2)
So my advise is to only "stick your neck out" and "go the extra mile" if you really care about the code or project. If it feels like someone else "owns" the project and you're not in control of its direction, advocating and persusing a re-write is probably unwise.
Re:You have three options: (Score:2)
One Tip (Score:2)
Doesn't sound good (Score:2)
If you've only maintained the system for a "few hours", I doubt you really understand the system well enough to pull off a redesign and rewrite. Not to mention that it'll be a hard sell to management.
I've seen new hire after new hire come into a company, look around at the steaming heap of code, and within a week want to rewrite it all. The best fun (as their underpaid "peer") is being supportive and getting them to go ahead and try it. It's fun watching wiseasses hang themselves.
On the other hand, almost every time I've done a rewrite like this, I kicked myself repeatedly for not doing it sooner. The only times I didn't? When I was that new wiseass that didn't really understand the system. I was slowly sucked into a downward spiral of rewriting more and more pieces until I scrapped it and hauled out the duct tape and chewing gum.
If you really want to pull off a rewrite, just go ahead and sneak it into your schedule. Don't bother trying to get management to understand the issues. They don't see things like you do. They're almost a different species. They figure once something is written and works, it's finished.
Re:What code are you talkin about???? (Score:2)
Re:Remember Netscape?? (I use IE now) (Score:2)
Another adage... (Score:2)
Actually, the adage "If it ain't broke, fix it 'til it is" works better here. When fixing it to make it do what you want breaks it, you need to redesign it. If you can fix it without breaking it, keep it.
It seems to me that your management wants you to keep fixing the system until it breaks, at which point you're allowed to replace it. If you honestly cannot fix it to work, rewrite it.
Some things to remember when rewriting, though:
For the love of God, don't rewrite! (Score:2)
At this point in my career, I believe quite strongly that the only time you should rewrite is if you are changing to another language. Even then translation is often a good idea.
If it is in the same language, make it better. Is a method too long? Find some lines of code that represent a task and move them into another function. Would this one really big procedure work better as three classes? Peel off a class at a time. Refactor the heck out of it until you are happy. Test often. Code a little, test a little. Before you know it, that Pinto will start to look like a Porsche. At the same time, you'll keep all that knowledge that someone worked so hard to put into that code.
Refactoring isn't a new idea. Martin Fowler wrote a book about it and there are some sites on the internet if you look hard enough. The important thing about the book is that it is liberating. It was the first thing that ever suggested to me that I might try fixing bad code.
Rearchitecting might sound like the best answer, but be brave. Software engineering is often about maturity. Someone who always says "if the company isn't right then find another company" might be missing out on something. If you can help your company success, you just might find yourself succeeding.
Re:Been there (Score:2)
If you ever meet the guy, you should probably thank him. Think how much you learned because of him!!!!!
__________________
from the inside out? (Score:2)
oh, you meen like Win9X?
Keep the old (Score:2)
Was is *really* important to throw out the ftp code in netscape that understood all 60 types of servers to connect to? surely there was something worth saving.
Yes, spaghetti code is nasty, but if developers spent more time reading the code they have on hand, re-use would be more prevalent.
An opinion is simply that, and I'm entitled to mine.
If you feel baited to flame me for a perspective that differs from your own, well then, that's more about you than me.
A host is a host from coast to coast, but no one uses a host that's close
You don't! (Score:2)
Management doesn't usually listen to a junior engineer who says the system needs to be redesigned. Chances are they have a senior engineer who is already saying something and trying to echo it up to an Architect or CTO who has some sway.
That isn't a nice thing, but it is the truth. Lower management wants to avoid the ire of higher management. Generally they are going to say something to the effect of "Just get this one item working" because just that one piece will take a fraction of the time it would take to redesign the system. Does it matter that they are going to lose the time taken to redesign and then some over the course of months? No, it doesn't.
Ultimately there is someone who wants the software and they want it now, and they are not going to tolerate having to wait two months while the infrastructure is refactored. You can show upper management all the supporting evidence you want, but if doing this is going to seriously impact their short term cash flow (or any number of analogous tangible items) then they are going to want to hear how a bandaid fix can be implemented.
Yes it sucks, yes I live it. One skill that moves you up that Junior to Senior ladder is your skill at finding interesting bandages to treat a sick project.
When to redesign... (Score:2)
My karma's bigger than yours!
Re:When to rewrite? (Score:2)
Re:read bout it (Score:2)
"that can actually put you behind because the new addons need time to catch up with the rest"
Yeah and sometimes these new "junior programmers" are opinionated young upstarts who get it into there head that the whole project needs a re-write when in fact their own programming abilities are not good enough to allow them to read other peoples code.
It is easy to see flaws in anyones code, I know where I work we sometimes joke about obvious inefficiencies etc in others peoples code, but a lot of the reasons you might think a code re-write is that code is generally written in a manner that isn't optimal. Yeah youll find stuff that is obviously poor but if it works then it isnt all that bad.
If you are a junior programmer on a project then you need to justify your place in the team. And if the first thing you do is go flaming other peoples code then you are not marking yourself as a team player. If possible talk to the guy who first wrote the code, maybe he had a different perspective on the code. Something that is complex might seem terrible to you. Try looking at a Boyer-Moore implementation someday and you wont be the first to thing that 2 nested for loops is much better, the fact is that what looks worse is actually better in that situation and the same in many.
Re:When it is time... (Score:2)
Worked with a really bright guy once, had an amusing story about his brother, another really bright guy, just a few circuits missing. Managed to convince management - through a very good looking cost benefit ananlysis study - that removing all documentation from the programs would save on storage and thus money. They bought it; he wrote the code that wiped out all comments; and they have had virtually zero new employees last a year before quitting in disgust (this for the last decade). The programs there are old, huge, and insanely modified/complex.
Yep, it's a gov't installation. Remember - most gov't employees are not paid for how well they do their job, they're paid for how many people they have (i.e. NEED!) working under them. This sorta qualifies as a negative incentive towards efficiency.
Re:Only with full user support (Score:2)
There is another reason - system performance. I'm not talking about stability - that should be part of any working program. I'm talking about things like actual run-time, memory usage, storage usage, and the like. I once worked at a company that had a very narrow batch window (8:00 p.m. California to 5:00 a.m. London), and was starting to get seriously concerned about programs which, due to the huge increases in number of records processed, was starting to exceed that window. Re-writes were ordered, and hours shaved off a number of the largest programs. Another I worked at had a program that built a DB (RedBrick) from nine different user-supplied files. All initially on tape. With 5 or six of them less than a thousand records (the rest were around the 100 million count). The original program could be using up to nine different tape drives, and ran for eight hours. Loading the 5 or so files into memory dropped run time to under an hour.
You _can_ generally sell management on a rewrite if you can cost justify it to them. Most places are scarce on personnel resources, which kinda conflicts, but telling the management that a week of your time will save x odd hundred thou a year tends to make them perk up and listen.
You should start from scratch... (Score:2)
What looks like bad code isn't always bad code (Score:2)
Take this medicine in very small doses (Score:2)
I spent a few years doing PL/I in a mainframe application that bills about $6,000,000,000/year, so please believe me when I say I know how you feel.
Long story short: been there, done that, should have bought the T-shirt ... and very nearly got fired. I learned the hard way that they don't pay you to make it look pretty or even to make it run more efficiently. THEY PAY YOU TO MAKE THE DATE.
Trust me: this is the kind of medicine that it's best to take in very small doses: I know how frustrating it can be to have to wade through a mess that someone else left behind, but try to stay detached and avoid the temptation to reinvent the wheel. As other have pointed out, you can improve small sections of the code, one release at a time, and you'll have 80 percent of the possible improvement done in 20 percent of the time a rewrite would take.
Here's some free advice: the next time the urge strikes, just take a walk around the floor, and remember that everyone else has felt the same way at one time or another. Then, get back to work and make your date, because that's what pros do, and it's all anyone will ever remember.
Then, go home on time, help your kids with their homework, and rest assured that you're doing a lot more to make the future better than you could ever do trying to re-re-rewrite some piece of spaghetti. As for the rest: that's what optimizing compilers are for, you know?
Bellhead
It worked for Linus (Score:2)
Linus himself stated it (Score:2)
There may never have been any minix code in any version of linux distributed as such, but by his own words, when Linus began writing Linux he started with the Minix codebase and worked by replacing it piece by piece. By the time he got around to showing it to other Minix users there was no Minix code left, but that's how he started. He actually used this as an example of how you can approach a seemingly huge project when there is something inadequate but similar to use as a starting platform.
Stop showing your ignorance.
Make a record... (Score:2)
Most importantly... it covers your ass when people ask later why you didn't fix the problem months ago.
Re:When it is time... (Score:2)
Never! (Score:2)
Go ahead and refactor; but for god's sake, don't dump & rewrite! Look at this [editthispage.com] for further explanation
Refactor. Use other Extreme Programming ideas (Score:3)
It's amazing how local transformations can introduce quality to systems with no conceptual integrity.
But to really be effective, you have to be aware of where you started and where you are headed. The big picture is a lot of little pictures, hopefully connected by a coherent architecture.
One of the best ways to move toward coherent architectures is to start thinking in terms of design by contract [eiffel.com]. Think of preconditions, postconditions, and invariants as part of the public interface, just like routines' signatures. If your language doesn't support them directly, you can probably get free 3rd party tools that will help. Even if you can't, put them in as comments. Program code is documentation of the write once, read many times, variety.
Learn and get comfortable with as many variants of as many patterns [hillside.net] as you can. When you use them, mention them in your comments.
This anonymous coward is Tom Morrisette
Stepwise refinement (Score:3)
Over the course of the next year or two as I maintained this utility I began by rewriting individual routines and then moved to modifying small sections of code in the end there were probably only a handful of the routines in the codebase that hadn't been completely rewritten.
Show management the code failing (Score:3)
If you cannot do that, then perhaps you are the one that needs the adjustment.
--
Re:You have three options: (Score:3)
Seriously, though - it's very hard to rewrite a program from the ground up; somehow, it's never as good. I'd clean up the code, but not rewrite.
This statement is wrong, just plain wrong. The axiom in computer science is "Don't fix bad code, rewrite it. Yes, I realize that's the CS take on things, and that it doesn't always jive with business needs. But I digress.
Second systems are typically easier to write because the hard part is already done: transforming the mental model of the system into real, working code. Plus, you get the benefit of lifting out portions of the old code that are worth saving. Also, second systems are easier to write because the entire idea has been realized via code once, and that means that you should see patterns and behavior that were not obvious when the initial coding began.
OTOH (because I like to argue, even with myself
The only constant is change.. (Score:3)
A lot of programmers start out re-writing with good intentions but usually wind up with something just as bad as what was originally there.
In order to avoid this problem, the programmer must analyze the code and have an intimate knowledge of what the code does and what it is SUPPOSED to do. After looking at these two things, the programmer must analyze what the impact is going to be for both fixing the code and replacing the code.
Once you have this information you can then assess the risks involved with both approaches. The programmer should also be aware of what the long term goals of development are. Its possible that you could be rewriting a section of code that will no longer be used in the next release. This would be a waste of your time as well as the companies.
One of the largest risks of rewriting a section of code is that you may change the intent of that code or introduce new side effects that you hadn't thought of. If the code has been around for a long time, chances are, that code for the most part works and should only be fixed. If the code has never worked, then it could be a good candidate for re-write.
When we fix bugs where I work, we are required to fill out a Design Change Request (DCR) form for anything that requires significant modification to the structure of the code. If it's a simple fix, then we fix it. The DCR is then submitted to a manager and they review it and approve it. In the DCR you state what problem you are trying to fix, what files it affects, what you plan to do, what side effects it might have, and any other alternative solutions that might be feasible as well.
What a DCR does for the programmer is it gets them to look at the problem in detail, and to understand what impact their solution may have. It also gets the programmer to look at alternative ways of solving the problem.
I can't count the number of times I've switched solutions because I thought of a better approach. Had I just started in and started fixing and or replacing code, I would have been wasting time and energy on something that might not have worked in the long run.
The last thing I want to say, is that Documentation and Planning for Change are critical to an application that is going to be easy to maintain down the road. Documentation (i.e. Good Comments) are not for you, but for that poor sap that follows you. Help them out!
When it is time... (Score:3)
Until then, they are correct in their judgement that it doesn't need rewriting. Elegance doesnt pay your salery.
Don't change it immediately... clean up in parts.. (Score:3)
In 5-7 years, the programming style you use will look ugly and primitive compared to whatever the latest buzzword in computer science is. If you rewrite whenever something "looks ugly" then you will be perpetually redesigning, and that isn't an efficient use of corporate resources.
Rewrites of small sections (Score:3)
Tonight I rewrote some code that traversed a tree depth-first in reverse. The old code wasn't working right, and it was too ugly to see by examination what was wrong. Rather than debug it, I wrote a class that represented position in the tree, with "next" and "prev" methods. The tree-traversal function then became much simpler.
Show them the bill (Score:3)
If they ask you for maintenance, give them the gory numbers for effort-required, especially after you spent some time understanding it (show them the bill for that time too). At some point they'll go, "You know, maybe it's time we cleaned that code up."
A friend of mine once rewrote the core of the application he was supporting by replacing bits and pieces here and there until he installed the new architecture from the inside-out. They do this with heritage buildings as well.
You have three options: (Score:3)
Two, add another layer of obfuscated code on top so you can get hired back as a consultant when the whole thing goes belly up and no-one can understand it.
Three, perform an 'accidental' disk wipe and force a code rewrite.
Seriously, though - it's very hard to rewrite a program from the ground up; somehow, it's never as good. I'd clean up the code, but not rewrite.
Refactor first (Score:3)
You might pay particular attention to his habit of writing unit tests before he does anything.
Re:When it is time... (Score:4)
One time at band camp. (Score:4)
1. Tell my manager about the nasty convoluted code.
2. Told her I could rewrite it, or try to hack what's there.
3. I Told here that in the long run it WOULD need to be rewritten, but then again it was a short term project, and I could probably make what was there work.
She decided that since we we're at feature complete, (about 2 months from shipping) It would be better for me to just hack at it. So I did, it worked out ok, I learned more about what the previous guy was doing and managed to work with it.
Sometimes there's code that no one likes, even the author. I think a lot of people have natural attraction toward rewriting something then learn and deal with what's out there. Personally I didn't care what my managers decision was, It wasn't my call, my job is to write the code for the tasks she gives me. Not to take on big projects that will risk the ship date.
-Jon
Streamripper [sourceforge.net]
Re:You have three options: (Score:4)
May I ask where you studied computer science? Rest assured, that is not what I've learnt. I didn't think computer science had an answer to this at all, and if it had, I would guess it would be something like "Interesting problem, let's conduct a study by doing both...".
Anyway, it is impossible to give advice about this in a general way, because each messy system is different, and messy is sort of hard to classify anyway! But I think the original posters advice was right on!
In a real-world project there are lots of code that doesn't seem to serve any purpose at all. But trust me, most of it does have some purpose, it might be a bugfix, some tweak requested by users, a kludge fixing some problems with third-party software, or even hardware, or just something really smart that you'd never understand anyway.
By reimplementing something from scratch, the new program has to work just as good as the old one (from a users standpoint). And first when your users complain that you find out that many of those small uninteresting tidpits needs to be in the new version as well. Which makes the new version almost as dull as the previous...
Only with full user support (Score:5)
don't do it. (Score:5)
If you are doing bugfixes, fix the bug.
If you are planning a rewrite, or find yourself rewriting major portions of the code, you are making a terrible mistake.
One, if you have to rewrite major portions, it was improperly designed in the beginning. This means a group of people, familiar with the task at hand, did not sit down and do a proper design. Note that you are not doing it either!
I don't know many times I've seen people start debugging and rewrite the code, and the next person sees it, says its all junk, and rewrites the code, ad infinitum.
Don't do it unless your serious, and if you are serious you need to be much more formal about it after the bug fixing is done.
BTW, has anyone else noticed what appears to be a pengiun sticker on the back of Johnny Depp's guitar in the "Chocolat" movie?
Rewrite versus Refactor (Score:5)
The best way to keep software clean over time is to redesign parts when you add functionality, otherwise things will only get worse over time. Tell management that it is called refactoring, if you think that helps. The wonderful fact about redesigning when needed is that you can do as little or as much of it as time and budget allows. And whenever you feel it is necessary.
The important thing is to stay in contact with management. Typically, you will be fixing a bug, or adding some functionality. In order to do that, you will often need to understand a component really good. At that point, you can tell the management that the chosen component, which you now understand perfectly, is a big mess. Tell them that since you have worked with it for over almost a month now, you know it's in and out's perfectly, and that you have some ideas for a better design. Tell them that you don't want to rewrite it all, because that would be to costly, but that there are some parts especially crufty, that it would be better to rewrite, and you would like to do just that. You can even tell them that it would probably be cheaper to to that then to just add your cruft to the other cruft, as it is probably true.
But don't lie to the management! Be honest, and tell them exactly why something needs a rewrite. There is very rarely a need for a total rewrite! Most code that exists are there for a reason! Sure, it can be cut&paste code (modularize), it can be deeply nested long blocks of hard-to-understand code (modularize), and it can be ugly non-portable code (clean up). All these are good reasons for a minor redesign.
But whatever you do, don't redo it all! Unless you wrote the program yourself, chances are that you will not do a better job than the previous guy! It might be hard to accept for someone who used to be a hot-shot programmer at the university, but there do exist other good programmers than yourself!
If management resists a total rewrite, then they are acting sensibly. While you don't understand all of the code, chances are that most of it is there for a reason. Even rewriting a single component from scratch is more than likely going over your head. The company is more interested in keeping software working than in having to do new regression tests three times over because your new module didn't work exactly up to specs.
And trust me, if you think coding is expensive, I tell you what, testing ill-designed components (which they have to be, given that you only redesigned a small part) is very expensive. And if the software is mission-critical (which it often is), or it is sold to actual customers, it will get even costier.
So, just grab a beer, start coding, and be thankful that you can at least make a small bit of the cruftiness disappear. If you want something else, you'd better wait untill you've proven yourself for the company, or start for yourself.