What is the Worst Tech Mistake You Ever Made?

CraigEagle asks: "Mistakes are made every day. The more technical knowledge you have the bigger your mistakes can be. What is the biggest technology related mistake you have ever made?"

"In the interest of full disclosure, this is mine:

I was working at a Fortune 50 bank as a consultant. I was due to go on vacation for a week and the company did not have webmail. I decided that I would try forwarding emails to my corporate account. (I know this was a bad idea, and probably against several corporate policies.) I set it up so that any email that came in would forward to my consulting company's account. My mistake was I also left Delivery Receipt on. This was not Microsoft, it was Lotus Notes. The system began forwarding the incoming mail to my account. But then it would get a Delivery Receipt, which in turn would be forwarded to my account, which would generate another delivery receipt, ad infinitum. When I got back from vacation they claimed I had brought down the email system for 4 hours. This incident caused the bank to stop allowing consultants to set up email rules. What's your story?"

One time...

I launched SkyNet, right before my daughter and future husband rushed in to warn me. Boy was my face red!

easy...

staking mine and my family's needs in a technical career!

Re:easy...

Anybody remember those commercials on TV: "There are 50 thousand ice machines in the US and somebody needs to fix them." And then they'd offer to send you to ice-machine-fixing school. There are others, fixing/ driving the "Big Rigs", learning how to be an X-Ray Technician in 6 months, etc. etc. etc.

Well recently I've seen at least one each for "IT Manager", "Computer Technician", and "Internet Technologist"... "learn in 6 months!"

They might as well be playing "Taps" [west-point.org] as far as I'm concerned. Well not quite but those ads are ALWAYS for commodity positions, "anyone can learn", etc etc.

And if there are any ice-machine repair-folks reading this, there's nothing wrong with that, it's just that most of us have spent our entire lives deeply involved with technology, and we are used to our compensation reflecting that. Those ads tell us that people think they can learn in 6 months, and schools are filling up with people doing just that.

No they're not going to directly take our jobs if we have 10 years of experience, but all those folks sure will lower the paycheck bar for the entire spectrum of IT workers. It's called Flooding the Market.

Add that to overseas outsourcing... it's so depressing.

I used to console myself with the adage "You get what you pay for.", but way too often the people doing the hiring don't understand what they need or what they're getting anyway, they don't understand the benefits of paying for something better, so they go cheaper, time and time again.

There are 50 million computers in the US and somebody needs to fix them.

[buries face in hands]

File errors

When I worked for a library I noticed a log of files with red ! as their icons. I determined that they must be erorrors or duplicates. So I removed them. Turns out that in windows 95 a red ! means that it is a critical system file.
And the library did not have the system source media anymore so we spend the next day looking for any machines with a similar version of the deleted file and moveing them back by hand.

DB + Audit Trails

Back in the mid 80s I was a jnr op on an old mainframe. Not much disk space so we used to save old audit trails to tape and remove them. Another pertinent fact is the DB starts UDX* and the audit trails start UDXA*

I wonder what might have happened if a certain jnr op had not being paying attention and thought he knew it all.

Yep, there goes the audit trails and the database :-( God knows why they kept me around.

Posting on Slashdot

"What is the biggest technology related mistake you have ever made?"

Statement by Slashdotters after the supoenas start rolling in: "Posting an admission of wrongdoing on a semi-anonymous public forum, whose owners will most likely cooperate with law enforcement when asked about an admission of wrong doing in a semi-anonymous public forum."

Logging on to /.

My biggest mistake was finding this website. I've wasted more time here that could have been spent doing my job and getting actual work done.

Failing to face reality...

...and force others to do the same thing.

Never, never, never, never commit to a schedule that is not realistic. If you know it isn't realistic before you get started, imagine what happens when you discover the unknown problems.

No matter how much that guy in marketing wants to meet his roadmap, he will not help you design, code, or test your product. If you are lucky, he will complete the requirements before you are supposed to ship the product.

A small one

After working on a bunch of NT servers all day I needed to make a quick change to our company's primary file server which ran Linux. After I logged out I hit control-alt-del and the machine started shutting down. About 30 seconds later the helpdesk switchboard lit up like a christmas tree. After working on the NT machines all day I didn't even think before I hit the keys. I guess I thought I was going to lock the screen or something. 15 minutes later I had the linux boxes configured to write to a log file when ctrl-alt-del was hit rather than rebooting. Lesson learned.

Thankfully, that's the worst I've done so far.

Damning evidence

Recently, I went to a work-from-home arrangement. Soon after I left, I got a call from several people at work asking me if I had emptied several SQL tables that were key to our QA environment. I swore up and down that I had done no such thing, and it was dropped.

The next day someone powered up the monitor to my old desktop (still at the office) and what did he see?

SQL Query Analyzer maximized with:

DELETE FROM IMPORTANT_TABLE

Query successful: 18,452 rows processed

(I still don't remember doing it.)

Re:Damning evidence

This wasn't my mistake, but I guy I work with left a where clause condition of an update SQL statement and released the work to production. In development he was working on a system with a reduced data volumes and it updated a few stray records and he didn't notice. In test, the test cases didn't cause the SQL to run. In production it wiped out about 20000 records and had to full scan a very large table to boot. We actually found it because of the performance problem first. It took us a few weeks to reconstruct all the erased data.

My biggest mistake was in my first programming job years ago. I intentionally wrote an infinite loop into a program that was running on a very powerful (for the time) reasearch unix box used at the Naval research lab where I had an internship. It was a sonar imaging optimization routine and I would let it run for short periods (10-30 seconds typically) and then CTRL-C it to force it to stop and inspect the log file to find the results. I was new to unix and so I would use "ps" as opposed to "ps -aux" to see what processes I had running. I had multiple sessions up and managed to leave one of my programs running, switched sessions, ran ps which showed no processes running and went to lunch. The sysadmin was also a meeting and then lunch. When I returned I had a bunch of nastygrams telling me to kill my job immediately, not to run processes that hog the CPU because other projects couldn't use the system and to get approval before running long running jobs because the CPU time was billed (this was around 1985). I actually sat down, ran ps again, saw no job, and wrote back saying I didn't know what they were talking about. The sysadmin (who had returned from lunch) came over to visit me and educated me on a whole bunch of things.

Re:Damning evidence

One of my former coworkers always typed "# rm" instead of "rm" at the beginning of a long delete command, and only when he was finished typing would he go back to the beginning of the line, delete the leading "#", and hit enter. Were he to accidentally hit enter before he was ready, then he did nothing more harmful than send a comment to his shell.

I've adapted that idea to a lot of other situations; my SQL queries always start out as "-- delete ..." until I'm sure about what I'm typing.

Re:Damning evidence

Similar to the if (0 == c) trick in C, I've been trying to train my fingers to type DELETE WHERE whenever I mean to type "DELETE". Then, I fill out the WHERE clause and only then go back to say what table to delete from.

This also gives you time to ponder the wisdom of first running a SELECT statement with the same WHERE clause and comtemplate whether you want to do this.

rm -rf *

...five minutes later after coming back from getting coffee: D'oh!

I actually did this once... while logged in as root... at the top level in /home... on a production server. Thank baphomet for nightly backups!

Hopefully none of my clients are reading this. :-)

Re:rm -rf *

Hopefully none of my clients are reading this. :-)

You're known as FunkyRat to them!? :)

One that I saw...

None that I've done come to mind - I tend to make lots of little stupid mistakes rather than occasional huge cock-ups. But I had a client that had a CIO who was actively hostile to the idea of any kind of computer security what-so-ever. Waste of time and money for a made up threat he said.

They were running 13 servers at remote locations (and I mean remote, as in out in the boonies 4 hours from nowhere on back roads) and these servers were unpatched, had out of date or innactive anti-virus and were connected to the net via a combination of satellite and dedicated (always on) dialup. Their communications were secured with nothing more than Windows 2000's built in VPN.

Needless to say, my audit report told them that they had big beefy powerful angels on their side since they hadn't yet had a noticable intrusion. (They had no way of detecting one, but at least the servers weren't hosting porn sites.) I warned them that a virus or worm would come along though and knock the whole thing out. The CIO scoffed at my report, called me an alarmist and said that my opinions were right up there with the Y2K doomsayers.

When Slammer hit, I had described the vulnerabilities and outcome so accurately that this guy actually accused me of writing it myself. Took the whole corporate network down and they couldn't bring it back up until their techs visited each site. It took two teams seven days to get to all the sites. The company lost 6 business days, three customers and a months worth of transaction records.

Needless to say the CIO was demoted (they didn't fire him, which I consider itself a major tech mistake) and had me re-issue my audit report which they then followed to the letter taking every precaution I suggested.

didnt really hurt anything, but ...

When I was first learning linux/unix I installed RH5.something on my computer (cyrix 6x86 133+ iirc), anyway I was having weird issues with several programs so I decided i needed a fresh start, those darn dot files must be currupted.
So I typed:
rm -rf .*
This disk started churning ...
about 30 seconds later
the disk is still churning ... damn I must have a LOT of those dot files.
about a minute later .... *DOH*
CTRL-C
Where did all the files go? DAMNIT! I recursively deleted .. ( I was running as root, It was my personal box what could be the harm)
I learned my lesson very well:
CREATE AND USE USER ACCOUNTS!! DONT RUN AS ROOT IF YOU CAN AVOID IT!

blowing up my computer

I was young (around 8 at the time, can't remember) and I was bored one afternoon. I started fiddling around with the back of the computer, the PSU, to be exact. The red button looked fun to play with.

It was on 220v. I turned the computer on. It worked. Then I tried putting it on 110v and turning it on. Nothing. Then I switched it back to 220v, turned it on, and switched it to 110v while it was on.

Boom.

Moral of the story is, trial and error isn't the best way to learn hardware, and don't throw water on the smoking PSU while it's still live.

more fire and smoke

Last year a friend gave me a pentium 200 mmx that he coundn't get working. Since my parents were in need of a firewall I figured I would drop a couple nics into this box and build one for them.

The first thing I did was plug in a keyboard, monitor, and turn the box's power on to see if it would reach the POST.

Smoke started coming from the box, and soon open flame. For a brief moment I just stood there looking at it thinking, "That's interesting. First time I've seen a computer catch fire." Then I pulled the plug from the wall and the flames soon stopped.

I looked into the case to see what went wrong. It seems that the power supply connector for a floppy drive is roughly the same size as a speaker connector on the sound card. My friend had plugged the power supply into the sound card which seems to have caused the fire when the power was turned on. I suppose I should have checked for something like this instead of just plugging in the machine.

Really? I believe the opposite is true

The more experience I and my coworkers have, the fewer mistakes we make, and they are less severe than before. This is because we learn from previous errors, and apply those lessons to entire classes of problems, not simply the error itself.

I think the articles implication of "the more we learn, the less we think" is wrong.

not so bad -- but still stupid

So this wasn't a production machine I screwed up or anything, but I'm still a moron.

I had a Linux workstation that was ultimately adopted by the development group I worked with in the late 90's. Anyway, for some reason I needed to make a boot disk from an image. For some other reason, while typing in my command line, I was thinking fd0 but managed to type hda. So my line was dd if=/wherever/whatever.img of=/dev/hda.

Anyway, before looking at what I had typed, I hit enter. About 2ms later, I glanced up at what was on the screen and exclaimed something along the lines of "holy fscking shit!" and simultaniously hit a ctrl+c. Interestingly enough, the drive still kind of worked. I tried copying the contents of the disk over to another device, but I found that with each command - nay, each disk access, the filesystem would disintegrate further. I was able to save /home -- but I otherwise had to reOS the system.

I guess I've done much more stupid things with production machines -- but these were better machines, with storage on a NetApp NAS, which all had snapshots, so recovery was nearly instantanous.

These are not things that I include on my resume. (So -- anyone want to hire a disaster waiting to happen?) ;)

Playing soundfile on remote UNIX box

Oh man. So I was a grad student, right? I was always trying to portray myself as a very serious, dedicated student to my thesis advisor. And he had the fastest computer in the department (a Sparc10!) and he gave me permission to use it for batch runs. So I pretty much kept one of my xterms as a remote terminal to his machine.

Anyhow, one day I found this funny .au (sound) file and wanted to play it for my office mates. So I did a 'cat naked.au > /dev/audio'. Nothing happened. So I turned up the volume and tried it again. Still nothing. Then I screached in horror! I was typing this command in on the xterm I use for my advisor's machine! Sure enough, two seconds later an email comes trickling in from my advisor stating 'Please note that you are logged into my machine so your sound file is coming through my speakers.'

So what was this sound file that I had inadvertently played for my advisor?

Butthead: "Whoa! Naked chicks!"
Beavis (excitedly): "Yeah! Naked chicks! Naked chicks!"

GMD

Re:Playing soundfile on remote UNIX box

Geez, kids these days.

sally.au

You old-schoolers know what I'm talkin' about.

Oh, come on, old schooler, get with the program. This is the web, for crying out loud! That should read as:

sally.au [ibiblio.org]

Much better.

Warning, not safe for work ;-)

(A class 2 sarcasm warning has been issued for this post.)

FoxPro-based MRP & Bad Networking

I have two, both at the same company. I was hired on as The Tech Guy(TM) at a local OEM Manufacturer. They were willing to take me on w/limited experience but my pay reflected it. My first job was to replace their existing DOS-Cobal Manufacturing Resource Planning (MRP) software before the Y2K problems hit. The original software company wasn't supporting the old version any longer and the Mfg was getting too large for the program. They looked at a number of demos that were slow and painful, but they decided to try a "favorable" application (See also: cheap).

My mistake was to give the techie "thumbs up" under pressure. I folded to the "We needed this yesterday" argument despite my misgivings about the software. I paid for that mistake for the next year in slavish tech support. We became the software company's test bed as we found bug after bug. The software "worked", but operator efficiency dropped, and uptime was sub-optimal. "Customization" caused problems, etc., etc.

The second mistake I made was to attempt to use VPN over Broadband with Citrix MetaFrame. Although MetaFrame was a pretty secure and slim protocol for remote desktops, the Internet provider on the remote site had horrible latency problems and was run by a group of amatures. I should have stuck with the original Sprint frame relay proposal.

Morals of the story: don't let PHB push you into a solution you don't trust, and when network reliability is important, pay for assured quality of frame relay.

My Commander told me to kill the network

I was a young pup in the Army, during a training exercise. My Commander told me to kill the network, to "simulate" it's loss. We were operating a frequency hopping radio network, which of course is based on time. As the master node, I controlled the time. I pumped my transmitter to full power, and slowly pulled the stations that could recieve my signal out of time. Lowered power, pulled a smaller number of stations even farther out of time. Wash, rinse, repeat.

Commander thought I was brilliant, and so did I. I had fractured our network into at least 10 different domains. No one could talk to anyone, effectively "simulating" an enemy jamming attempt. It would take hours to restore the network, with many mad commo guys having to drive about with Pluggers, early GPS devices, to restore each radio to propper time.

Then a tank flipped. Someone died. No one could call for help. I am so damn smart.

No moon black, At 2 in the morning, in an upside down tank, the gunner figured out how to put his radio in plain text to call for help. It took him almost half an hour.

Re:My Commander told me to kill the network

It sounds like you did exactly what you were supposed to do. Isn't part of the point of a training exercise that the danger is real and freak accidents like that can happen? Maybe the real mistake was not having an emergency backup communication system (and the same would apply for a real conflict situation).

Re:My Commander told me to kill the network

You know, while that had bad consequences, it also isn't something that I'd call your mistake. Your CO asked you to kill the network, you did so. The issue that came up wasn't a technical one specific to the issue, so you weren't in a position where I'd expect you to be able to advise your CO using specialty information. This wasn't something unethical, where you might have taken another course of action, or illegal, where you should have refused to follow an order. You did exactly what your orders stipulated that you do.

Frankly, from the post, I doubt that the person dying could have been saved if the radio had been active.

Furthermore, unless the Army has rules requiring that personnel on training exercises have a comm system always up, I'm not sure that even your CO made a poor decision. There is such a thing as tragedy -- where everyone really did do the right thing, and someone still gets hurt as a result.

It's kind of like deciding to drive down Main Street in a town instead of Lambert Street and hitting and killing a kid that ran out into the street. Yes, had you taken the other road, the kid would have been alive. However, you can't be expected to or be *able* to always make the decision that produces the best outcome, or you'd be the best gambler in the world. The only thing that you can do is what seems the most sensible thing given the information that you have at the time. You did that, and nobody could ask for more.

Re:My Commander told me to kill the network

This sucks, but you did what you were told. And it is not your fault the tank flipped. Better simulate a loss in comms and have your men learn that they can send plain text over their radio, than lose 10 men in wartime.

Still, I'd feel horrible. But I wouldn't hold you responsible.

I flooded the building.

Not about computers, but: While working for a Physics research lab, I made a laser water jacket without a bulge at the end of the inlet pipe. The water pressure rose at night, the tubing slipped off, and the 2nd floor and part of the first was flooded, including expensive test gear like oscilloscopes.

Lotus Notes... er..

Working as a consultant I turned up at a new customer (moderate sized pharmacy) to see what they needed. Walked in, all confident, the local tech guy met me, and I asked to look at their server room (I always liked seeing the hardware).

Anyway, as we are standing there, I think, well lets see how many users they have, so I ask if I could look at the Name & Address book. Opened up the people view, hit Control-A to see the count at the bottom of the screen of the number of records. Unfortunalty it was a very small compaq keyboard, hit delete as I turned to the local tech..

Oooops.

I was doing phone support for a national bank in Canada. One of the problems we routinely had was a connection would freeze-up on a teller's terminal in one of the 1000s of bank branchs across the country.. We'd have to go into a program running on our AS-400 and reset the connection. On the odd occasion it wasn't just one terminal but serveral at the branch. We'd have to get all the tellers to exit out of their terminals for a second, then, in the program, we'd esentially hit the 'back' button, be up one level so we saw all the connections by bank branch instead of by terminal, hit 'backspace' to send the command to reset the connection and then 'y - enter' to confirm.

I got one of these calls, and I went one level up the tree, got distracted by something, and without thinking hit up-backspace-y-enter, going up two levels in the tree instead of one. This reset all the connections for the whole network, to all the banks, all across Canada.

Every phone in the call center started ringing. Every LED that could flash red did so. Everyone in the call centre looked around frantically. I looked at my terminal and almost died on the spot.

Not only had I reset all the terminal connection, but trying to bring them back online flooded the network so as soon as they tried to come back up they all went offline again. It took several hours to get things stabalized and the banks could start serving customers again.

Fortunatly my boss was a decent guy. He saw it as an accident and something that no one should be able to accidently do. The command to reset the entire network was modified so you had to type in your password to confirm, instead of just 'y-enter'

Worst...Tech...Mistake...Ever

(comic book guy voice)

By far, my worst tech mistake was dropping out of college to take a full time job as an outsourced computer admin. Not having my degree has kept me from being competitive for better jobs with larger companies.

I love job now, but I don't have much room to grow, being as I'm the top IT guy in a 70-person company that's family owned (and I'm not in the family). I'm working on finishing my degree now so that when the time comes to move on, I'll be able to find jobs that have room for growth.

Ahh, stories.

My best singke mistake was after a long night of re-installing an updated version of solaris on a SparcServer 2, I needed to clear out the /tmp dir sor some stupid reason. So, I did the old: "mkdir newdir ; mv * newdir"

I wasn't in /tmp. I was in /.

My next command was 'ls'. It returned: unable to find /usr/lib/libc.so.0

AAAAARGH!

I now know how to solve that under solaris. Under /usr/sbin/static there are 5 statically compiled binaries: cp, ln, mv, rcp, and tar. /newdir/usr/sbin/static/mv /newdir/* / would have fixed it.

Ever since then, my prompt has had my current directory in it. That experience certainly made me more careful.

Better (or worse) was when a stupid service rep came in to replace a bad CPU on a sun e10000. The idiot shut down the sub-system, and powered off the board correctly. He then managed to pull out the wrong board, despite the blinken lights. Of course it was the peoplesoft domain. Running year end reporting.

AAAAARGH!

I ruined a phone book

I work for a telephone directory publisher. A few years back, we were pushing a deadline and the man was not happy with the completeness of zip (postal) code info in the book. I purchased a new zip coding utility, ran it against the listings, and told the production dept to proceed with pagination, thinking that the army of proofreaders we have would notice any errors introduced by the new software.

I mean, what, I'm supposed to proofread the entire phone book by myself?

Anyway, the software used some kind of crazy soundex routine to "fix" addresses that it wasn't able to resolve, and thousands of people ended up with completely incorrect address information. The book went to press, was distributed, and a day later the phones were ringing off the hook. We had to pick up the old books, fix the data, schedule more press time (no easy feat), re-print, and re-distribute.

Total cost to correct was around $1M, got my ass chewed royally, but managed to keep my job anyway.

Must be doing *something* right!

Trying out Kmail

Trying out Kmail was my biggest mistake, because it had a different interpretation of the file OUTBOX than did my previous mailer. My previous mailer stored every email (6 years worth) in OUTBOX. And kmail took OUTBOX to be the file where messages written offline were temporarily stored until next coming online. The first time I fired up Kmail, a indeterminate-time progress bar came up, and it kinda hung. I went to get a coke, giving it time to snap out of its funk. Unbeknownst to me, during that time it re-sent every email I'd ever sent. When I got back and checked my INBOX, I screeched in horror.

Funny thing is, people from my previous job were getting work related emails from me again, and they didn't seem to mind that (1) they were on outdated topics and (2) the company was defunct, they played right along and replied stuff like "yeah what ever happened to that issue?".

Click here!

Well, this is more of a graphical error than a text error, but it's still amusing. My company developed a technology where you can watch video from mulitple angles. (note: this is going back a few years.) So we were pumping out demos like mad. At one point, we got some stock footage of a horse show or something. It had a horse jumping over a fence, filmed from different angles. I had to insert the words "click here" at the bottom of the video because I was going to make that clickable. If you click there, then you get s'more info about our software.

Back then, we didn't do letterboxing like Media Player does. If the window you play the video in isn't the same as the aspect ratio of the video, then cropping occurs. I did not consider this little fact about our player, rather I got it up on the site as fast as I possibly could. Then, I went to lunch.

When I got back from lunch, I noticed the CEO was looking at the demo. So I poked my head in to say hi. He says "Why is this video telling me to lick it?" Wha? I go up to the screen, look at what he's watching, and... eep. The c in click here was perfectly cropped out of the shot. I mean perfectly. I mean you didn't know it was missing. So here's a horse, reared up on its hind legs, with the words "LICK HERE" just below its.. uh.. tail.

I am so glad that we had the one CEO in our industry that understood what took place.

Very Enterprising

As a joke, I once set the transporter to low resolution. The Captain was not amused.