Anyone who works at an IT service desk long enough collects their share of horror stories. Ridiculous support requests. Nightmare viruses. Or the days where everything goes wrong, and you’re the only one on call. We’ve put together some of the most memorable horror stories we’ve heard from our colleagues. You’ve probably suffered through IT trials of your own, especially if your company doesn’t have an IT Service Management solution in place. So if you think you can outdo these stories, share your own, at the bottom of this article!

Well, That’s One Way to Retrieve a Forgotten Password

Back in the 90′s, one tech we know finished up a marathon weekend rollout, doing final testing and user training on Monday. The company had been informed that all employees would need to choose new passwords. A receptionist informed the staff.

As the install team was leaving, the tech happened by a company billboard and saw, out of the corner of his eye, a neatly printed list of every user password, posted for all to see. The same receptionist had requested each employee email her their passwords, and she had compiled a list so she could handle the inevitable forgotten passwords over the next few days. “I thought it would be easier to post them someplace,” she explained, “than to deal with all those password requests.”

 

Make your password hard to guess … unless someone pastes it in the breakroom.

 

A Dish Best Served Cold

This one came to us from a medical device company in Massachusetts, where a support tech was fired for an issue that ended up being the fault of a sales rep. It was not an amicable parting. The CEO yelled at the tech in a staff meeting, insulted him, and fired him on the spot, informing him in front of everyone that he would receive no severance. He was sent out of the room to pack his desk, unaccompanied. He did just that, and left with only his personal belongings.

But what he left behind was the real problem.

At 5:01 p.m., the storm broke loose. Every server the company owned began to rebuild itself, as did the desktop computers of the CEO, CFO and office manager. It took almost three days for the system to be restored, from backup tapes that were a week old. Emails and documents, even code samples in development were gone. And the scripts that ran to wreak all this havoc were gone, too, erasing the evidence.

No ID, No Credentials, No Problem

A repair tech walked into an office building in California just after 5:30 on a Friday. Everyone had gone home but the office manager. He wore a button-down shirt with the logo of the contract support company. He showed a “work order” to do some repair on the servers and informed the office manager that there might be some brief interruptions while he worked. The office manager glanced at the work order, hurriedly let the tech into the server room, and ran off to a long conference call.

 

“He has on a company shirt. He must be official.”

Some time later, the office manager tried to send an email, and found she could not. She picked up the phone and called the support company, demanding they send their tech back out. The support company said they never sent a tech, and wouldn’t be able to send anyone for several hours. The company was dead in the water.

When the real support guys finally arrived, they found that hard drives, CPUs and system RAM had been removed from every server. Security cameras later showed the “tech” walked out the door, carrying them in his tool bag. They couldn’t even begin repairs until replacement hardware could be ordered.

The corporation blamed the support group; the support group blamed the corporation. It took two days (and the threat of a lawsuit) to get the company fully back up and running. On the bright side, security got a lot tighter after that.

Not Exactly a Bundle of Joy

For years, a large, well-known telecommunications company had bundled DSL service with landline phone service — you couldn’t get one without the other. At some point, this policy was changed — but in telco style, this was never clearly explained to customers, who kept paying for phone service that most of them never used.

One day, a large tech blog wrote an angry post on this policy, calling it “DSLGate.” They posted a phone number for customers to get their phone service unbundled from their DSL. The number they published, however, sent furious customers to the company’s internal support desk.

For eight horrific hours, the two dozen brave men and women in that support desk were flooded with calls from customers, demanding phone service to be cancelled and refunds to be issued. At first, the team tried to route all the calls back to customer support, but the phone systems had been “upgraded” over the weekend, and they could not forward calls. Literally, the “Forward” button did nothing.

It was a perfect storm, because the IT management team was out at a corporate event, trying to “manage” by sending unhelpful emails via Blackberry. “Write down the name and number of everyone who calls in,” the VP suggested, and that’s what they did for the next four hours. $200 million worth of technology, and they were writing down customer names in Google Docs.

But things got worse. Later in the day, there was a fire in the breakroom, and everyone had to be evacuated to the parking lot. None of them were sad to abandon their posts, and about half the department left early for drinks. Or maybe to shiver uncontrollably.

Why Your “Secret Question” Is Useless

Hackers broke into the website of a large electronics retailer, publishing the passwords of about 10,000 users, including company employees. The story made the homepage of CNN, setting off a media firestorm.

“The company was totally unprepared,” said Tom, the support desk manager on call that day. “They sent out an internal email that they were launching a ‘thorough investigation,’ but in the meantime, we were getting slammed with calls from employees who wanted to know whether their passwords had been compromised. We had no information.”

Tom’s team told users how to change their passwords, but as the company furiously tried to patch security holes, they accidentally disabled the password reset function. That sparked more calls, escalating the confusion.

“They also accidentally wiped out the password tables, a backup server was corrupted, and the offsite backups were more than a week old. It was a comedy of errors, except it wasn’t a comedy, but a nightmare.”

“Oh yeah,” Tom added, “Did I mention this all happened on the Friday before Memorial Day?”