What? Where?
On Sunday December 29th, Level3 Communications, one of the largest IP transit networks in North America and Europe, suffered an outage affecting sites such as ESPN, Amazon, and CNN. Noticeable missing from Level3? Any official acknowledgment or discussion of the incident; and an outage that disrupts ESPN on a Sunday will definitely be noticed.
When one of the largest internet backbones shows trouble, there are two ways to handle the situation (after fixing the issue of course). The first would be what we here call the “Duck and Cover” method (sometimes referred to as the Vinnie Barbarino approach). In this method, the company either makes no mention of the incident or does a classic Vinnie impersonation… such as:
Concerned User: I think there’s an outage.
Level3: What? Where?
Concerned User: I think there’s a problem with the network.
Level3: I’m So Confused.
(mugs to the camera and audience goes crazy)
The other approach would be to acknowledge the incident, explain what is known, and explain what will be done in the future to prevent similar issues. This approach is, to paraphrase Billy Joel, is hardly ever heard but mostly what we need. We’re not sure why so few companies use the “acknowledge” approach — maybe it’s an old school, big business ideal that’s ingrained into company culture, but who knows? Open source companies, such as Digium, tend to disclose issues and discuss them, well, openly.
Open discussion tends not to turn folks away. Problems happen. How companies handle issues can either increase customer loyalty or send customers searching for alternatives. In our tech circles, most people seem to embrace the companies they can trust — and those companies tend to avoid the duck and cover approach.
So, where does that leave us with Level3?
Well, sadly, Level3 reminds us that your VoIP powered telephone system depends on Internet. Choosing several providers can certainly increase uptime, and choosing a company you can rely on will certainly help you sleep better at night. As for the 12/29 incident, we here (much like others) can only speculate. Current theories suggest a weather storm in Michigan may have been ground zero.
Other readings:
- Level3 Press Releases (notice no mention)
- TechCrunch reports “The Tubes are Clogged”
- Broadband Reports “Outages Plague AT&T and Level3”
- SANS “Level3 Outage Confirmed”
- Slashdot “Storm Causes AT&T Outage Across Midwest”
About Level3:
Level 3 Communications (Nasdaq: LVLT) is an international communications company headquartered in Broomfield, Colorado. The company operates one of the largest communications and Internet backbones in the world.
[...] VoIP Tech Chat posted an opinion piece regarding Level 3’s handling of the outage. And, as with most opinion pieces from VoIP Tech Chat, we like to pass it on. [...]
VoIP Tech Chat discusses Level 3 Outage - Team Forrest
31 Dec 08 at 2:22 pm
It’s an imperfect world. It’s full of imperfect companies offering imperfect services, based on imperfect technologies, staffed and managed by imperfect people. Plan accordingly.
It’s during such outages that I treasure my SIP-to-cellular gateway device. Just like the warm feeling I get when the power goes out and the UPS kicks in.
The old boy scout motto returns. Be prepared!
Michael Graves
31 Dec 08 at 2:36 pm
This made me think of the massive power outage of Aught-3. Everyone was to blame, so no one was to blame. At the end of the day, the power grid is “public”, the internet is “public” and no one person is likely to ever be held accountable when there is a catastrophic outage.
Sadly it looks like some of the doomsday prophesies of the last few years may be on the horizon, with Level3′s stock at an all time low and languishing below the $1 mark, who takes over the core of the internet in the fire sale?
Patrick
31 Dec 08 at 3:24 pm
I couldn’t agree more. Companies that develop persistent and transparent communications between customers, management and employees are much more likely to operate better and to have more loyal customers.
Thanks for sharing your insights!
Kevin
Kevin Stirtz
3 Jan 09 at 2:20 pm