May 2011 - Outages - Outages Lists

Level 3 core router in Chicago down?
by Brooks Bridges 31 May '11

31 May '11

We just lost a bunch of routes out of Chicago over Level 3. Seems to be dying at edge routers right before their core. 1. XXXXXXXXXXXXXXX 0.0% 10 0.5 0.4 0.3 0.6 0.1 2. core1.te2-2-bbnet2.chg.pnap. 0.0% 10 1.0 3.5 1.0 26.0 7.9 3. TenGigE0-3-0-0.GW2.CHI13.ALT 0.0% 10 1.6 1.4 1.2 1.6 0.2 4. 0.ge-1-0-0.XL4.CHI13.ALTER.N 0.0% 10 1.1 1.1 1.1 1.1 0.0 5. 0.ae4.BR3.CHI13.ALTER.NET 0.0% 10 1.1 4.6 1.0 32.8 10.0 6. ae6.edge4.chicago3.level3.ne 0.0% 10 1.1 3.0 1.1 19.9 5.9 7. vlan51.ebr1.Chicago2.Level3. 0.0% 10 1.3 1.3 1.2 1.3 0.0 8. ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 1. XXXXXXXXXXXXXXX 0.0% 10 1.3 1.4 1.2 2.5 0.4 2. Te6-1.ar5.CHI2.gblx.net 0.0% 10 68.8 7.9 1.1 68.8 21.4 3. 4.68.62.25 0.0% 10 4.2 1.4 1.0 4.2 1.0 4. ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 -- Brooks R. Bridges Telecommunications Manager Ifbyphone, Inc. Phone: (847) 983-3000 Fax: (847) 676-6553 bbridges(a)ifbyphone.com http://www.ifbyphone.com

5 5

Verizon fiber-cuts NY & PA (was Re: Further Verizon outages in northeast?)
by Christopher Pilkington 28 May '11

28 May '11

Verizon is experiencing two fiber cuts, one in a manhole/fault fire in Manhattan (previously reported as Mahopac), NY and a second cut on their leg between Binghamton, NY and Olyphant, PA, just north of Scranton, which was weather related. The latter appaears to be a Cavalier Tel fiber. These failures have isolated multiple OC-192 between upstate NY and New York City. They still do not have access to repair the Manhattan vault, and a crew is working the PA cut. On Fri, May 27, 2011 at 10:48 AM, Christopher Pilkington <cjp(a)0x1.net> wrote: > Anyone heard of further Verizon outages around 2250 EDT / 0250 UTC? > Our service that was taking the long way around their ring went > completely down. > > Thanks, > -cjp >

3 2

Outages in VA (Fiber Cut)
by Mauricio Lizano 27 May '11

27 May '11

All, The Virginia Department of Transportation continues to clear a .5 mile area of trees and debris before the power company can access the area to begin pole replacements. It appears that 4-8 poles are damaged, broken in half, or missing. Once the power company has completed pole replacement, the fiber vendor crews can assess damage to fiber and begin restoration efforts. There is no estimated time of restoration for the fiber cut. I have both Qwest and Level 3 circuits affected in the area. -Mauricio Lizano

3 3

Further Verizon outages in northeast?
by Christopher Pilkington 27 May '11

27 May '11

Anyone heard of further Verizon outages around 2250 EDT / 0250 UTC? Our service that was taking the long way around their ring went completely down. Thanks, -cjp

1 0

Manhole fire in Mahopac, NY
by Christopher Pilkington 26 May '11

26 May '11

There was a vault fire in Mahopac, NY affecting Verizon, possibly others', fiber. ConEd has priority to the manhole, not letting Verizon in yet. Our services are protected, we're going the long way around the ring, but thought others may want to know.

1 0

TWTC outage in WDC/Falls Church
by Ryan Rawdon 26 May '11

26 May '11

TWTC has confirmed an outage in the Washington, DC area. We have two T1s in Falls Church, VA which are down. We saw the first sign of trouble around 6:25AM Eastern and have seen a few flaps but been largely down since then. TWTC confirmed that they are actively working on it with a master ticket open and that it looks like a T3 has been impacted. no ETR yet as of 20 minutes ago

1 1

Android Market/Sprint problems?
by Jay Ashworth 26 May '11

26 May '11

Starting last night some time, my Android market app thought that I needed to update *every single app on my Evo* (some 90 of them). But, of course, it couldn't actually successfully download *anything*. Until about 11am, my Google sync stuff also was broken, but the Sprint Guy "fixed something" ("the trouble's leaving here *fine*!" :-), and now sync and email pickup seems to work ok... but the Market still doesn't like me. I've tried Clearing Cache and Data on the downloader and market apps, and apparently I can *push* a new app to the phone from the market, but I can't pull a new one, nor update anything. I have a suspicion this is actually my handset, not their service, but I figured I'd prospect for others having trouble first. Anyone? Anyone?? Bueller? Cheers, -- jra -- Jay R. Ashworth Baylink jra(a)baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274

1 1

FW: AT&T Opti-man Dallas down
by Frank Bulk 25 May '11

25 May '11

Forwarded from NANOG. -----Original Message----- From: Nikos Mouat [mailto:nikm@cyberflunk.com] Sent: Wednesday, May 25, 2011 12:15 PM To: Tim Connolly Cc: Nanog Subject: Re: AT&T Opti-man Dallas down Yes, we lost all our Opteman circuits in Dallas from roughly 8:30AM to 10:30AM Central. The cause communicated to us from AT&T was: "the router took a hit and the SUP Card failed which resulted in our outage" our Opteman circuits in Houston were not impacted. Nikos On Wed, 25 May 2011, Tim Connolly wrote: > The Dallas area had a round of storms go through last night. Looks > like this morning's flaming bag on the front porch was AT&T's "OptiMan" > metro-E service was down. Can anyone confirm that this was a wide outage > and not just a single customer? Any details released? > >

1 0

Sprint issue in the midwest
by Frank Bulk 23 May '11

23 May '11

A broad range of customers are having problems with calling certain long-distance numbers. First report was regarding toll-free at 10:39 Central. In our first call our tandem wasn't aware of the issue, but they have now told us that Sprint's having a major issue. Frank

2 3

Any outages on Cogent?
by Dale Amon 22 May '11

22 May '11

Is anyone aware of any network outages this afternoon between NYC and Boston? We had about a 15 minute drop out that took out 8 channels of streaming video for a major financial conference and need to find out what happened.

2 2