March 2015 - Outages - Outages Lists

Level3 not reaching AWS / Netflix in Southwest with IPv6
by Ca By 27 Apr '15

27 Apr '15

Folks, I posted a ticket with Level3 on this issue, wondering if others are seeing the same thing Using the Level3 looking glass in Chicago i can ping the www.netflix.com at 2620:108:700f::36f5:7e0f -- which is hosted on AWS See image https://imgur.com/cROsraJ But i cannot ping the same IPv6 address from the Level3 Phoenix looking glass site, which lines up with the issues i see in the southwest on Level3 IPv6 https://i.imgur.com/0AhUqnO.png Hopefully someone at Level3 can resolve this issue soon.

3 4

Re: [outages] Outages Digest, Vol 82, Issue 14
by Jim Witherell 25 Mar '15

25 Mar '15

>From Level 3 awhile ago... *** CASCADED EXTERNAL NOTES 25-Mar-2015 23:15:29 GMT From CASE: 9045140 - Event The IP NOC has reported that traffic transiting high speed links on a device in Cleveland, OH made an intermediate-hop routing change to use a low speed link instead, which created congestion and caused customers to become impacted. The low speed routing link was removed to correct the issue, and service restoral has been verified with affected customers. No further impact is anticipated, and the IP NOC is currently auditing the network to ensure that this condition does not exist elsewhere." Sent from Yahoo Mail on Android From:"Mike Bartz via Outages" <outages(a)outages.org> Date:Wed, Mar 25, 2015 at 3:28 PM Subject:Re: [outages] Outages Digest, Vol 82, Issue 14 Seeing traffic loss with Level 3 from one location in NYC. I'm on hold trying to get through to their support to see what's up. Unusually long hold time. On Wed, Mar 25, 2015 at 3:24 PM, Starr, Steve via Outages <outages(a)outages.org> wrote: We are seeing traffic loss with Level3 through Detroit. To get around it we had to drop our peer who is directly connected to Level3. :: Steve Starr :: Daystarr Communications :: steve.starr(a)corp.daystarr.net :: v - 989.720.6000 :: f - 989.720.6060 On Wed, Mar 25, 2015 at 3:17 PM, Rob Castillo via Outages <outages(a)outages.org> wrote: Is anyone experiencing a service outage in Atlanta today via Level 3? On Tue, Mar 24, 2015 at 10:19 AM, <outages-request(a)outages.org> wrote: Send Outages mailing list submissions to outages(a)outages.org To subscribe or unsubscribe via the World Wide Web, visit https://puck.nether.net/mailman/listinfo/outages or, via email, send a message with subject or body 'help' to outages-request(a)outages.org You can reach the person managing the list at outages-owner(a)outages.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Outages digest..." Today's Topics: 1. Problem between Level3 (TW Telecom) and ATT near Los Angeles? (ML) 2. Comcast Outage in Freedom East Region? (chris) 3. Re: Comcast Outage in Freedom East Region? (Ryan Abbott) 4. Re: Comcast Outage in Freedom East Region? (chris) 5. Re: Comcast Outage in Freedom East Region? (Tony Patti) ---------------------------------------------------------------------- Message: 1 Date: Mon, 23 Mar 2015 17:51:47 -0400 From: ML <ml(a)kenweb.org> To: "outages(a)outages.org" <outages(a)outages.org> Subject: [outages] Problem between Level3 (TW Telecom) and ATT near Los Angeles? Message-ID: <55108AF3.4020003(a)kenweb.org> Content-Type: text/plain; charset=utf-8; format=flowed I've noticed an increase in latency and packet loss between two locations near the Los Angeles area. One has a TW Telecom fiber based connection and the other an ATT DSL line. The latency between two location used to be ~30ms and now is about ~60ms. Based on mtrs (See below) I'd say the ATT DSL line is the culprit for the packet loss issue. If the traceroutes rDNS is accurate it seems TW Telecom use Level3 to reach ATT (as expected). However it seems that Level3 goes to Dallas to reach ATT. That seems strange. I would assume Level3 and ATT would peered in Los Angeles. Level3 looking glass confirms that 107.128.0.0/12 is learned in Dallas from ATT from the point of view of Level3's LA router. MTRs: http://pastebin.com/eg3AUhAn ------------------------------ Message: 2 Date: Tue, 24 Mar 2015 08:54:46 -0400 From: chris <tknchris(a)gmail.com> To: "outages(a)outages.org" <outages(a)outages.org> Subject: [outages] Comcast Outage in Freedom East Region? Message-ID: <CAKnNFz_ySd6fn1Zke59x42X5gY2AZR-m0qeuX2rzKo9L2sve0w(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Anyone know of an outage? I have 20+ customers with business cable within somerset/mercer county (central NJ) down with no internet and traces to their IP ranges from outside appear to die early in comcast's core in NYC of PHILA. From one comcast circuit thats up in PA we cant even reach any of these nj customers statics even within internal comcast <-> comcast routes. chris

1 0

Re: [outages] Outages Digest, Vol 82, Issue 14
by Rob Castillo 25 Mar '15

25 Mar '15

Is anyone experiencing a service outage in Atlanta today via Level 3? On Tue, Mar 24, 2015 at 10:19 AM, <outages-request(a)outages.org> wrote: > Send Outages mailing list submissions to > outages(a)outages.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://puck.nether.net/mailman/listinfo/outages > or, via email, send a message with subject or body 'help' to > outages-request(a)outages.org > > You can reach the person managing the list at > outages-owner(a)outages.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Outages digest..." > > > Today's Topics: > > 1. Problem between Level3 (TW Telecom) and ATT near Los Angeles? > (ML) > 2. Comcast Outage in Freedom East Region? (chris) > 3. Re: Comcast Outage in Freedom East Region? (Ryan Abbott) > 4. Re: Comcast Outage in Freedom East Region? (chris) > 5. Re: Comcast Outage in Freedom East Region? (Tony Patti) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 23 Mar 2015 17:51:47 -0400 > From: ML <ml(a)kenweb.org> > To: "outages(a)outages.org" <outages(a)outages.org> > Subject: [outages] Problem between Level3 (TW Telecom) and ATT near > Los Angeles? > Message-ID: <55108AF3.4020003(a)kenweb.org> > Content-Type: text/plain; charset=utf-8; format=flowed > > I've noticed an increase in latency and packet loss between two > locations near the Los Angeles area. One has a TW Telecom fiber based > connection and the other an ATT DSL line. > The latency between two location used to be ~30ms and now is about > ~60ms. Based on mtrs (See below) I'd say the ATT DSL line is the > culprit for the packet loss issue. If the traceroutes rDNS is accurate > it seems TW Telecom use Level3 to reach ATT (as expected). However it > seems that Level3 goes to Dallas to reach ATT. That seems strange. I > would assume Level3 and ATT would peered in Los Angeles. > > Level3 looking glass confirms that 107.128.0.0/12 is learned in Dallas > from ATT from the point of view of Level3's LA router. > > MTRs: > > http://pastebin.com/eg3AUhAn > > > > > ------------------------------ > > Message: 2 > Date: Tue, 24 Mar 2015 08:54:46 -0400 > From: chris <tknchris(a)gmail.com> > To: "outages(a)outages.org" <outages(a)outages.org> > Subject: [outages] Comcast Outage in Freedom East Region? > Message-ID: > < > CAKnNFz_ySd6fn1Zke59x42X5gY2AZR-m0qeuX2rzKo9L2sve0w(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Anyone know of an outage? I have 20+ customers with business cable within > somerset/mercer county (central NJ) down with no internet and traces to > their IP ranges from outside appear to die early in comcast's core in NYC > of PHILA. From one comcast circuit thats up in PA we cant even reach any of > these nj customers statics even within internal comcast <-> comcast routes. > > chris >

5 4

Comcast Outage in Freedom East Region?
by chris 24 Mar '15

24 Mar '15

Anyone know of an outage? I have 20+ customers with business cable within somerset/mercer county (central NJ) down with no internet and traces to their IP ranges from outside appear to die early in comcast's core in NYC of PHILA. From one comcast circuit thats up in PA we cant even reach any of these nj customers statics even within internal comcast <-> comcast routes. chris

3 3

Problem between Level3 (TW Telecom) and ATT near Los Angeles?
by ML 23 Mar '15

23 Mar '15

I've noticed an increase in latency and packet loss between two locations near the Los Angeles area. One has a TW Telecom fiber based connection and the other an ATT DSL line. The latency between two location used to be ~30ms and now is about ~60ms. Based on mtrs (See below) I'd say the ATT DSL line is the culprit for the packet loss issue. If the traceroutes rDNS is accurate it seems TW Telecom use Level3 to reach ATT (as expected). However it seems that Level3 goes to Dallas to reach ATT. That seems strange. I would assume Level3 and ATT would peered in Los Angeles. Level3 looking glass confirms that 107.128.0.0/12 is learned in Dallas from ATT from the point of view of Level3's LA router. MTRs: http://pastebin.com/eg3AUhAn

1 0

Re: [outages] Outages Digest, Vol 82, Issue 12
by Samer Ziadeh 21 Mar '15

21 Mar '15

It is, I've been getting same results for the past couple of weeks. // Samer > On Mar 21, 2015, at 12:00, outages-request(a)outages.org wrote: > > Send Outages mailing list submissions to > outages(a)outages.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://puck.nether.net/mailman/listinfo/outages > or, via email, send a message with subject or body 'help' to > outages-request(a)outages.org > > You can reach the person managing the list at > outages-owner(a)outages.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Outages digest..." > > > Today's Topics: > > 1. Cogent issue in Southern California? (Outages) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 20 Mar 2015 13:41:55 -0700 > From: Outages <virendra.rode(a)outages.org> > To: "outages(a)outages.org" <outages(a)outages.org> > Subject: [outages] Cogent issue in Southern California? > Message-ID: <CE5B6C4B-23A1-464A-AD90-BA2F289C36A7(a)outages.org> > Content-Type: text/plain; charset=us-ascii > > Seeing high latency (50+ ms) via cogent path? This has been intermittent, sorry I don't have mtr data to report but wanted to reach out to see if anyone else was experiencing the same? > > /virendra > > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Outages mailing list > Outages(a)outages.org > https://puck.nether.net/mailman/listinfo/outages > > > ------------------------------ > > End of Outages Digest, Vol 82, Issue 12 > ***************************************

1 0

Cogent issue in Southern California?
by Outages 20 Mar '15

20 Mar '15

Seeing high latency (50+ ms) via cogent path? This has been intermittent, sorry I don't have mtr data to report but wanted to reach out to see if anyone else was experiencing the same? /virendra

1 0

Re: [outages] [Outages-discussion] dns cache flush request
by Jay Ashworth 16 Mar '15

16 Mar '15

----- Original Message ----- > From: "Damian Menscher" <damian(a)google.com> > That one was a legitimate request, not a scam. (I have off-list > knowledge.) Which leads to an interesting question (that I wouldn't want to thread on too long, but I can't set a Reply-To Discussions :-) -- How many eyeball networks are actually set up to execute that request without requiring manual Tier 3 intervention? Any? Cheers, -- jra -- Jay R. Ashworth Baylink jra(a)baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274

4 3

Windstream Total Outage - Nashville
by John Pinkerton 16 Mar '15

16 Mar '15

Windstream circuit dropped an hour ago. Alarm LED showing on the wall panel. It was an old Nevox line back in the day. Current hold times for support are 30min. Request for call forwarding to working number still not implemented 20 minutes later. -- John

2 1

Little Rock to Memphis - Possible Fiber Cut
by Erik Sundberg 15 Mar '15

15 Mar '15

We are seeing a wave down between 1950 Stemsons FWY, Dallas to 56 Marietta in Atlanta, GA. The waves goes thru Nashville. XO is saying span failure between Little Rock, AR and Memphis, TN. Assuming it's a fiber cut. ________________________________ CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You must destroy the original transmission and its attachments without reading or saving in any manner. Thank you.

1 0