May 2012 - Outages - Outages Lists

Packet loss between L3 and GBLX
by Steve Dispensa 31 May '12

31 May '12

Anyone else noticing continuing packet loss between L3 and GBLX? This has been going on and off for about 18 hours (more?). |----------------------------------------------------------------------- -------------------| | WinMTR statistics | | Host - % | Sent | Recv | Best | Avrg | Wrst | Last | |------------------------------------------------|------|------|------|- -----|------|------| ... | ae-5-5.ebr2.dallas1.level3.net - 0 | 166 | 166 | 10 | 10 | 21 | 11 | | ae-72-72.csw2.dallas1.level3.net - 0 | 166 | 166 | 10 | 12 | 22 | 11 | | ae-2-70.edge4.dallas3.level3.net - 0 | 166 | 166 | 10 | 14 | 77 | 11 | | gblx-level3-40g.dallas3.level3.net - 0 | 166 | 166 | 10 | 17 | 108 | 36 | | te2-3-10g.ar1.snv2.gblx.net - 12 | 115 | 102 | 55 | 70 | 211 | 67 | | internap.te1-3.ar1.snv2.gblx.net - 19 | 95 | 77 | 56 | 71 | 256 | 81 | ...

4 4

Omnis.com registrar outage
by Graham Freeman 31 May '12

31 May '12

Registrar & DNS provider Omnis.com (and domains served by them) appear to be offline. I'm seeing sporadic connectivity to ns1.omnis.com and ns2.omnis.com from multiple points on the internet. Depending on when I try, my traceroutes fail within both of their upstream networks, Level3 and Cogent. -Graham

1 0

ATT fiber cut near Lake Charles, LA
by Ben Bartsch 25 May '12

25 May '12

we have confirmed there is an 'issue' in the area with ATT

1 0

Vzw voicemail from cell phone
by doug schmidt 24 May '12

24 May '12

Can not access vm, fast busy signal, Nj area.

1 0

Cogent <---> L3 latency
by Chris Stone 21 May '12

21 May '12

Had some customers not some slow web sites and looking appears to be the result of latency between Cogent and Level3. The http://www.internetpulse.net/ site is also reflecting high latency at the Cogent/Level3 connections. Seeing up and down from here in Denver: [root@sys~]# mtr --report --report-cycles=25 --report-wide 8-25-240-45.netecin.net HOST: sys.axint.net Loss% Snt Last Avg Best Wrst StDev 1. edge.axint.net 0.0% 25 0.6 0.7 0.3 3.1 0.6 2. fa0-2.na01.b009854-0.den01.atlas.cogentco.com 0.0% 25 0.9 1.1 0.7 3.1 0.5 3. vl3509.mag02.den01.atlas.cogentco.com 0.0% 25 1.6 5.6 0.7 58.4 15.8 4. te7-4.mpd02.den01.atlas.cogentco.com 0.0% 25 135.1 11.0 0.6 135.1 34.4 5. te0-4-0-2.mpd22.mci01.atlas.cogentco.com 0.0% 25 12.8 12.9 12.6 14.8 0.5 6. te0-3-0-6.ccr22.dfw01.atlas.cogentco.com 0.0% 25 23.5 23.6 23.2 26.1 0.6 7. te0-0-0-0.ccr21.dfw01.atlas.cogentco.com 0.0% 25 22.9 23.3 22.7 27.7 1.0 8. te7-3.mpd01.dfw03.atlas.cogentco.com 8.0% 25 193.8 31.6 23.2 193.8 35.5 9. te-3-2.car3.Dallas1.Level3.net 4.0% 25 144.8 65.8 46.6 228.5 44.5 10. vlan70.csw2.Dallas1.Level3.net 32.0% 25 147.5 151.4 147.5 159.6 4.4 11. ae-72-72.ebr2.Dallas1.Level3.net 24.0% 25 147.5 147.7 147.3 148.3 0.3 12. ae-2-2.ebr1.Denver1.Level3.net 36.0% 25 149.5 150.3 146.2 159.3 4.2 13. ae-12-51.car2.Denver1.Level3.net 0.0% 25 46.0 79.9 45.9 237.3 58.0 14. EASTERN-COL.car2.Denver1.Level3.net 0.0% 25 47.4 47.4 47.1 48.2 0.3 15. 8-25-240-45.netecin.net 32.0% 25 158.4 154.4 152.1 158.4 1.9 [root@sys~]# mtr --report --report-cycles=25 --report-wide 8-25-240-45.netecin.net HOST: sys.axint.net Loss% Snt Last Avg Best Wrst StDev 1. edge.axint.net 0.0% 25 0.6 0.6 0.4 1.5 0.2 2. fa0-2.na01.b009854-0.den01.atlas.cogentco.com 0.0% 25 0.9 1.0 0.7 1.4 0.2 3. vl3509.mag02.den01.atlas.cogentco.com 0.0% 25 67.0 6.0 0.6 67.0 17.6 4. te7-4.mpd02.den01.atlas.cogentco.com 0.0% 25 139.0 32.7 0.8 173.3 55.2 5. te0-4-0-2.mpd22.mci01.atlas.cogentco.com 0.0% 25 12.8 13.0 12.4 15.5 0.6 6. te0-3-0-6.ccr22.dfw01.atlas.cogentco.com 0.0% 25 23.4 23.8 23.2 25.1 0.5 7. te0-0-0-0.ccr21.dfw01.atlas.cogentco.com 0.0% 25 22.9 23.0 22.6 24.5 0.4 8. te7-3.mpd01.dfw03.atlas.cogentco.com 8.0% 25 28.7 84.7 23.2 368.1 94.9 9. te-3-2.car3.Dallas1.Level3.net 0.0% 25 23.4 28.3 23.2 116.9 19.1 10. vlan70.csw2.Dallas1.Level3.net 0.0% 25 29.8 26.7 23.2 34.9 4.3 11. ae-72-72.ebr2.Dallas1.Level3.net 0.0% 25 23.6 23.9 23.5 25.9 0.5 12. ae-2-2.ebr1.Denver1.Level3.net 0.0% 25 38.0 41.4 37.4 50.1 4.5 13. ae-12-51.car2.Denver1.Level3.net 0.0% 25 37.8 68.3 37.5 229.7 58.8 14. EASTERN-COL.car2.Denver1.Level3.net 0.0% 25 38.4 38.9 38.4 40.3 0.5 15. 8-25-240-45.netecin.net 0.0% 25 49.1 47.4 44.3 51.9 2.1 Anyone hear of anything? (waiting to hear from Cogent) Chris

2 1

TcpDump.org ? Cogent ? Something else ?
by Jason Hellenthal 20 May '12

20 May '12

Anyone else seeing this ? traceroute to bpf.tcpdump.org (132.213.238.6), 64 hops max, 40 byte packets [...] 3 [AS65534] 10.179.128.1 (10.179.128.1) 19.811 ms 17.924 ms 19.907 ms 4 [AS20115] dtr01hlldmi-gbe-1-14.hlld.mi.charter.com (96.34.33.162) 19.865 ms 17.277 ms 19.886 ms 5 [AS20115] crr02aldlmi-tge-0-2-0-2.aldl.mi.charter.com (96.34.32.76) 20.099 ms 17.531 ms 19.782 ms 6 [AS20115] bbr01aldlmi-tge-0-0-0-5.aldl.mi.charter.com (96.34.1.162) 19.832 ms 17.328 ms 19.993 ms 7 [AS20115] bbr01chcgil-tge-0-0-0-6.chcg.il.charter.com (96.34.0.57) 29.771 ms 27.124 ms 29.923 ms 8 [AS20115] bbr02chcgil-tge-0-3-0-1.chcg.il.charter.com (96.34.1.185) 19.973 ms 27.418 ms 29.916 ms 9 [AS174] te0-4-0-4.ccr21.ord03.atlas.cogentco.com (38.104.102.233) 19.841 ms 17.317 ms 19.748 ms 10 [AS174] te0-2-0-1.mpd21.ord01.atlas.cogentco.com (154.54.2.93) 22.040 ms [AS174] te0-5-0-7.mpd22.ord01.atlas.cogentco.com (154.54.44.181) 27.478 ms [AS174] te0-5-0-7.mpd21.ord01.atlas.cogentco.com (154.54.44.173) 16.951 ms 11 [AS174] te0-1-0-3.mpd22.yyz02.atlas.cogentco.com (154.54.6.181) 51.184 ms [AS174] te0-1-0-3.mpd21.yyz02.atlas.cogentco.com (154.54.2.162) 47.444 ms [AS174] te0-3-0-3.mpd22.yyz02.atlas.cogentco.com (154.54.42.14) 47.155 ms 12 [AS174] te0-3-0-5.ccr21.ymq02.atlas.cogentco.com (154.54.42.226) 47.212 ms [AS174] te0-3-0-3.ccr21.ymq02.atlas.cogentco.com (154.54.7.214) 47.067 ms [AS174] te0-5-0-2.ccr21.ymq02.atlas.cogentco.com (154.54.44.117) 46.930 ms 13 [AS174] 38.104.154.62 (38.104.154.62) 57.172 ms 47.932 ms 39.896 ms 14 [AS852] dmtrl-rq.risq.net (192.77.63.50) 39.825 ms 47.695 ms 49.698 ms 15 [AS852] uq-gw.risq.net (132.202.61.142) 50.180 ms 47.221 ms 59.678 ms 16 [AS852] uq-membre.risq.net (206.167.128.90) 49.880 ms 47.619 ms 49.141 ms 17 * * * 18 [AS376] 132.213.254.254 (132.213.254.254) 3790.347 ms !H 3907.398 ms !H 4000.063 ms !H -- - (2^(N-1))

5 6

Re: [outages] Outages in Hawaii?
by Scott Weeks 15 May '12

15 May '12

--- surfer(a)mauigateway.com wrote: --- Brent.Roberts(a)progressive-solutions.com wrote: I have 3 circuits in the Area and all have been flapping all day. When I opened the tickets with the provider this is the Verbal speel I got. At last check the flakiest circuit has been stable for about an hour & a half. ------------------------------------------------ Ah, ok. And it appears to be HT only. Thanks for the info. ------------------------------------------------ Just a brief update. Please direct any other conversation to the discuss list... It does appear to be HT only. My circuit to them was still down this morning as I left for work. It was down all day yesterday (HST). No troubles with my Oceanic TW yesterday or this morning. Also, I have found nothing in the media or at the HT site, which doesn't seem to be kept up to date any longer: http://www.hawaiiantel.com/Support/ResidentialSupport/SupportAlertCategory/… scott (who is happy he doesn't work for HT at this moment as those're hell days where you may be up for 30-40 hours) ----------- -------

1 0

Re: [outages] Cogent issues in New Orleans
by Troy Bourque 14 May '12

14 May '12

Folks.... FYI we peer at 38.104.196.201 with Cogent. Frank reminded me that I did not mention our peering point. thanks Frank. troy From: outages-bounces(a)outages.org [mailto:outages-bounces@outages.org] On Behalf Of Troy Bourque Sent: Monday, May 14, 2012 4:23 PM To: outages(a)outages.org Subject: [outages] Cogent issues in New Orleans We're getting a few spikes here in there in our internet bound traffic through the cogent feed. One of their routers is spiking in latency intermittently and slowing our upstream speeds down. Anyone seeing anything similar events? Troy, Troy Bourque, MBA CCNA, CQS-CWLSS Troy(a)idsno.com

1 0

Re: [outages] Outages in Hawaii?
by Scott Weeks 14 May '12

14 May '12

--- Brent.Roberts(a)progressive-solutions.com wrote: From: "Roberts, Brent" <Brent.Roberts(a)progressive-solutions.com> I have 3 circuits in the Area and all have been flapping all day. When I opened the tickets with the provider this is the Verbal speel I got. At last check the flakiest circuit has been stable for about an hour & a half. ------------------------------------------------ Ah, ok. And it appears to be HT only. Thanks for the info. scott ----------- -------

1 0

Re: [outages] Outages in Hawaii?
by Scott Weeks 14 May '12

14 May '12

-----Original Message----- --- surfer(a)mauigateway.com wrote: --- Brent.Roberts(a)progressive-solutions.com wrote: TW/HawaiianTel is reporting an outage in Hawaii as an FYI. Multiple Last mile breaks. Existing Breaks specifically on Oahu at the moment. ----------------------------------------------------- Which one? TW or Hawaiian Telcom or both? Where did you hear it from? Got link? :) ----------------------------------------------------- Looks like it's HT only. I found this: http://www.aroundhawaii.com/support/cable_status/ No trouble reported there... --- Brent.Roberts(a)progressive-solutions.com wrote: I was going to make mention to last Wednesday's "maintenance". And that the True last mile section was via HT to a few different endpoints. I was told that this was something left over from last week's work. ------------------------------------------------------- Can you point to a source URL, or is this known from some circuits you have here? I did (and still see) my home circuit to them down. My home circuit to Oceanic TW doesn't seem to be affected, though. scott ---------- ----------- --------- _______________________________________________ Outages mailing list Outages(a)outages.org https://puck.nether.net/mailman/listinfo/outages ________________________________ This email and any attached files may contain confidential and/or privileged material and is intended solely for the use of the person to whom it is addressed. Any review, retransmission, dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender immediately and delete it and all attachments from your computer. Progressive Solutions is not liable for any errors or omissions in the content or transmission of this email.

2 1