August 2013 - Outages - Outages Lists

fiber cut - NOLA - Houston
by Ben Bartsch 28 Aug '13

28 Aug '13

we have confirmed a cut between NOLA on our circuit to Houston. we are supposed to be on a protected L3 path (same thing happened less than a year ago near Lake Charles, LA) but we are down. FYI.

2 1

Level 3 Peering with Verizon/MCI
by Daniel Green 28 Aug '13

28 Aug '13

Anyone else seeing huge latency between level 3 and Verizon this morning? I am have complaints of voice quality and call failures tracing to certain locations my times start jumping up. traceroute 98.109.171.43 Type escape sequence to abort. Tracing the route to static-98-109-171-43.nwrknj.fios.verizon.net (98.109.171.43) 1 4 msec 0 msec 0 msec 2 ge-6-18.car1.LasVegas1.Level3.net (4.71.176.101) 68 msec 4 msec 4 msec 3 ae-2-5.bar1.LasVegas1.Level3.net (4.69.148.130) [MPLS: Label 306688 Exp 0] 0 msec 0 msec 4 msec 4 ae-0-11.bar2.LasVegas1.Level3.net (4.69.148.126) [MPLS: Label 303936 Exp 0] 0 msec 0 msec 0 msec 5 ae-6-6.ebr1.Denver1.Level3.net (4.69.148.2) [MPLS: Label 1948 Exp 0] 36 msec 36 msec 32 msec 6 ae-2-2.ebr2.Dallas1.Level3.net (4.69.132.106) [MPLS: Label 1935 Exp 0] 36 msec 36 msec 32 msec 7 ae-62-62.csw1.Dallas1.Level3.net (4.69.151.129) [MPLS: Label 1921 Exp 0] 36 msec 36 msec ae-92-92.csw4.Dallas1.Level3.net (4.69.151.165) [MPLS: Label 1728 Exp 0] 32 msec 8 * 4.69.145.82 32 msec * 9 MCI-level3-ae.dallas3.level3.net (4.68.62.34) 352 msec mci-level3-30g.dallas3.level3.net (4.68.62.166) 136 msec 420 msec 10 P0-15-0-0.NWRKNJ-LCR-22.verizon-gni.net (130.81.199.17) 384 msec 348 msec P0-4-0-0.NWRKNJ-LCR-22.verizon-gni.net (130.81.22.65) 176 msec 11 * * * 12 * * static-98-109-171-43.nwrknj.fios.verizon.net (98.109.171.43) 336 msec tracert 98.109.171.43 Tracing route to static-98-109-171-43.nwrknj.fios.verizon.net [98.109.171.43] over a maximum of 30 hops: 1 <1 ms <1 ms <1 ms 2 1 ms 1 ms 1 ms 3 4 ms 4 ms 4 ms atlanta2.level3.net 4 24 ms 24 ms 24 ms ae-13-13.car1.atlanta2.level3.net [4.69.151.17] 5 26 ms 24 ms 24 ms vlan52.ebr2.atlanta2.level3.net [4.69.150.126] 6 24 ms 24 ms 24 ms ae-73-73.ebr3.atlanta2.level3.net [4.69.148.253] 7 24 ms 24 ms 24 ms ae-7-7.ebr3.dallas1.level3.net [4.69.134.21] 8 24 ms 24 ms 24 ms ae-83-83.csw3.dallas1.level3.net [4.69.151.157] 9 24 ms 24 ms 24 ms ae-3-80.edge10.dallas1.level3.net [4.69.145.146] 10 332 ms 329 ms * mci-level3-ae.dallas3.level3.net [4.68.62.34] 11 341 ms 343 ms 339 ms p0-4-0-0.nwrknj-lcr-21.verizon-gni.net [130.81.2 2.63] 12 * * * Request timed out. 13 352 ms 356 ms 359 ms static-98-109-171-43.nwrknj.fios.verizon.net [98 .109.171.43] --- Respectfully, Daniel L. Green II Network Engineer Momentum | www.gomomentum.com<http://www.gomomentum.com/> O: 702.933.1545 Ext: 1337 W: 702.403.1234 F: 702.365.8647 Facebook<http://www.facebook.com/MomentumBusiness> | Twitter<https://twitter.com/#!/MomentumTelecom> | LinkedIn<http://www.linkedin.com/company/55869> [Description: Description: Description: Momentum Logo] 8350 S. Durango | Suite 220 | Las Vegas | Nevada | 89113

4 3

nytime.com dns borked
by Grant Ridder 27 Aug '13

27 Aug '13

I think someone hijacked NYTimes dns... http://www.chicagotribune.com/business/technology/chi-new-york-times-websit… Non-authoritative answer: Name: nytimes.com Address: 141.105.64.37 ~~~ dig any nytimes.com ; <<>> DiG 9.8.3-P1 <<>> any nytimes.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15335 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 2 ;; QUESTION SECTION: ;nytimes.com. IN ANY ;; ANSWER SECTION: nytimes.com. 11560 IN A 141.105.64.37 nytimes.com. 5 IN NS ns1.syrianelectronicarmy.com. nytimes.com. 5 IN NS ns2.syrianelectronicarmy.com. ;; ADDITIONAL SECTION: ns1.syrianelectronicarmy.com. 47 IN A 141.105.64.37 ns2.syrianelectronicarmy.com. 47 IN A 141.105.64.37

12 12

Fwd: HostGator down?
by Neil Ticktin 25 Aug '13

25 Aug '13

It appears to not be down -- but suddenly, every one of my networks is blocked. (Multiple IP blocks). I can see our server is up when I go over cellular. The IPs are not listed in our box's firewall. Neils-MacBook-Pro-Retina:~ neil$ traceroute forums.applecentral.com traceroute to hostgator.xplain.com (142.4.22.215), 64 hops max, 52 byte packets 1 10.20.30.1 (10.20.30.1) 1.824 ms 10.707 ms 4.159 ms 2 cpe-76-91-96-1.socal.res.rr.com (76.91.96.1) 194.445 ms 26.414 ms 31.556 ms 3 tge7-1.wlvgcabn-cer01.socal.rr.com (76.166.19.161) 11.092 ms 15.447 ms 13.650 ms 4 tge0-8-0-9.vnnycajz-ccr01.socal.rr.com (72.129.13.98) 17.038 ms 23.493 ms 19.795 ms 5 agg29.lsancarc-ccr01.socal.rr.com (72.129.13.0) 16.988 ms 20.820 ms 20.948 ms 6 107.14.19.32 (107.14.19.32) 22.480 ms 18.375 ms 19.711 ms 7 * ae0.pr1.lax10.tbone.rr.com (107.14.19.50) 84.856 ms * 8 xe-2-0-4.ar1.lax1.us.nlayer.net (69.31.127.73) 1660.474 ms * 862.011 ms 9 ae0-110g.cr1.lax1.us.nlayer.net (69.31.127.141) 40.289 ms ae0-110g.cr1.lax1.us.nlayer.net (69.31.127.137) 16.043 ms ae0-110g.cr1.lax1.us.nlayer.net (69.31.127.141) 16.887 ms 10 ae4-50g.cr1.lax2.us.nlayer.net (69.31.124.114) 17.750 ms 20.795 ms 18.714 ms 11 ae2-50g.ar2.lax2.us.nlayer.net (69.31.127.89) 18.248 ms 35.168 ms 24.975 ms 12 as11798.xe-2-0-7.ar2.lax2.us.nlayer.net (69.31.127.214) 21.488 ms 30.395 ms 31.875 ms 13 ve56.ar04.prov.acedc.net (199.58.196.81) 86.739 ms 40.538 ms 58.246 ms 14 rtr-a.unifiedlayer.com (199.58.199.114) 38.192 ms 37.956 ms 36.955 ms 15 mlx02.prov.unifiedlayer.com (69.195.64.42) 43.098 ms 36.830 ms 38.708 ms 16 * * * 17 * * * 18 * * * 19 * * * 20 * *^C Neils-MacBook-Pro-Retina:~ neil$ ---------- Forwarded message ---------- From: Jason Kline <teile1(a)gmail.com> Date: Sun, Aug 25, 2013 at 4:43 PM Subject: Re: [outages] HostGator down? To: Neil Ticktin <neil-lists(a)xplain.com> Got any trace routes? Which location is your server located? *Sent from my Verizon Wireless 4G LTE DROID* Neil Ticktin <neil-lists(a)xplain.com> wrote: Anyone noticing HostGator down again? We're contacting them about our server -- but trying to figure out if it's wider spread or not … especially given the outage from a couple of weeks ago during the "upgrade". Thanks! Neil

1 0

HostGator down?
by Neil Ticktin 25 Aug '13

25 Aug '13

Anyone noticing HostGator down again? We're contacting them about our server -- but trying to figure out if it's wider spread or not … especially given the outage from a couple of weeks ago during the "upgrade". Thanks! Neil

2 1

Feedly borked?
by Paul Ferguson 22 Aug '13

22 Aug '13

Getting no content at this moment. I am a "lifetime" Feedly Pro user, and yes, I have a message out to them on status. But figured it might be worth asking others who use Feedly if they are experiencing the same thing. Thanks, - ferg -- "Fergie", a.k.a. Paul Ferguson fergdawgster(at)gmail.com

2 2

Bandwidth (P2) Service Impairment - Dash Global - Final
by Chris Stone 20 Aug '13

20 Aug '13

*Final Update:* Bandwidth reports that all services to the network have been restored.

1 0

Update: Bandwidth.com VoIP issue on Denver POP
by Chris Stone 20 Aug '13

20 Aug '13

We are experiencing indications of a Network Impairment Issue on the Bandwidth network. If you determine you are having any issues, please open a ticket and our NOC team will assess and determine if it is related to the following issue: *Update:* We are still experiencing issues with the Denver SBC. Bandwidth engineering currently investigating.

1 0

Bandwidth.com VoIP outage update for Denver POP
by Chris Stone 20 Aug '13

20 Aug '13

We are experiencing indications of a Network Impairment Issue on the Bandwidth network. If you determine you are having any issues, please open a ticket and our NOC team will assess and determine if it is related to the following issue: *Update:* Currently we believe service levels have been restored, however we will be monitoring for the next 30 minutes to verify stability.

1 0

Bandwidth.com VoIP outage update
by Chris Stone 20 Aug '13

20 Aug '13

*Update:* The Denver SBC is processing calls again, we are still in the process of working on it. You may experience some audio quality issues in the meantime. More updates will follow.

1 0