February 2021 - Outages - Outages Lists

IPv4 Issues Accessing apnews.com
by Erik Ogren 27 Feb '21

27 Feb '21

Is anybody else experiencing reachability issues getting to https://apnews.com over IPv4? If IPv6 is used it seems to work fine but if IPv4 is used occasionally we're confronted by a custom Google 404 error page. Sometimes you can get the 404, refresh, get the expected/full APNews site, refresh again, then get another 404. APNews seems to be protected/hosted through Google's Project Shield. We're in the Mountain West region (Wyoming/Montana) and I have validated this using our own network (AS4474), Charter (Formerly Bresnan)/AS33588, AT&T cellular/AS20057, and CenturyLink/AS209 across multiple browsers and multiple devices with the same results. All devices across the various networks are resolving "apnews.com" to 34.96.67.30. We have a couple different connections to Google ASN15169 in different cities which I've tried to shift the egress traffic from our network towards 34.96.0.0/17 between the various Google datacenters we're connected to but it didn't seem to help. I did raise a "Reachability" ticket on our Google ISP portal for their NOC to investigate, was just curious how widespread this might be since none of the "usual" indicators like downdetector seem to be indicating a widespread issue at least not that I could see. Google Cloud's status page also doesn't seem to indicate any active issues, granted Project Shield is not explicitly mentioned there. Thank you, Erik Ogren Network Administrator (TCT AS4474)

9 10

Level3 or NTT US New Jersey outage
by Goli, Matthew W. 24 Feb '21

24 Feb '21

Is anyone aware of a major outage on the east coast, or New Jersey specifically that’s affecting either some Level3 or NTT circuits. I had a WAN circuit go down at 6:09 pm Eastern tonight. The circuits NOC claims it’s a known major outage with the local carrier but they aren’t sharing any more details at this time. TIA. Sent from my iPhone CONFIDENTIALITY AND LEGAL NOTICE This electronic mail transmission is confidential, may be privileged and should be read or retained only by the intended recipient. If the reader of this transmission is not the intended recipient, you are hereby notified that any distribution or copying hereof is strictly prohibited. If you have received this transmission in error, please immediately notify the sender and remove it from your system. E-mail is not necessarily a secure communication method, therefore the sender shall not be responsible for any changes that occur during its transfer. All e-mails are scanned for viruses, however, files attached to this e-mail may contain viruses that could harm the systems of the recipient. Any opinions expressed in this email must be confirmed in writing and signed by the sender to have legal validity.

3 3

Recovered 21:40:18 EST - Re: Massive Spectrum outage in the Northwest LA suburbs
by Cary Wiedemann 22 Feb '21

22 Feb '21

We have full recovery as of 21:40:18 EST / 18:48:18 PST. Massive, but at least brief. Thanks all for your reports. - Cary On Mon, Feb 22, 2021 at 9:30 PM Cary Wiedemann <carywiedemann(a)gmail.com> wrote: > Early report that I have several Spectrum sites down in the Northwestern > Los Angeles suburbs from Santa Clarita to Thousands Oaks and everything in > between. > > Our LA proper circuits seem unaffected. > > Outages of this scale are rare. This goes well beyond a single node (or > hub) outage. We're dealing with 35 sq miles here at least. Power and AT&T > service confirmed still operational. > > Event start time 2/22/21 20:59:58 EST / 17:59:58 PST. > > FYI for all. > > - Cary > > >

1 0

Fwd: TimeWarner/Spectrum down?
by William Kern 22 Feb '21

22 Feb '21

yeah sorry, I hit send as I was adding that. Anyway, yes, we are in Thousand Oaks, and its a dedicated circuit. Customers with Spectrum home seems to be down and was pointed out DownDetector is all red -bill On 2/22/21 6:31 PM, Joseph Jackson wrote: > You are going to get a bunch of flak on the list by not having more > details like - location and type of service. > > > > -----Original Message----- > From: Outages [mailto:outages-bounces@outages.org] On Behalf Of > William Kern via Outages > Sent: Monday, February 22, 2021 8:27 PM > To: outages(a)outages.org > Subject: [outages] TimeWarner/Spectrum down? > > Lost our connection. > > Cant reach their side from outside either. > > William Kern > > PixelGate Networks > > _______________________________________________ > Outages mailing list > Outages(a)outages.org > https://puck.nether.net/mailman/listinfo/outages

1 0

Massive Spectrum outage in the Northwest LA suburbs
by Cary Wiedemann 22 Feb '21

22 Feb '21

Early report that I have several Spectrum sites down in the Northwestern Los Angeles suburbs from Santa Clarita to Thousands Oaks and everything in between. Our LA proper circuits seem unaffected. Outages of this scale are rare. This goes well beyond a single node (or hub) outage. We're dealing with 35 sq miles here at least. Power and AT&T service confirmed still operational. Event start time 2/22/21 20:59:58 EST / 17:59:58 PST. FYI for all. - Cary

1 0

Spectrum issues in LA
by James Sharp 22 Feb '21

22 Feb '21

Looks like Spectrum is hard down, at least in the San Fernando valley. Several coworkers are out and our dedicated Spectrum circuit from The Valley to downtown is out. Downdetector is going nuts.

1 0

TimeWarner/Spectrum down?
by William Kern 22 Feb '21

22 Feb '21

Lost our connection. Cant reach their side from outside either. William Kern PixelGate Networks

1 0

Lumen Issues in Dallas
by Hershberger, Jeff 22 Feb '21

22 Feb '21

Is anyone else seeing issues with Lumen in the Dallas area? We are seeing SIP and IP issues on multiple links. Thanks, Jeff Hershberger | Telecom Manager 2323 Ross Avenue, Suite 600 | Dallas, TX | 75201 V: (214) 953-5748 | F: (214) 661-6665 | jhershberger(a)jw.com<mailto:jhershberger@jw.com> [cid:CF52621A-DBF7-42CC-8289-0DAD97F4F825@jwllp.com]

3 2

automated response
by Lenny 19 Feb '21

19 Feb '21

Hi! I'm at home with no power or internet or cell service for the next couple days - Try my cell at 304-546-2385 or call Quinn at 304-414-8000 if urgent. Thanks! Lenny

2 1

from Verizon FiOS to cogent packets being lost - NYC Region
by Izzy Goldstein - TeleGo 19 Feb '21

19 Feb '21

seems a issue between Verizon fios reaching Cogent packets being lost i can't PING IP with cogent from certain FiOS connection, packets from fios to cogent are lost, we also experienced issues with out SIP traffic, once our data center routed away from using cogents hop, issue has been resolved any verizon or cogent engineers on here who i can work with to provide some traces and hopefully you can resolve this ? -- Izzy Goldstein Chief Technology Officer Main: (212) 477-1000 x 2085 <(212)%20477-1000> Direct: (929) 477-2085 Website: www.telego.com <http://www.telego.net/> <http://www.telego.com/> Confidentiality Notice: This e-mail may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error please notify us immediately by email reply and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of TeleGo (T). Employees of TeleGo are expressly required not to make defamatory statements and not to infringe or authorize any infringement of copyright or any other legal right by email communications. Any such communication is contrary to TeleGo policy and outside the scope of the employment of the individual concerned. TeleGo will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising. TeleGo Hosted PBX <https://youtu.be/DaT8YAZ4V0w>

1 0

CloudFlare DNS and 502 errors
by Daniel Marks 19 Feb '21

19 Feb '21

Anyone noticing issues with CloudFlare? We’re getting failed DNS requests and 502 Errors on the dashboard. Mainly seeing this in Detroit but we’re also seeing similar issues in Osaka. -- Sent from my iPhone

7 7

FiOS issue in NYC Area?
by Izzy Goldstein - TeleGo 16 Feb '21

16 Feb '21

Many of our clients who are using fios are reporting issues with their voip phones when i switch them to their backup ISP (Optimum etc) it works fine anyone else experiencing any fios issues ? -- Izzy Goldstein Chief Technology Officer Main: (212) 477-1000 x 2085 <(212)%20477-1000> Direct: (929) 477-2085 Website: www.telego.com <http://www.telego.net/> <http://www.telego.com/> Confidentiality Notice: This e-mail may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error please notify us immediately by email reply and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of TeleGo (T). Employees of TeleGo are expressly required not to make defamatory statements and not to infringe or authorize any infringement of copyright or any other legal right by email communications. Any such communication is contrary to TeleGo policy and outside the scope of the employment of the individual concerned. TeleGo will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising. TeleGo Hosted PBX <https://youtu.be/DaT8YAZ4V0w>

7 18

Southwest Power Pool down
by Andy Ringsmuth 15 Feb '21

15 Feb '21

Well, electricity too in many places, but their web site: https://www.spp.org/ is also down. As recent (within the last hour) news reports have indicated, the Southwest Power Pool, which manages electricity supply for more than 14 states, is experiencing insane demand due to the equally insane cold temperatures across the United States. They are working with their member organizations/power companies to implement rolling blackouts in an effort to preserve the entire electricity grid. Wouldn’t surprise me if their site was down simply because millions of people in affected areas tried to look them up. Just a hunch though. ---- Andy Ringsmuth 5609 Harding Drive Lincoln, NE 68521-5831 (402) 304-0083 andy(a)andyring.com “Better even die free, than to live slaves.” - Frederick Douglas, 1863

2 1

Wash DC / Ashburn outage
by Brian Knight 15 Feb '21

15 Feb '21

We have a couple of circuits down since Saturday afternoon. Latest update from VZ is, they are awaiting access from the local transit authority to get access to the tunnels. Not too much more than that. Anyone have further info? Thanks, -Brian

1 0

Outage: PSN, many games
by Ben Salvidrim 10 Feb '21

10 Feb '21

Not very telecom-related but: PSN, as well as several online games (Warzone, Apex Legends, among others), have been offline for an hour or more, leading me to suspect the cause is a bigger outage of an underlying cloud service these might all be using. https://status.playstation.com/en-us https://support.activision.com/onlineservices/event-details.html?event=9957… Ben Landry Salvidrim<https://www.salvidrim.net/> ⁂ English Wikipedia: 8+ years of editing experience · Former Administrator / SPI / UTRS / OTRS (2013–2018) ⁂ Work: Telecom coordinator for Luminet<https://luminet.ca/> / Dimension<http://www.dimension10.ca/>10<http://www.dimension10.ca/> · Freelance administrative assistant on my own time ⁂ Personal: ☎ 514-691-6738 ✉ sal<mailto:salvidrim@salvidrim.net>vidrim(a)salvidrim.net<mailto:salvidrim@salvidrim.net> ⇄ https://www.linkedin.com/in/salvidrim/

2 2

DFW Issues
by aaron1＠gvtc.com 09 Feb '21

09 Feb '21

Yes, me too. I'm in Northern San Antonio I just got off the phone with cogent ticket number for our circuit - xxxxxxxxx ticket number link to master ticket - 11498067 master ticket details of known outage... cogent said span down el paso and dallas (or fort worth, don't recall what she said) ...and... cogent said span down san antonio and houston Aaron aaron1(a)gvtc.com

1 0

DFW Issues
by Lee Riemer 09 Feb '21

09 Feb '21

I'm seeing high latency and loss through multiple providers in DFW, Cogent, Bandwidth, Zayo. Anyone else noticing?

2 1

burma hijacks twitter
by Randy Bush 05 Feb '21

05 Feb '21

i am amused that this list has not mentioned that burma, aka myanmar, hijacked twitter some hours back. a la pakistan and youtube back when. but this time, it only seems to have affected se asia. this is really really good news, as it means major intercontinentals did not accept the hijack. and it was not origin validation, as there is no roa for the prefix which was hijacked. randy

1 0

Cogent Florida Issues
by Nick Olsen 05 Feb '21

05 Feb '21

Seeing weird issues with Cogent AS174 in Tampa. Saw multiple blips in transit and Layer 2 circuits. Cogent NOC tech was super transparent, said the cogent device we were hooked to lost multiple BGP sessions. Also stated that their NOC was on fire with calls from Miami, Orlando, Tampa..etc. I've since de-peered them for transit. Occasionally still seeing traffic inbound on the circuit leading me to believe they've got latent routes of mine ping-ponging around the network.

2 1

CCI Southern Minnesota inbound connections from other providers failing
by Nathan Engelhardt 04 Feb '21

04 Feb '21

Outbound connections succeed, and inbound connections from other CCI sites succeed. Inbound connections from other providers fail. Charter/Spectrum to CCI: Tracing route to redacted [209.240.redacted] over a maximum of 30 hops: 1 <1 ms <1 ms <1 ms redacted.com [10.0.0.1] 2 * * * Request timed out. 3 17 ms 12 ms 10 ms 096-034-026-152.biz.spectrum.com [96.34.26.152] 4 14 ms 17 ms 13 ms cts01rhnlwi-tge-3-0-0.rhnl.wi.charter.com [96.34.16.147] 5 19 ms 15 ms 15 ms bbr02slidla-tge-0-1-0-6.slid.la.charter.com [96.34.1.188] 6 33 ms 21 ms 26 ms prr01mplsmn-bue-1.mpls.mn.charter.com [96.34.3.31] 7 25 ms 29 ms 21 ms AS12042.micemn.net [206.108.255.18] 8 * * * Request timed out. 9 * * * Request timed out. 10 * * * Request timed out. 11 * * * Request timed out. 12 * * * Request timed out. 13 * * * Request timed out. 14 * * * Request timed out. 15 * * * Request timed out. 16 * * * Request timed out. 17 * * * Request timed out. 18 * * * Request timed out. 19 * * * Request timed out. 20 * * * Request timed out. 21 * * * Request timed out. 22 * * * Request timed out. 23 * * * Request timed out. 24 * * * Request timed out. 25 * * * Request timed out. 26 * * * Request timed out. 27 * * * Request timed out. 28 * * * Request timed out. 29 * * * Request timed out. 30 * * * Request timed out.

2 1

Small Comcast outage southern suburb of Pittsburgh
by Allen Kitchen 03 Feb '21

03 Feb '21

Apparently quite small and localized, centering around south – southwest PGH suburb of Dormont, PA. Allen Kitchen / DSSC Solutions Co, Pittsburgh (Dormont) PA

1 0

Google Finance down?
by Hank Nussbacher 02 Feb '21

02 Feb '21

4 4