December 2018 - Outages - Outages Lists

Cox outage?
by Rob Hall 27 Mar '19

27 Mar '19

I'm getting reports that there's a nationwide Cox outage ; anyone else seeing this? Thanks! -- Rob Hall, CCNP R&S, CCP-V, CCA-N I.T. Manager @Work Personnel 1040 Tidewater Court Kingsport, TN 37660 Office: (423)272-7897, x10113 Mobile: (423)312-6003

5 6

Re: [outages] Outages Digest, Vol 127, Issue 31
by Aaron Henderson 28 Dec '18

28 Dec '18

My apologies if this message breaks the mailing list, I have never replied to one of these before. I work for a rural ISP, we do not use CenturyLink as an upstream; however, some of our customers are reporting issues since the outage. Management has tasked my office with putting together a timeline/details about the outage to brief upper management. Is there a detailed timeline of events documenting this outage? Thanks, A On Fri, Dec 28, 2018 at 12:13 PM <outages-request(a)outages.org> wrote: > Send Outages mailing list submissions to > outages(a)outages.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://puck.nether.net/mailman/listinfo/outages > or, via email, send a message with subject or body 'help' to > outages-request(a)outages.org > > You can reach the person managing the list at > outages-owner(a)outages.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Outages digest..." > > > Today's Topics: > > 1. Re: CenturyLink Outages this morning (Aaron C. de Bruyn) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 28 Dec 2018 09:12:03 -0800 > From: "Aaron C. de Bruyn" <aaron(a)heyaaron.com> > To: Dom Colangelo <dom.colangelo(a)simmons.edu> > Cc: Bob Antia <antia(a)ba14.com>, outages(a)outages.org > Subject: Re: [outages] CenturyLink Outages this morning > Message-ID: > <CAEE+rGqaysPos= > VB_QtS++CBs1bNuRzXcLig0iBmntLQcVVk4w(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Several times late last night we had our weather alert radio activate and > broadcast an unintelligible message about 911 being down. It wasn't > reception problems. The cadence of the dumb computer voice made it nearly > impossible to understand. > > https://photos.app.goo.gl/M4spwKAkKPnoSrJw9 > > The alert came through ~5 hours after 911 went offline everywhere. > > -A > > > On Fri, Dec 28, 2018, 08:44 Dom Colangelo via Outages <outages(a)outages.org > wrote: > > > Received a WEA from MEMA in Boston, MA noting that 911 is down for > > cellular users. > > > > Dom Colangelo > > > > Senior Windows Systems Administrator > > > > Infrastructure and Online Services > > > > Simmons University > > > > 617.521.2669 > > > > > > On Fri, Dec 28, 2018 at 9:56 AM Bob Antia via Outages < > outages(a)outages.org> > > wrote: > > > >> Massachusetts emergency management agency (MEMA) sent this out this > >> morning and followed up at 0736 that the issues had been resolved. > >> > >> > >> *Massachusetts Emergency Management Agency* > >> ------------------------------ > >> > >> *Massachusetts Emergency Management Agency* > >> > >> *Situational Awareness Statement* > >> > >> *December 28, 2018 7:00 AM* > >> > >> > >> > >> *Re: Cellular 911 Experiencing Technical Difficulties and Sporadic > >> Outages* > >> > >> > >> > >> The Massachusetts Emergency Management Agency is sharing the following > >> information from the Commonwealth Fusion Center (CFC) with our local > >> partners and stakeholders. MEMA also is in touch with the State 911 > >> Department. > >> > >> > >> > >> *From the Commonwealth Fusion Center* > >> > >> > >> > >> *Situation:* > >> > >> A nationwide outage affecting CenturyLink, an Internet communications > >> company, is impacting wireless 911 call capability in various parts of > the > >> country. We believe Massachusetts has been impacted by the outage and > urge > >> residents using cellular phones, if necessary, to contact emergency > >> first-responders by using the full 10-digit number of a public safety > >> answering point emergency call center (PSAP) or their local police > >> department. > >> > >> > >> > >> The Massachusetts State Police operate three regional PSAPs and a fourth > >> is staffed by State 911 at MSP headquarters in Framingham. > >> Residents using a cell phone to call for emergency services should first > >> try to dial 911, and if it does not work, instead call one of these four > >> 10-digit numbers: > >> Shelburne Control PSAP 413-625-8200 > >> New Braintree Control PSAP 508-867-1170 > >> Northampton Control PSAP 413-586-1508 > >> Framingham PSAP 508-820-2121. > >> > >> > >> Residents using cell phones may also, if dialing 911 does not work, may > >> call the 10-digit general number of their local police department. They > may > >> also call the 10-digit number of a State Police Barracks in their area > >> (search for barracks location and phone numbers on the MSP web site at > >> www.mass.gov and search for State Police locations). > >> Other agencies operate additional PSAPs in Massachusetts beyond those > >> listed above. > >> > >> > >> > >> The outage seems to only be affecting wireless 911 calls; landline calls > >> to 911 appear to be working. > >> > >> > >> CenturyLink has indicted it is working on repairing what the company > >> called a ?network element? that is causing the problem. > >> > >> > >> > >> *MEMA Operations* > >> > >> The State Emergency Operations Center (SEOC) is currently operating at > >> Level 1 (Steady State Monitoring). MEMA will continue to monitor the > >> situation and will disseminate additional Situational Awareness > Statements > >> as necessary. Cities and towns wishing to share information or request > >> assistance should contact their MEMA Regional office or MEMA?s 24/7 > >> communications center at 508-820-2000. > >> > >> *Stay Informed:* > >> > >> Utilize *Massachusetts Alerts* to receive emergency notifications and > >> information from the Massachusetts Emergency Management Agency and the > >> National Weather Service. *Massachusetts Alerts* is a free app that is > >> available for Android and iPhones. To learn more about *Massachusetts > >> Alerts*, and for information on how to download the free app onto your > >> smartphone, visit: > >> http://www.mass.gov/service-details/massachusetts-alerts-smartphone-app > . > >> > >> Utilize MEMA?s real-time power outage viewer to stay informed about > >> current power outages in your community and region, and across the > state, > >> including information from utility companies about restoration times: > >> http://mema.mapsonline.net/public.html > >> > >> Utilize MEMA?s live weather radar and forecasting tools: > >> http://www.mass.gov/map-resources > >> > >> > >> *Online Resources:*For additional information and resources, visit: > >> Massachusetts Emergency Management Agency at www.mass.gov/mema > >> MEMA's Facebook page: http://www.facebook.com/MassachusettsEMA > >> MEMA Twitter: @MassEMA <http://twitter.com/massema> > >> Federal Emergency Management Agency at www.fema.gov > >> National Weather Service Boston/Norton at www.weather.gov/boston > >> National Weather Service Albany, NY at www.weather.gov/albany > >> National Weather Service Weather Prediction Center: > >> http://www.wpc.ncep.noaa.gov > >> National Weather Service Storm Prediction Center: > >> http://www.spc.noaa.gov/ > >> Mass211 at www.mass211.org > >> > >> > >> > >> > >> ------------------------------ > >> > >> This message has been sent from MEMA's ESF Team Listserv. > >> To unsubscribe from this listserv, call MEMA Operations at (508) > 820-2000. > >> > >> +1.781.300.0000 > >> > >> > >> On Fri, Dec 28, 2018 at 9:52 AM Matt Brennan via Outages < > >> outages(a)outages.org> wrote: > >> > >>> Most public safety agencies in Massachusetts posted on Facebook and > >>> Twitter between 6 AM and 8 AM EST about wireless 911 outages caused by > >>> CenturyLink. It was affecting wireless routing, so some calls were > being > >>> misrouted to other towns, or even other states; some callers were > getting > >>> reorder tones or other error messages. The problem was reported to be > >>> resolved prior to 8 AM EST. > >>> > >>> Seeing reports from PSAP's that the issue started again in the last 15 > >>> minutes. > >>> > >>> This is wireless only. Landline 911 calls were / are not affected. > >>> > >>> > >>> On Fri, Dec 28, 2018 at 9:32 AM Mel Beckman via Outages < > >>> outages(a)outages.org> wrote: > >>> > >>>> Don't be a fear monger. I searched Facebook and found *three* posts > >>>> about 911 outages, all of them simply reposts of local news stories. > >>>> > >>>> Facebook is not "blowing up". > >>>> > >>>> Sheesh! > >>>> > >>>> [image: image1.PNG] > >>>> > >>>> -mel via cell > >>>> > >>>> -mel via cell > >>>> > >>>> On Dec 28, 2018, at 5:23 AM, Josh Luthman via Outages < > >>>> outages(a)outages.org> wrote: > >>>> > >>>> Facebook is blowing up with 911 outages and claiming it's CenturyLink > >>>> to blame. > >>>> > >>>> Josh Luthman > >>>> Office: 937-552-2340 > >>>> Direct: 937-552-2343 > >>>> 1100 Wayne St > >>>> Suite 1337 > >>>> Troy, OH 45373 > >>>> > >>>> On Fri, Dec 28, 2018, 2:09 AM Erik Sundberg via Outages < > >>>> outages(a)outages.org wrote: > >>>> > >>>>> Maybe light at the end of the tunnel... > >>>>> > >>>>> > >>>>> > >>>>> latest we got from Centurylink was at 10:45pm CST. > >>>>> > >>>>> Our engineers and technicians have identified the network element > that > >>>>> has affected our customer services. Services are restoring, and the > current > >>>>> estimated time for full recovery is four hours. > >>>>> > >>>>> CenturyLink will be conducting an extensive post-incident > >>>>> investigation and root cause analysis to provide follow-up > information to > >>>>> our customers impacted by this event. We will provide one more Sales > Alert > >>>>> from this address once services are fully restored. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> *From:* Rich Breton <richdbreton(a)gmail.com> > >>>>> *Sent:* Thursday, December 27, 2018 2:59 PM > >>>>> *To:* Christopher Conley <CConley(a)forsmarshgroup.com> > >>>>> *Cc:* Erik Sundberg <ESundberg(a)nitelusa.com>; outages(a)outages.org > >>>>> *Subject:* Re: [outages] CenturyLink Outages this morning > >>>>> > >>>>> > >>>>> > >>>>> I understand that the transport waves went out first, and then they > >>>>> try to reroute to something in Arizona that begin introducing all of > the > >>>>> broadcast traffic. How legit that is I?m not sure > >>>>> > >>>>> Sent from my iPhone > >>>>> > >>>>> > >>>>> On Dec 27, 2018, at 12:58 PM, Christopher Conley via Outages < > >>>>> outages(a)outages.org> wrote: > >>>>> > >>>>> > Finally received something useful from centurylink? I have more > >>>>> questions now like how broadcast traffic effect 4x Transport waves. > >>>>> > >>>>> > >>>>> > >>>>> Ineptitude. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> Christopher Conley > >>>>> > >>>>> Systems Administrator | Fors Marsh Group > >>>>> > >>>>> Arlington, VA > >>>>> > >>>>> > >>>>> > >>>>> *From:* Outages <outages-bounces(a)outages.org> *On Behalf Of *Erik > >>>>> Sundberg via Outages > >>>>> *Sent:* Thursday, December 27, 2018 2:46 PM > >>>>> *To:* outages(a)outages.org > >>>>> *Subject:* Re: [outages] CenturyLink Outages this morning > >>>>> > >>>>> > >>>>> > >>>>> Finally received something useful from centurylink? I have more > >>>>> questions now like how broadcast traffic effect 4x Transport waves. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> Update: > >>>>> > >>>>> On December 27, 2018 at 02:40 GMT, CenturyLink identified a service > >>>>> impact in New Orleans, LA. The NOC is engaged and investigating in > order to > >>>>> isolate the cause. Field Operations were engaged and dispatched for > >>>>> additional investigations. Tier IV Equipment Vendor Support was later > >>>>> engaged. During cooperative troubleshooting a device in San Antonio, > TX was > >>>>> isolated from the network as it was seeming to broadcast traffic > consuming > >>>>> capacity, which seemed to alleviate some impact. Investigations > remained > >>>>> ongoing. Following the isolation of the San Antonio, TX device > >>>>> troubleshooting efforts focused on additional sites that teams were > >>>>> remotely unable to troubleshoot. Field Operations were dispatched to > sites > >>>>> in Kansas City, MO, Atlanta, GA, New Orleans, LA and Chicago, IL. > Tier IV > >>>>> Equipment Vendor Support continued to investigate the equipment logs > to > >>>>> further assist with isolation. Once visibility was restored to the > site in > >>>>> Kansas City, MO a filter was applied to the equipment to further > alleviate > >>>>> the impact observed. All of the necessary troubleshooting teams in > >>>>> cooperation with Tier IV Equipment Vendor Support are working to > restore > >>>>> remote visibility to the remaining sites at this time. We understand > how > >>>>> important these services are to our clients and the issue has been > >>>>> escalated to the highest levels within CenturyLink Service Assurance > >>>>> Leadership. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> *From:* Outages <outages-bounces(a)outages.org> *On Behalf Of *Bruce > >>>>> Freshwater via Outages > >>>>> *Sent:* Thursday, December 27, 2018 1:24 PM > >>>>> *To:* outages(a)outages.org > >>>>> *Subject:* Re: [outages] CenturyLink Outages this morning > >>>>> > >>>>> > >>>>> > >>>>> CenturyLink finally acknowledged: > >>>>> > >>>>> Great Lakes Region CenturyLink Partners/Agents: > >>>>> > >>>>> > >>>>> > >>>>> FYI. Our network is experiencing a disruption affecting customer > >>>>> services. Customers are currently unable to generate tickets through > the > >>>>> Control Center portal. > >>>>> > >>>>> > >>>>> > >>>>> We know how important these services are to our customers, and we are > >>>>> working to restore services as quickly as possible. > >>>>> > >>>>> > >>>>> > >>>>> Our technical teams are working to resolve the issue, and we will > >>>>> provide updates as the situation progresses. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> -bruce > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On Thu, Dec 27, 2018 at 10:16 AM -0500, "Erik Sundberg via Outages" < > >>>>> outages(a)outages.org> wrote: > >>>>> > >>>>> So it's a bad day for centurylink. > >>>>> > >>>>> > >>>>> > >>>>> The lady who answered the phone for there NOC said that we are having > >>>>> a nation wide outage and we are unable to open tickets at this time. > Please > >>>>> call be in 60-90 mins. > >>>>> > >>>>> > >>>>> > >>>>> She had no info on what was going on > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> ------------------------------ > >>>>> > >>>>> *From:* Michael Bolton <mbolton(a)holmeselectricsecurity.com> > >>>>> *Sent:* Thursday, December 27, 2018 8:55:03 AM > >>>>> *To:* Erik Sundberg; outages(a)outages.org > >>>>> *Subject:* RE: CenturyLink Outages this morning > >>>>> > >>>>> > >>>>> > >>>>> Seeing the same in NC. Circuits are up but the routing between POPs > >>>>> has changed and we are seeing higher than normal latency. > >>>>> > >>>>> > >>>>> > >>>>> VoIP portal is down also. Allows you to login but you cannot make any > >>>>> changes or view any details. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> *From:* Outages <outages-bounces(a)outages.org> *On Behalf Of *Erik > >>>>> Sundberg via Outages > >>>>> *Sent:* Thursday, December 27, 2018 9:45 AM > >>>>> *To:* outages(a)outages.org > >>>>> *Subject:* [outages] CenturyLink Outages this morning > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> *CAUTION:* This email originated from outside of the organization. Do > >>>>> not click links or open attachments unless you recognize the sender > and > >>>>> know the content is safe. > >>>>> > >>>>> We are seeing a lot of centurylink circuits and services down this > >>>>> morning. > >>>>> > >>>>> > >>>>> > >>>>> Centurylink Control Center portal comes up but auth failing unable to > >>>>> login > >>>>> > >>>>> > >>>>> > >>>>> Waves down or bouncing > >>>>> > >>>>> Denver - Seattle (Bouncing) > >>>>> > >>>>> Denver - Chicago (Bouncing) > >>>>> > >>>>> New York - Los Angeles (Down) > >>>>> > >>>>> Chicago - Atlanta (Down) > >>>>> > >>>>> > >>>>> ENNI Down in Denver (Down) > >>>>> > >>>>> > >>>>> > >>>>> Still tying to open tickets with them there portal is not working and > >>>>> stuck on hold right now. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> ------------------------------ > >>>>> > >>>>> > >>>>> CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, > >>>>> files or previous e-mail messages attached to it may contain > confidential > >>>>> information that is legally privileged. If you are not the intended > >>>>> recipient, or a person responsible for delivering it to the intended > >>>>> recipient, you are hereby notified that any disclosure, copying, > >>>>> distribution or use of any of the information contained in or > attached to > >>>>> this transmission is STRICTLY PROHIBITED. If you have received this > >>>>> transmission in error please notify the sender immediately by > replying to > >>>>> this e-mail. You must destroy the original transmission and its > attachments > >>>>> without reading or saving in any manner. Thank you. > >>>>> > >>>>> IMPORTANT NOTICE: This e-mail message is intended to be received only > >>>>> by persons entitled to receive the confidential information it may > contain. > >>>>> E-mail messages to clients of Holmes Security Systems may contain > >>>>> information that is confidential and legally privileged. Please do > not > >>>>> read, copy, forward, or store this message unless you are an intended > >>>>> recipient of it. If you have received this message in error, please > forward > >>>>> it to the sender and delete it completely from your computer system. > >>>>> > >>>>> > >>>>> ------------------------------ > >>>>> > >>>>> > >>>>> CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, > >>>>> files or previous e-mail messages attached to it may contain > confidential > >>>>> information that is legally privileged. If you are not the intended > >>>>> recipient, or a person responsible for delivering it to the intended > >>>>> recipient, you are hereby notified that any disclosure, copying, > >>>>> distribution or use of any of the information contained in or > attached to > >>>>> this transmission is STRICTLY PROHIBITED. If you have received this > >>>>> transmission in error please notify the sender immediately by > replying to > >>>>> this e-mail. You must destroy the original transmission and its > attachments > >>>>> without reading or saving in any manner. Thank you. > >>>>> > >>>>> > >>>>> ------------------------------ > >>>>> > >>>>> > >>>>> CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, > >>>>> files or previous e-mail messages attached to it may contain > confidential > >>>>> information that is legally privileged. If you are not the intended > >>>>> recipient, or a person responsible for delivering it to the intended > >>>>> recipient, you are hereby notified that any disclosure, copying, > >>>>> distribution or use of any of the information contained in or > attached to > >>>>> this transmission is STRICTLY PROHIBITED. If you have received this > >>>>> transmission in error please notify the sender immediately by > replying to > >>>>> this e-mail. You must destroy the original transmission and its > attachments > >>>>> without reading or saving in any manner. Thank you. > >>>>> > >>>>> _______________________________________________ > >>>>> Outages mailing list > >>>>> Outages(a)outages.org > >>>>> https://puck.nether.net/mailman/listinfo/outages > >>>>> > >>>>> > >>>>> ------------------------------ > >>>>> > >>>>> CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, > >>>>> files or previous e-mail messages attached to it may contain > confidential > >>>>> information that is legally privileged. If you are not the intended > >>>>> recipient, or a person responsible for delivering it to the intended > >>>>> recipient, you are hereby notified that any disclosure, copying, > >>>>> distribution or use of any of the information contained in or > attached to > >>>>> this transmission is STRICTLY PROHIBITED. If you have received this > >>>>> transmission in error please notify the sender immediately by > replying to > >>>>> this e-mail. You must destroy the original transmission and its > attachments > >>>>> without reading or saving in any manner. Thank you. > >>>>> _______________________________________________ > >>>>> Outages mailing list > >>>>> Outages(a)outages.org > >>>>> https://puck.nether.net/mailman/listinfo/outages > >>>>> > >>>> _______________________________________________ > >>>> Outages mailing list > >>>> Outages(a)outages.org > >>>> https://puck.nether.net/mailman/listinfo/outages > >>>> > >>>> _______________________________________________ > >>>> Outages mailing list > >>>> Outages(a)outages.org > >>>> https://puck.nether.net/mailman/listinfo/outages > >>>> > >>> _______________________________________________ > >>> Outages mailing list > >>> Outages(a)outages.org > >>> https://puck.nether.net/mailman/listinfo/outages > >>> > >> _______________________________________________ > >> Outages mailing list > >> Outages(a)outages.org > >> https://puck.nether.net/mailman/listinfo/outages > >> > > _______________________________________________ > > Outages mailing list > > Outages(a)outages.org > > https://puck.nether.net/mailman/listinfo/outages > > >

2 1

Re: [outages] CenturyLink Outages this morning
by Chris Rogers 28 Dec '18

28 Dec '18

The only detail I think one could get would be from the CenturyLink status page (https://status.ctl.io/) You could also piece together information from the thread in the outages archives ( https://puck.nether.net/pipermail/outages/2018-December/thread.html) Best, Chris

1 0

CenturyLink Outages this morning
by Erik Sundberg 28 Dec '18

28 Dec '18

We are seeing a lot of centurylink circuits and services down this morning. Centurylink Control Center portal comes up but auth failing unable to login Waves down or bouncing Denver - Seattle (Bouncing) Denver - Chicago (Bouncing) New York - Los Angeles (Down) Chicago - Atlanta (Down) ENNI Down in Denver (Down) Still tying to open tickets with them there portal is not working and stuck on hold right now. ________________________________ CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You must destroy the original transmission and its attachments without reading or saving in any manner. Thank you.

16 19

CenturyLink Outage in Tennessee
by Jason Lester 26 Dec '18

26 Dec '18

Our Metro Ethernet Internet service with CenturyLink went down this morning. When I called in to their Advanced Network Services group, there's a recording saying they have a major transport outage in the Tennessee area. We're in Virginia, but right on the state line, so we go through their TN network. No ETR yet. -- Jason Lester Supervisor of Technology Washington County Public Schools Tel: 276-739-3060 Fax: 276-628-1893 http://www.wcs.k12.va.us *When you do things right, people won't be sure you've done anything at all.*

1 1

Re: [outages] usno.navy.mil disappeared?
by Weinberg, Matt 21 Dec '18

21 Dec '18

Warren is on the right track. More specifically, there are seven authoritative name servers for navy.mil. Three of them respond properly for usno.navy.mil: $ dig @ns.jtfgno.mil. usno.navy.mil soa +short psyche.usno.navy.mil. hostmaster.usno.navy.mil. 201709123 3600 1800 2147483647 3600 $ dig @uforiftr02.csd.disa.mil. usno.navy.mil soa +short psyche.usno.navy.mil. hostmaster.usno.navy.mil. 201709123 3600 1800 2147483647 3600 $ dig @updciftr02.csd.disa.mil. usno.navy.mil soa +short psyche.usno.navy.mil. hostmaster.usno.navy.mil. 201709123 3600 1800 2147483647 3600 Four of the authoritative name servers for navy.mil respond with NXDOMAIN for usno.navy.mil: $ dig @ns.cybercom.mil. usno.navy.mil soa $ dig @UFORIFTR01.CSD.DISA.MIL. usno.navy.mil soa $ dig @NS1.CSD.DISA.MIL. usno.navy.mil soa $ dig @UPDCIFTR01.CSD.DISA.MIL. usno.navy.mil soa The DNS administrators of navy.mil need to fix four of their seven authoritative name servers to respond properly for usno.navy.mil… Matt From: Outages <outages-bounces(a)outages.org> on behalf of Warren Kumari via Outages <outages(a)outages.org> Reply-To: Warren Kumari <warren(a)kumari.net> Date: Friday, December 21, 2018 at 10:00 AM To: Jared Mauch <jared(a)puck.nether.net> Cc: "outages(a)outages.org" <outages(a)outages.org> Subject: [EXTERNAL] Re: [outages] usno.navy.mil disappeared? Actually, that is what I was going to say... and then I poked again and saw: $ dig ns usno.navy.mil<http://usno.navy.mil> @ns.cybercom.mil<http://ns.cybercom.mil>. ; <<>> DiG 9.11.1-P3 <<>> ns usno.navy.mil<http://usno.navy.mil> @ns.cybercom.mil<http://ns.cybercom.mil>. ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 13693 ;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;usno.navy.mil<http://usno.navy.mil>. IN NS ;; Query time: 52 msec ;; SERVER: 131.77.60.235#53(131.77.60.235) ;; WHEN: Fri Dec 21 09:56:49 EST 2018 ;; MSG SIZE rcvd: 42 navy.mil<http://navy.mil> seems to be returning NXDOMAIN for usno.navy.mil<http://usno.navy.mil> (or, I haven't had sufficient coffee..) W On Fri, Dec 21, 2018 at 9:56 AM Jared Mauch via Outages <outages(a)outages.org<mailto:outages@outages.org>> wrote: Looks like either a DNSSEC problem or their servers are down/unreachable. http://dnsviz.net/d/usno.navy.mil/dnssec/ - Jared > On Dec 21, 2018, at 9:48 AM, Steven Surdock via Outages <outages(a)outages.org<mailto:outages@outages.org>> wrote: > > Anyone know what is going on with The US Naval Observatory? Usno.navy.mil<http://Usno.navy.mil> appears to be completely unresolvable. Affecting access to tick and tock time servers. > > $ nslookup -type=NS usno.navy.mil<http://usno.navy.mil> 8.8.8.8 > *** google-public-dns-a.google.com<http://google-public-dns-a.google.com> can't find usno.navy.mil<http://usno.navy.mil>: Server failed > Server: google-public-dns-a.google.com<http://google-public-dns-a.google.com> > Address: 8.8.8.8 > > -Steve S. > > _______________________________________________ > Outages mailing list > Outages(a)outages.org<mailto:Outages@outages.org> > https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list Outages(a)outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages -- I don't think the execution is relevant when it was obviously a bad idea in the first place. This is like putting rabid weasels in your pants, and later expressing regret at having chosen those particular rabid weasels and that pair of pants. ---maf

1 0

usno.navy.mil disappeared?
by Steven Surdock 21 Dec '18

21 Dec '18

Anyone know what is going on with The US Naval Observatory? Usno.navy.mil appears to be completely unresolvable. Affecting access to tick and tock time servers. $ nslookup -type=NS usno.navy.mil 8.8.8.8 *** google-public-dns-a.google.com can't find usno.navy.mil: Server failed Server: google-public-dns-a.google.com Address: 8.8.8.8 -Steve S.

9 10

Internet outages in TN/KY/VA/GA areas
by Rob Hall 19 Dec '18

19 Dec '18

Hello everyone, Are there any transport or backbone issues that you're aware of in the southeast? We're getting multiple reports from branch locations spread across multiple states that they are having internet outages, but there are multiple carriers involved. Thanks! -- Rob Hall, CCNP R&S, CCP-V, CCA-N I.T. Manager @Work Personnel 1040 Tidewater Court Kingsport, TN 37660 Office: (423)272-7897, x10113 Mobile: (423)312-6003 [cisco-certified-ccnp][CCP-V]

8 7

Level3 Coast Routing Issues?
by Brian Bisbee 15 Dec '18

15 Dec '18

Anyone seeing any routing issues from Level3 on the west coast? I've got users having issues accessing servers on the East coast, looks like Level3 is trying to send them through London first. Thanks, Brian

2 1

Outgoing calls to VZ Phone customers failing in SE region.
by Christopher Smalling 14 Dec '18

14 Dec '18

Hello, We are experiencing an issue where outgoing calls reach VZ customers, but only have an echo of themselves on the line, unable to hear the other. The VZ customer can hear them. Logs on the phone systems show no issues, and nothing that would suggest our equipment is at fault. Does anyone know of an outage ongoing that may be at fault? -- Christopher Smalling Network Operations

5 7