Hi, Comcast DNS servers (75.75.75.75, 75.75.76.76) seem to be down. Web pages were suck at dns resolution phase, switching to Google DNS solved the problem immediately. Tested from San Francisco. sajal@sajal-lappy:~$ mtr --report 75.75.75.75 Start: Mon Jun 1 20:03:02 2015 HOST: sajal-lappy Loss% Snt Last Avg Best Wrst StDev 1.|-- 10.0.1.1 0.0% 10 1.2 3.1 1.0 5.9 1.2 2.|-- 50.174.142.1 0.0% 10 15.9 20.7 14.6 49.7 10.4 3.|-- te-0-7-0-5-sur03.sffolsom 0.0% 10 16.9 24.1 16.4 54.6 11.6 4.|-- be-209-ar01.santaclara.ca 0.0% 10 18.2 33.4 17.0 146.1 39.9 5.|-- he-0-14-0-0-ar01.sfsutro. 0.0% 10 22.4 26.9 20.0 42.9 6.8 6.|-- te-9-4-ur01-d.sanjose.ca. 0.0% 10 22.9 23.1 20.6 28.4 2.0 7.|-- te-8-2-ur01-d.sanjose.ca. 0.0% 10 21.5 23.4 20.6 29.2 3.1 8.|-- cdns01.comcast.net 0.0% 10 32.2 47.3 20.5 89.7 29.0 sajal@sajal-lappy:~$ dig google.com @75.75.75.75 ; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>> google.com @75.75.75.75 ;; global options: +cmd ;; connection timed out; no servers could be reached sajal@sajal-lappy:~$ https://pulse.turbobytes.com/results/556d1b32ecbe4023eb0000ab/ - Using our tool we see timeout from Seattle and Mountain View. Is this a national level outage or something? -Sajal
Its working fine here in Atlanta, but 75.75.7[56].7[56] are anycasted so the cluster you are hitting from SFO is almost certainly not the same cluster I am hitting from ATL. --------------------- Aubrey Wells Manager, Network Operations Digium Cloud Services Main: 888.305.3850 Support: 877.344.4861 or http://www.digium.com/en/support <http://www.digium.com/en/support?elq=65516445a5964d3597e25eaf566bc2cf&elqCampaignId=> On Mon, Jun 1, 2015 at 11:06 PM, Sajal Kayan via Outages < outages@outages.org> wrote:
Hi,
Comcast DNS servers (75.75.75.75, 75.75.76.76) seem to be down. Web pages were suck at dns resolution phase, switching to Google DNS solved the problem immediately.
Tested from San Francisco.
sajal@sajal-lappy:~$ mtr --report 75.75.75.75 Start: Mon Jun 1 20:03:02 2015 HOST: sajal-lappy Loss% Snt Last Avg Best Wrst StDev 1.|-- 10.0.1.1 0.0% 10 1.2 3.1 1.0 5.9 1.2 2.|-- 50.174.142.1 0.0% 10 15.9 20.7 14.6 49.7 10.4 3.|-- te-0-7-0-5-sur03.sffolsom 0.0% 10 16.9 24.1 16.4 54.6 11.6 4.|-- be-209-ar01.santaclara.ca 0.0% 10 18.2 33.4 17.0 146.1 39.9 5.|-- he-0-14-0-0-ar01.sfsutro. 0.0% 10 22.4 26.9 20.0 42.9 6.8 6.|-- te-9-4-ur01-d.sanjose.ca. 0.0% 10 22.9 23.1 20.6 28.4 2.0 7.|-- te-8-2-ur01-d.sanjose.ca. 0.0% 10 21.5 23.4 20.6 29.2 3.1 8.|-- cdns01.comcast.net 0.0% 10 32.2 47.3 20.5 89.7 29.0 sajal@sajal-lappy:~$ dig google.com @75.75.75.75
; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>> google.com @75.75.75.75 ;; global options: +cmd ;; connection timed out; no servers could be reached sajal@sajal-lappy:~$
https://pulse.turbobytes.com/results/556d1b32ecbe4023eb0000ab/ - Using our tool we see timeout from Seattle and Mountain View. Is this a national level outage or something?
-Sajal
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
It's localized to the west coast. We're investigating now. Thanks, John On Mon, Jun 1, 2015 at 9:10 PM, Aubrey Wells via Outages < outages@outages.org> wrote:
Its working fine here in Atlanta, but 75.75.7[56].7[56] are anycasted so the cluster you are hitting from SFO is almost certainly not the same cluster I am hitting from ATL.
--------------------- Aubrey Wells Manager, Network Operations Digium Cloud Services Main: 888.305.3850 Support: 877.344.4861 or http://www.digium.com/en/support <http://www.digium.com/en/support?elq=65516445a5964d3597e25eaf566bc2cf&elqCampaignId=>
On Mon, Jun 1, 2015 at 11:06 PM, Sajal Kayan via Outages < outages@outages.org> wrote:
Hi,
Comcast DNS servers (75.75.75.75, 75.75.76.76) seem to be down. Web pages were suck at dns resolution phase, switching to Google DNS solved the problem immediately.
Tested from San Francisco.
sajal@sajal-lappy:~$ mtr --report 75.75.75.75 Start: Mon Jun 1 20:03:02 2015 HOST: sajal-lappy Loss% Snt Last Avg Best Wrst StDev 1.|-- 10.0.1.1 0.0% 10 1.2 3.1 1.0 5.9 1.2 2.|-- 50.174.142.1 0.0% 10 15.9 20.7 14.6 49.7 10.4 3.|-- te-0-7-0-5-sur03.sffolsom 0.0% 10 16.9 24.1 16.4 54.6 11.6 4.|-- be-209-ar01.santaclara.ca 0.0% 10 18.2 33.4 17.0 146.1 39.9 5.|-- he-0-14-0-0-ar01.sfsutro. 0.0% 10 22.4 26.9 20.0 42.9 6.8 6.|-- te-9-4-ur01-d.sanjose.ca. 0.0% 10 22.9 23.1 20.6 28.4 2.0 7.|-- te-8-2-ur01-d.sanjose.ca. 0.0% 10 21.5 23.4 20.6 29.2 3.1 8.|-- cdns01.comcast.net 0.0% 10 32.2 47.3 20.5 89.7 29.0 sajal@sajal-lappy:~$ dig google.com @75.75.75.75
; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>> google.com @75.75.75.75 ;; global options: +cmd ;; connection timed out; no servers could be reached sajal@sajal-lappy:~$
https://pulse.turbobytes.com/results/556d1b32ecbe4023eb0000ab/ - Using our tool we see timeout from Seattle and Mountain View. Is this a national level outage or something?
-Sajal
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
I don’t see anything on dns.comcast.net. I will forward to my Comcast contacts. Frank From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Sajal Kayan via Outages Sent: Monday, June 01, 2015 10:06 PM To: OUTAGES Mailing List Subject: [outages] Comcast DNS broken? Hi, Comcast DNS servers (75.75.75.75, 75.75.76.76) seem to be down. Web pages were suck at dns resolution phase, switching to Google DNS solved the problem immediately. Tested from San Francisco. sajal@sajal-lappy:~$ mtr --report 75.75.75.75 Start: Mon Jun 1 20:03:02 2015 HOST: sajal-lappy Loss% Snt Last Avg Best Wrst StDev 1.|-- 10.0.1.1 0.0% 10 1.2 3.1 1.0 5.9 1.2 2.|-- 50.174.142.1 0.0% 10 15.9 20.7 14.6 49.7 10.4 3.|-- te-0-7-0-5-sur03.sffolsom 0.0% 10 16.9 24.1 16.4 54.6 11.6 4.|-- be-209-ar01.santaclara.ca <http://be-209-ar01.santaclara.ca> 0.0% 10 18.2 33.4 17.0 146.1 39.9 5.|-- he-0-14-0-0-ar01.sfsutro. 0.0% 10 22.4 26.9 20.0 42.9 6.8 6.|-- te-9-4-ur01-d.sanjose.ca. 0.0% 10 22.9 23.1 20.6 28.4 2.0 7.|-- te-8-2-ur01-d.sanjose.ca <http://te-8-2-ur01-d.sanjose.ca> . 0.0% 10 21.5 23.4 20.6 29.2 3.1 8.|-- cdns01.comcast.net <http://cdns01.comcast.net> 0.0% 10 32.2 47.3 20.5 89.7 29.0 sajal@sajal-lappy:~$ dig google.com <http://google.com> @75.75.75.75 <http://75.75.75.75> ; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>> google.com <http://google.com> @75.75.75.75 <http://75.75.75.75> ;; global options: +cmd ;; connection timed out; no servers could be reached sajal@sajal-lappy:~$ https://pulse.turbobytes.com/results/556d1b32ecbe4023eb0000ab/ - Using our tool we see timeout from Seattle and Mountain View. Is this a national level outage or something? -Sajal
Seems to work now.... only occasionally dropping queries... On Mon, Jun 1, 2015 at 8:12 PM, Frank Bulk <frnkblk@iname.com> wrote:
I don’t see anything on dns.comcast.net.
I will forward to my Comcast contacts.
Frank
*From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Sajal Kayan via Outages *Sent:* Monday, June 01, 2015 10:06 PM *To:* OUTAGES Mailing List *Subject:* [outages] Comcast DNS broken?
Hi,
Comcast DNS servers (75.75.75.75, 75.75.76.76) seem to be down. Web pages were suck at dns resolution phase, switching to Google DNS solved the problem immediately.
Tested from San Francisco.
sajal@sajal-lappy:~$ mtr --report 75.75.75.75
Start: Mon Jun 1 20:03:02 2015
HOST: sajal-lappy Loss% Snt Last Avg Best Wrst StDev
1.|-- 10.0.1.1 0.0% 10 1.2 3.1 1.0 5.9 1.2
2.|-- 50.174.142.1 0.0% 10 15.9 20.7 14.6 49.7 10.4
3.|-- te-0-7-0-5-sur03.sffolsom 0.0% 10 16.9 24.1 16.4 54.6 11.6
4.|-- be-209-ar01.santaclara.ca 0.0% 10 18.2 33.4 17.0 146.1 39.9
5.|-- he-0-14-0-0-ar01.sfsutro. 0.0% 10 22.4 26.9 20.0 42.9 6.8
6.|-- te-9-4-ur01-d.sanjose.ca. 0.0% 10 22.9 23.1 20.6 28.4 2.0
7.|-- te-8-2-ur01-d.sanjose.ca. 0.0% 10 21.5 23.4 20.6 29.2 3.1
8.|-- cdns01.comcast.net 0.0% 10 32.2 47.3 20.5 89.7 29.0
sajal@sajal-lappy:~$ dig google.com @75.75.75.75
; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>> google.com @75.75.75.75
;; global options: +cmd
;; connection timed out; no servers could be reached
sajal@sajal-lappy:~$
https://pulse.turbobytes.com/results/556d1b32ecbe4023eb0000ab/ - Using our tool we see timeout from Seattle and Mountain View. Is this a national level outage or something?
-Sajal
http://arstechnica.com/business/2015/06/comcast-issuing-5-credits-after-inte... What happened A piece of hardware in our backbone network failed yesterday morning. Hardware issues of this sort are fairly commonplace and redundancy in our network addressed any immediate concern. Most backbone networks, including Comcast’s, are designed to heal themselves and route traffic along alternate paths much like a detour would route automobile traffic around a street closed for construction. One type of traffic that gets automatically rerouted is Domain Name System (DNS) traffic. Unfortunately, some of that traffic shifted in an unexpected way and overloaded local DNS server capacity causing many customers to experience service interruptions. What we are doing about it Since the incident, we have been working non-stop to fully understand the problem and bring additional DNS capacity online in the affected areas, so we can prevent it from happening again. On Tue, Jun 2, 2015 at 11:00 AM Sajal Kayan <sajal83@gmail.com> wrote:
Seems to work now.... only occasionally dropping queries...
On Mon, Jun 1, 2015 at 8:12 PM, Frank Bulk <frnkblk@iname.com> wrote:
I don’t see anything on dns.comcast.net.
I will forward to my Comcast contacts.
Frank
*From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Sajal Kayan via Outages *Sent:* Monday, June 01, 2015 10:06 PM *To:* OUTAGES Mailing List *Subject:* [outages] Comcast DNS broken?
Hi,
Comcast DNS servers (75.75.75.75, 75.75.76.76) seem to be down. Web pages were suck at dns resolution phase, switching to Google DNS solved the problem immediately.
Tested from San Francisco.
sajal@sajal-lappy:~$ mtr --report 75.75.75.75
Start: Mon Jun 1 20:03:02 2015
HOST: sajal-lappy Loss% Snt Last Avg Best Wrst StDev
1.|-- 10.0.1.1 0.0% 10 1.2 3.1 1.0 5.9 1.2
2.|-- 50.174.142.1 0.0% 10 15.9 20.7 14.6 49.7 10.4
3.|-- te-0-7-0-5-sur03.sffolsom 0.0% 10 16.9 24.1 16.4 54.6 11.6
4.|-- be-209-ar01.santaclara.ca 0.0% 10 18.2 33.4 17.0 146.1 39.9
5.|-- he-0-14-0-0-ar01.sfsutro. 0.0% 10 22.4 26.9 20.0 42.9 6.8
6.|-- te-9-4-ur01-d.sanjose.ca. 0.0% 10 22.9 23.1 20.6 28.4 2.0
7.|-- te-8-2-ur01-d.sanjose.ca. 0.0% 10 21.5 23.4 20.6 29.2 3.1
8.|-- cdns01.comcast.net 0.0% 10 32.2 47.3 20.5 89.7 29.0
sajal@sajal-lappy:~$ dig google.com @75.75.75.75
; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>> google.com @75.75.75.75
;; global options: +cmd
;; connection timed out; no servers could be reached
sajal@sajal-lappy:~$
https://pulse.turbobytes.com/results/556d1b32ecbe4023eb0000ab/ - Using our tool we see timeout from Seattle and Mountain View. Is this a national level outage or something?
-Sajal
participants (4)
-
Aubrey Wells -
Frank Bulk -
John Neiberger -
Sajal Kayan