Friday evening, Sunday morning, and Sunday evening, our monitoring system has seen sporadic HTTPv6 access issues to www.netflix.com. Starting since 10:36 am Central this morning, I've seen sporadic ICMPv6 failures to www.netflix.com. Now www.netflix.com has many AAAA records that are likely different around the world and I don't have our monitoring system set up to individually monitor each AAAA record, but wondering if anyone else has seen the same thing? Regards, Frank ================ Current AAAA's, currently pingable: 2406:da00:ff00::b849:f83b 2406:da00:ff00::b849:fcb3 2406:da00:ff00::6b16:ebed ================= Former AAAA's during non-working state: 2620:108:700f::36f4:7696 2620:108:700f::36f4:dd14 2620:108:700f::36f4:e97a 2620:108:700f::3270:5c77 2620:108:700f::3270:5c7b 2620:108:700f::36d6:752 2620:108:700f::36d6:b87 2620:108:700f::36d6:22b2
The issue started happening today again. It sounds like Netflix is having some challenges with advertising AAAA's that are operating. When www.netflix response I get three AAAA's that on AWS East. When it's not working, I get eight AAAA's that are on AWS West. Here's the three AAAA's when it's good and the eight AAAA's when it's not good. nagios:/tmp# ./netflix.sh ; <<>> DiG 9.7.3 <<>> AAAA www.netflix.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26 ;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 8, ADDITIONAL: 8 ;; QUESTION SECTION: ;www.netflix.com. IN AAAA ;; ANSWER SECTION: www.netflix.com. 290 IN CNAME dualstack.wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. dualstack.wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 50 IN AAAA 2406:da00:ff00::b849:f83b dualstack.wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 50 IN AAAA 2406:da00:ff00::b849:fcb3 dualstack.wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 50 IN AAAA 2406:da00:ff00::6b16:ebed ;; AUTHORITY SECTION: wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-947.amazonaws.com. wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-927.amazonaws.com. wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-941.amazonaws.com. wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-925.amazonaws.com. wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-935.amazonaws.com. wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-934.amazonaws.com. wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-944.amazonaws.com. wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-916.amazonaws.com. ;; ADDITIONAL SECTION: ns-916.amazonaws.com. 350 IN A 207.171.178.7 ns-925.amazonaws.com. 590 IN A 72.21.208.213 ns-927.amazonaws.com. 590 IN A 72.21.204.209 ns-934.amazonaws.com. 590 IN A 87.238.84.215 ns-935.amazonaws.com. 590 IN A 87.238.80.213 ns-941.amazonaws.com. 7532 IN A 204.246.160.5 ns-944.amazonaws.com. 1772 IN A 205.251.228.6 ns-947.amazonaws.com. 590 IN A 204.246.162.10 ;; Query time: 0 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Tue Oct 1 17:58:05 2013 ;; MSG SIZE rcvd: 496 PING 2406:da00:ff00::6b16:ebed(2406:da00:ff00::6b16:ebed) 56 data bytes 64 bytes from 2406:da00:ff00::6b16:ebed: icmp_seq=1 ttl=48 time=57.7 ms 64 bytes from 2406:da00:ff00::6b16:ebed: icmp_seq=2 ttl=48 time=58.9 ms 64 bytes from 2406:da00:ff00::6b16:ebed: icmp_seq=3 ttl=48 time=57.8 ms --- 2406:da00:ff00::6b16:ebed ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 57.709/58.149/58.938/0.624 ms PING 2406:da00:ff00::b849:f83b(2406:da00:ff00::b849:f83b) 56 data bytes 64 bytes from 2406:da00:ff00::b849:f83b: icmp_seq=1 ttl=49 time=58.6 ms 64 bytes from 2406:da00:ff00::b849:f83b: icmp_seq=2 ttl=49 time=58.4 ms 64 bytes from 2406:da00:ff00::b849:f83b: icmp_seq=3 ttl=49 time=58.4 ms --- 2406:da00:ff00::b849:f83b ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 58.468/58.537/58.646/0.212 ms PING 2406:da00:ff00::b849:fcb3(2406:da00:ff00::b849:fcb3) 56 data bytes 64 bytes from 2406:da00:ff00::b849:fcb3: icmp_seq=1 ttl=49 time=74.8 ms 64 bytes from 2406:da00:ff00::b849:fcb3: icmp_seq=2 ttl=49 time=58.7 ms 64 bytes from 2406:da00:ff00::b849:fcb3: icmp_seq=3 ttl=49 time=65.3 ms --- 2406:da00:ff00::b849:fcb3 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 58.732/66.318/74.827/6.606 ms nagios:/tmp# ./netflix.sh ==================================================== ; <<>> DiG 9.7.3 <<>> AAAA www.netflix.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38593 ;; flags: qr rd ra; QUERY: 1, ANSWER: 9, AUTHORITY: 8, ADDITIONAL: 0 ;; QUESTION SECTION: ;www.netflix.com. IN AAAA ;; ANSWER SECTION: www.netflix.com. 124 IN CNAME dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::3270:5c77 dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::3270:5c7b dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::36d6:752 dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::36d6:20b0 dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::36d6:22b2 dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::36d6:24e1 dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::36f4:7696 dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::36f4:e97a ;; AUTHORITY SECTION: us-west-2.elb.amazonaws.com. 124 IN NS ns-925.amazonaws.com. us-west-2.elb.amazonaws.com. 124 IN NS ns-947.amazonaws.com. us-west-2.elb.amazonaws.com. 124 IN NS ns-916.amazonaws.com. us-west-2.elb.amazonaws.com. 124 IN NS ns-934.amazonaws.com. us-west-2.elb.amazonaws.com. 124 IN NS ns-941.amazonaws.com. us-west-2.elb.amazonaws.com. 124 IN NS ns-935.amazonaws.com. us-west-2.elb.amazonaws.com. 124 IN NS ns-944.amazonaws.com. us-west-2.elb.amazonaws.com. 124 IN NS ns-927.amazonaws.com. ;; Query time: 0 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Tue Oct 1 18:10:51 2013 ;; MSG SIZE rcvd: 508 PING 2620:108:700f::36f4:e97a(2620:108:700f::36f4:e97a) 56 data bytes --- 2620:108:700f::36f4:e97a ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2000ms PING 2620:108:700f::3270:5c77(2620:108:700f::3270:5c77) 56 data bytes --- 2620:108:700f::3270:5c77 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2008ms PING 2620:108:700f::3270:5c7b(2620:108:700f::3270:5c7b) 56 data bytes --- 2620:108:700f::3270:5c7b ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms PING 2620:108:700f::36d6:752(2620:108:700f::36d6:752) 56 data bytes --- 2620:108:700f::36d6:752 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2016ms PING 2620:108:700f::36d6:20b0(2620:108:700f::36d6:20b0) 56 data bytes --- 2620:108:700f::36d6:20b0 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2014ms PING 2620:108:700f::36d6:22b2(2620:108:700f::36d6:22b2) 56 data bytes --- 2620:108:700f::36d6:22b2 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2013ms PING 2620:108:700f::36d6:24e1(2620:108:700f::36d6:24e1) 56 data bytes --- 2620:108:700f::36d6:24e1 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2009ms PING 2620:108:700f::36f4:7696(2620:108:700f::36f4:7696) 56 data bytes --- 2620:108:700f::36f4:7696 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2013ms nagios:/tmp# -----Original Message----- From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Frank Bulk Sent: Monday, September 30, 2013 12:26 PM To: outages@outages.org Subject: [outages] Netflix over IPv6 Friday evening, Sunday morning, and Sunday evening, our monitoring system has seen sporadic HTTPv6 access issues to www.netflix.com. Starting since 10:36 am Central this morning, I've seen sporadic ICMPv6 failures to www.netflix.com. Now www.netflix.com has many AAAA records that are likely different around the world and I don't have our monitoring system set up to individually monitor each AAAA record, but wondering if anyone else has seen the same thing? Regards, Frank ================ Current AAAA's, currently pingable: 2406:da00:ff00::b849:f83b 2406:da00:ff00::b849:fcb3 2406:da00:ff00::6b16:ebed ================= Former AAAA's during non-working state: 2620:108:700f::36f4:7696 2620:108:700f::36f4:dd14 2620:108:700f::36f4:e97a 2620:108:700f::3270:5c77 2620:108:700f::3270:5c7b 2620:108:700f::36d6:752 2620:108:700f::36d6:b87 2620:108:700f::36d6:22b2 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Confirmed from my POV in Texas, as well as v6 servers in NY, FL, and IL. As a consumer, I've had to straight up disable IPv6 when I want to watch Netflix. On Tue, Oct 1, 2013 at 6:17 PM, Frank Bulk <frnkblk@iname.com> wrote:
The issue started happening today again. It sounds like Netflix is having some challenges with advertising AAAA's that are operating. When www.netflix response I get three AAAA's that on AWS East. When it's not working, I get eight AAAA's that are on AWS West.
Here's the three AAAA's when it's good and the eight AAAA's when it's not good.
nagios:/tmp# ./netflix.sh
; <<>> DiG 9.7.3 <<>> AAAA www.netflix.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26 ;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 8, ADDITIONAL: 8
;; QUESTION SECTION: ;www.netflix.com. IN AAAA
;; ANSWER SECTION: www.netflix.com. 290 IN CNAME dualstack.wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. dualstack.wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 50 IN AAAA 2406:da00:ff00::b849:f83b dualstack.wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 50 IN AAAA 2406:da00:ff00::b849:fcb3 dualstack.wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 50 IN AAAA 2406:da00:ff00::6b16:ebed
;; AUTHORITY SECTION: wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-947.amazonaws.com. wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-927.amazonaws.com. wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-941.amazonaws.com. wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-925.amazonaws.com. wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-935.amazonaws.com. wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-934.amazonaws.com. wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-944.amazonaws.com. wwwservice--frontend-san-756424683.us-east-1.elb.amazonaws.com. 1490 IN NS ns-916.amazonaws.com.
;; ADDITIONAL SECTION: ns-916.amazonaws.com. 350 IN A 207.171.178.7 ns-925.amazonaws.com. 590 IN A 72.21.208.213 ns-927.amazonaws.com. 590 IN A 72.21.204.209 ns-934.amazonaws.com. 590 IN A 87.238.84.215 ns-935.amazonaws.com. 590 IN A 87.238.80.213 ns-941.amazonaws.com. 7532 IN A 204.246.160.5 ns-944.amazonaws.com. 1772 IN A 205.251.228.6 ns-947.amazonaws.com. 590 IN A 204.246.162.10
;; Query time: 0 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Tue Oct 1 17:58:05 2013 ;; MSG SIZE rcvd: 496
PING 2406:da00:ff00::6b16:ebed(2406:da00:ff00::6b16:ebed) 56 data bytes 64 bytes from 2406:da00:ff00::6b16:ebed: icmp_seq=1 ttl=48 time=57.7 ms 64 bytes from 2406:da00:ff00::6b16:ebed: icmp_seq=2 ttl=48 time=58.9 ms 64 bytes from 2406:da00:ff00::6b16:ebed: icmp_seq=3 ttl=48 time=57.8 ms
--- 2406:da00:ff00::6b16:ebed ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 57.709/58.149/58.938/0.624 ms PING 2406:da00:ff00::b849:f83b(2406:da00:ff00::b849:f83b) 56 data bytes 64 bytes from 2406:da00:ff00::b849:f83b: icmp_seq=1 ttl=49 time=58.6 ms 64 bytes from 2406:da00:ff00::b849:f83b: icmp_seq=2 ttl=49 time=58.4 ms 64 bytes from 2406:da00:ff00::b849:f83b: icmp_seq=3 ttl=49 time=58.4 ms
--- 2406:da00:ff00::b849:f83b ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 58.468/58.537/58.646/0.212 ms PING 2406:da00:ff00::b849:fcb3(2406:da00:ff00::b849:fcb3) 56 data bytes 64 bytes from 2406:da00:ff00::b849:fcb3: icmp_seq=1 ttl=49 time=74.8 ms 64 bytes from 2406:da00:ff00::b849:fcb3: icmp_seq=2 ttl=49 time=58.7 ms 64 bytes from 2406:da00:ff00::b849:fcb3: icmp_seq=3 ttl=49 time=65.3 ms
--- 2406:da00:ff00::b849:fcb3 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 58.732/66.318/74.827/6.606 ms nagios:/tmp# ./netflix.sh
====================================================
; <<>> DiG 9.7.3 <<>> AAAA www.netflix.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38593 ;; flags: qr rd ra; QUERY: 1, ANSWER: 9, AUTHORITY: 8, ADDITIONAL: 0
;; QUESTION SECTION: ;www.netflix.com. IN AAAA
;; ANSWER SECTION: www.netflix.com. 124 IN CNAME dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::3270:5c77 dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::3270:5c7b dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::36d6:752 dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::36d6:20b0 dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::36d6:22b2 dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::36d6:24e1 dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::36f4:7696 dualstack.wwwservice--frontend-san-608070354.us-west-2.elb.amazonaws.com. 4 IN AAAA 2620:108:700f::36f4:e97a
;; AUTHORITY SECTION: us-west-2.elb.amazonaws.com. 124 IN NS ns-925.amazonaws.com. us-west-2.elb.amazonaws.com. 124 IN NS ns-947.amazonaws.com. us-west-2.elb.amazonaws.com. 124 IN NS ns-916.amazonaws.com. us-west-2.elb.amazonaws.com. 124 IN NS ns-934.amazonaws.com. us-west-2.elb.amazonaws.com. 124 IN NS ns-941.amazonaws.com. us-west-2.elb.amazonaws.com. 124 IN NS ns-935.amazonaws.com. us-west-2.elb.amazonaws.com. 124 IN NS ns-944.amazonaws.com. us-west-2.elb.amazonaws.com. 124 IN NS ns-927.amazonaws.com.
;; Query time: 0 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Tue Oct 1 18:10:51 2013 ;; MSG SIZE rcvd: 508
PING 2620:108:700f::36f4:e97a(2620:108:700f::36f4:e97a) 56 data bytes
--- 2620:108:700f::36f4:e97a ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2000ms
PING 2620:108:700f::3270:5c77(2620:108:700f::3270:5c77) 56 data bytes
--- 2620:108:700f::3270:5c77 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2008ms
PING 2620:108:700f::3270:5c7b(2620:108:700f::3270:5c7b) 56 data bytes
--- 2620:108:700f::3270:5c7b ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms
PING 2620:108:700f::36d6:752(2620:108:700f::36d6:752) 56 data bytes
--- 2620:108:700f::36d6:752 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2016ms
PING 2620:108:700f::36d6:20b0(2620:108:700f::36d6:20b0) 56 data bytes
--- 2620:108:700f::36d6:20b0 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2014ms
PING 2620:108:700f::36d6:22b2(2620:108:700f::36d6:22b2) 56 data bytes
--- 2620:108:700f::36d6:22b2 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2013ms
PING 2620:108:700f::36d6:24e1(2620:108:700f::36d6:24e1) 56 data bytes
--- 2620:108:700f::36d6:24e1 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2009ms
PING 2620:108:700f::36f4:7696(2620:108:700f::36f4:7696) 56 data bytes
--- 2620:108:700f::36f4:7696 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2013ms
nagios:/tmp#
-----Original Message----- From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Frank Bulk Sent: Monday, September 30, 2013 12:26 PM To: outages@outages.org Subject: [outages] Netflix over IPv6
Friday evening, Sunday morning, and Sunday evening, our monitoring system has seen sporadic HTTPv6 access issues to www.netflix.com. Starting since 10:36 am Central this morning, I've seen sporadic ICMPv6 failures to www.netflix.com.
Now www.netflix.com has many AAAA records that are likely different around the world and I don't have our monitoring system set up to individually monitor each AAAA record, but wondering if anyone else has seen the same thing?
Regards,
Frank
================ Current AAAA's, currently pingable:
2406:da00:ff00::b849:f83b 2406:da00:ff00::b849:fcb3 2406:da00:ff00::6b16:ebed
================= Former AAAA's during non-working state:
2620:108:700f::36f4:7696 2620:108:700f::36f4:dd14 2620:108:700f::36f4:e97a 2620:108:700f::3270:5c77 2620:108:700f::3270:5c7b 2620:108:700f::36d6:752 2620:108:700f::36d6:b87 2620:108:700f::36d6:22b2
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
participants (2)
-
Blair Trosper -
Frank Bulk