CloudFlare - ipv6 issues ?
Affected sites seem to have been fine ~ 12 hours ago. Noticed page loading issues this AM (I'm in EST). I notice these issues more because my version of squid seems to not fallback to ipv4 if ipv6 is unreachable. bapplegate@phosphor:~$ host en.bitcoin.it en.bitcoin.it is an alias for cf-ssl27811-protected-en.bitcoin.it. cf-ssl27811-protected-en.bitcoin.it has address 141.101.127.218 cf-ssl27811-protected-en.bitcoin.it has address 108.162.200.219 cf-ssl27811-protected-en.bitcoin.it has IPv6 address 2400:cb00:2048:1::8d65:7fda cf-ssl27811-protected-en.bitcoin.it has IPv6 address 2400:cb00:2048:1::6ca2:c8db bapplegate@phosphor:~$ curl -I -4 http://en.bitcoin.it/ HTTP/1.1 302 Found Server: cloudflare-nginx Date: Tue, 11 Jun 2013 12:41:46 GMT Content-Type: text/html Connection: keep-alive X-Powered-By: PHP/5.3.13 Location: /wiki/ Set-Cookie: __cfduid=d53afc7455f2f3b1730d9e695b4aba0921370954505; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.bitcoin.it CF-RAY: 7ce609e639f0436 bapplegate@phosphor:~$ curl -I -6 http://en.bitcoin.it/ curl: (7) couldn't connect to host Other v6 seems to be working fine from my network: bapplegate@phosphor:~$ curl -I -6 http://www.google.com HTTP/1.1 200 OK Date: Tue, 11 Jun 2013 12:44:48 GMT Expires: -1 Cache-Control: private, max-age=0 Content-Type: text/html; charset=ISO-8859-1 Set-Cookie: PREF=ID=962ff74f01950819:FF=0:TM=1370954688:LM=1370954688:S=1kW4WbpuUSXf9sqv; expires=Thu, 11-Jun-2015 12:44:48 GMT; path=/; domain=.google.com Set-Cookie: NID=67=tQ-Lk7-QdXd4jPhTqGw5vSEuwknGV-I-IbEcn8tm_8TM9sZA530i3kCy4rfZVbB-birw-ajlM8LE2A8nA3IBnceC4FfdlSuxuu6T3bDauYFktTamAn4cKnmxmtPbt8vM; expires=Wed, 11-Dec-2013 12:44:48 GMT; path=/; domain=.google.com; HttpOnly P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info." Server: gws X-XSS-Protection: 1; mode=block X-Frame-Options: SAMEORIGIN Transfer-Encoding: chunked -- Brandon Applegate - CCIE 10273 PGP Key fingerprint: 8779 B023 7637 CEC8 C5C6 4052 664D 7E08 3CBB 1739 "SH1-0151. This is the serial number, of our orbital gun."
Works for me: nagios:/home/fbulk# curl -I -6 http://en.bitcoin.it/ HTTP/1.1 302 Found Server: cloudflare-nginx Date: Tue, 11 Jun 2013 13:22:18 GMT Content-Type: text/html Connection: keep-alive X-Powered-By: PHP/5.3.13 Location: /wiki/ Set-Cookie: __cfduid=d5570c9c80966d1ed7ba8772d6c9e76ba1370956938; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.bitcoin.it CF-RAY: 7ce9c01572901ee nagios:/home/fbulk# Frank -----Original Message----- From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Brandon Applegate Sent: Tuesday, June 11, 2013 7:50 AM To: outages@outages.org Subject: [outages] CloudFlare - ipv6 issues ? Affected sites seem to have been fine ~ 12 hours ago. Noticed page loading issues this AM (I'm in EST). I notice these issues more because my version of squid seems to not fallback to ipv4 if ipv6 is unreachable. bapplegate@phosphor:~$ host en.bitcoin.it en.bitcoin.it is an alias for cf-ssl27811-protected-en.bitcoin.it. cf-ssl27811-protected-en.bitcoin.it has address 141.101.127.218 cf-ssl27811-protected-en.bitcoin.it has address 108.162.200.219 cf-ssl27811-protected-en.bitcoin.it has IPv6 address 2400:cb00:2048:1::8d65:7fda cf-ssl27811-protected-en.bitcoin.it has IPv6 address 2400:cb00:2048:1::6ca2:c8db bapplegate@phosphor:~$ curl -I -4 http://en.bitcoin.it/ HTTP/1.1 302 Found Server: cloudflare-nginx Date: Tue, 11 Jun 2013 12:41:46 GMT Content-Type: text/html Connection: keep-alive X-Powered-By: PHP/5.3.13 Location: /wiki/ Set-Cookie: __cfduid=d53afc7455f2f3b1730d9e695b4aba0921370954505; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.bitcoin.it CF-RAY: 7ce609e639f0436 bapplegate@phosphor:~$ curl -I -6 http://en.bitcoin.it/ curl: (7) couldn't connect to host Other v6 seems to be working fine from my network: bapplegate@phosphor:~$ curl -I -6 http://www.google.com HTTP/1.1 200 OK Date: Tue, 11 Jun 2013 12:44:48 GMT Expires: -1 Cache-Control: private, max-age=0 Content-Type: text/html; charset=ISO-8859-1 Set-Cookie: PREF=ID=962ff74f01950819:FF=0:TM=1370954688:LM=1370954688:S=1kW4WbpuUSXf9sqv ; expires=Thu, 11-Jun-2015 12:44:48 GMT; path=/; domain=.google.com Set-Cookie: NID=67=tQ-Lk7-QdXd4jPhTqGw5vSEuwknGV-I-IbEcn8tm_8TM9sZA530i3kCy4rfZVbB-birw- ajlM8LE2A8nA3IBnceC4FfdlSuxuu6T3bDauYFktTamAn4cKnmxmtPbt8vM; expires=Wed, 11-Dec-2013 12:44:48 GMT; path=/; domain=.google.com; HttpOnly P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info." Server: gws X-XSS-Protection: 1; mode=block X-Frame-Options: SAMEORIGIN Transfer-Encoding: chunked -- Brandon Applegate - CCIE 10273 PGP Key fingerprint: 8779 B023 7637 CEC8 C5C6 4052 664D 7E08 3CBB 1739 "SH1-0151. This is the serial number, of our orbital gun." _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Curl works for me too but trying to actually load the page in firefox on ubuntu just gets stuck endlessly loading. Just had a colleague complaing of similar problems with opening another cloudflare hosted site: http://www.mikogo.com/ It kinda feels like an MTU issue though. 1400 byte packets make it: 8 xe-2-3-0.cr1.ams2.nl.nlayer.net (2001:590::4516:8e0e) 201.968 ms 189.039 ms 200.609 ms 9 ae1-20g.ar1.ams3.nl.nlayer.net (2001:590::4516:8b3d) 191.834 ms 191.97 ms 189.011 ms 10 as13335.xe-3-0-2.ar1.ams3.nl.nlayer.net (2001:590::3f8d:df1e) 220.754 ms 220.786 ms 220.778 ms 11 2400:cb00:2048:1::8d65:7fda (2400:cb00:2048:1::8d65:7fda) 212.063 ms 210.701 ms 210.638 ms while 1500 byte packets don't: 8 xe-2-3-0.cr1.ams2.nl.nlayer.net (2001:590::4516:8e0e) 188.975 ms 205.167 ms 190.259 ms 9 ae1-20g.ar1.ams3.nl.nlayer.net (2001:590::4516:8b3d) 200.561 ms 190.452 ms 194.873 ms 10 as13335.xe-3-0-2.ar1.ams3.nl.nlayer.net (2001:590::3f8d:df1e) 232.169 ms 222.212 ms 222.093 ms 11 * * * On 11/06/2013 15:23, Frank Bulk (iname.com) wrote:
Works for me:
nagios:/home/fbulk# curl -I -6 http://en.bitcoin.it/ HTTP/1.1 302 Found Server: cloudflare-nginx Date: Tue, 11 Jun 2013 13:22:18 GMT Content-Type: text/html Connection: keep-alive X-Powered-By: PHP/5.3.13 Location: /wiki/ Set-Cookie: __cfduid=d5570c9c80966d1ed7ba8772d6c9e76ba1370956938; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.bitcoin.it CF-RAY: 7ce9c01572901ee
nagios:/home/fbulk#
Frank
-----Original Message----- From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Brandon Applegate Sent: Tuesday, June 11, 2013 7:50 AM To: outages@outages.org Subject: [outages] CloudFlare - ipv6 issues ?
Affected sites seem to have been fine ~ 12 hours ago. Noticed page loading issues this AM (I'm in EST). I notice these issues more because my version of squid seems to not fallback to ipv4 if ipv6 is unreachable.
bapplegate@phosphor:~$ host en.bitcoin.it en.bitcoin.it is an alias for cf-ssl27811-protected-en.bitcoin.it. cf-ssl27811-protected-en.bitcoin.it has address 141.101.127.218 cf-ssl27811-protected-en.bitcoin.it has address 108.162.200.219 cf-ssl27811-protected-en.bitcoin.it has IPv6 address 2400:cb00:2048:1::8d65:7fda cf-ssl27811-protected-en.bitcoin.it has IPv6 address 2400:cb00:2048:1::6ca2:c8db
bapplegate@phosphor:~$ curl -I -4 http://en.bitcoin.it/ HTTP/1.1 302 Found Server: cloudflare-nginx Date: Tue, 11 Jun 2013 12:41:46 GMT Content-Type: text/html Connection: keep-alive X-Powered-By: PHP/5.3.13 Location: /wiki/ Set-Cookie: __cfduid=d53afc7455f2f3b1730d9e695b4aba0921370954505; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.bitcoin.it CF-RAY: 7ce609e639f0436
bapplegate@phosphor:~$ curl -I -6 http://en.bitcoin.it/ curl: (7) couldn't connect to host
Other v6 seems to be working fine from my network:
bapplegate@phosphor:~$ curl -I -6 http://www.google.com HTTP/1.1 200 OK Date: Tue, 11 Jun 2013 12:44:48 GMT Expires: -1 Cache-Control: private, max-age=0 Content-Type: text/html; charset=ISO-8859-1 Set-Cookie: PREF=ID=962ff74f01950819:FF=0:TM=1370954688:LM=1370954688:S=1kW4WbpuUSXf9sqv ; expires=Thu, 11-Jun-2015 12:44:48 GMT; path=/; domain=.google.com Set-Cookie: NID=67=tQ-Lk7-QdXd4jPhTqGw5vSEuwknGV-I-IbEcn8tm_8TM9sZA530i3kCy4rfZVbB-birw- ajlM8LE2A8nA3IBnceC4FfdlSuxuu6T3bDauYFktTamAn4cKnmxmtPbt8vM; expires=Wed, 11-Dec-2013 12:44:48 GMT; path=/; domain=.google.com; HttpOnly P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info." Server: gws X-XSS-Protection: 1; mode=block X-Frame-Options: SAMEORIGIN Transfer-Encoding: chunked
-- Brandon Applegate - CCIE 10273 PGP Key fingerprint: 8779 B023 7637 CEC8 C5C6 4052 664D 7E08 3CBB 1739 "SH1-0151. This is the serial number, of our orbital gun."
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Graham Beneke Network Engineer | Neology (PTY) Ltd. graham@neology.co.za | http://www.neology.co.za/ Dir: +27-10-500-5906 | Suite 301, Block C, Eva Park Tel: +27-11-476-1933 | Cresta, Johannesburg Skype: grbeneke | Jabber: graham@neology.co.za
On 6/11/13 4:13 PM, Graham Beneke wrote:
Curl works for me too but trying to actually load the page in firefox on ubuntu just gets stuck endlessly loading.
Just had a colleague complaing of similar problems with opening another cloudflare hosted site: http://www.mikogo.com/
It kinda feels like an MTU issue though.
hi, from my pov in europe I see intermittent issues: - via a frankfurt HE tunnel the CF host (through TELIA) is reached with a MTU of 1432 - via a london HE tunnel the CF host (through NLAYER) is reached with a MTU of 1432 - via native connection through INTEROUTE is reached with a MTU of 1432 I can confirm what Graham said: the browser cannot load the page. same issue, lasted 6 hours, reported on 30 may -- antonio
Hmm, there may be MTU issues, but I think I have just routing problems to start. All of my traffic is going to go out Level3 (at least right now). Doing a trace from level3 (Cincinnati - which is where our circuits come from) results in timeouts: Traceroute results from Cincinnati, OH to 2400:cb00:2048:1::8d65:7fda 1 vl-5.bar1.Cincinatti1.Level3.net (2001:1900:23::1) 0 msec 0 msec 0 msec 2 vl-4066.car2.Chicago1.Level3.net (2001:1900:4:1::27D) 8 msec 8 msec 8 msec 3 vl-4080.car1.Chicago1.Level3.net (2001:1900:4:1::15) 40 msec 60 msec 8 msec 4 vl-4040.edge1.Chicago2.Level3.net (2001:1900:4:1::1E) 8 msec 8 msec 4 msec 5 vl-4042.edge6.Denver1.Level3.net (2001:1900:4:1::35) 32 msec 84 msec 28 msec 6 vl-4060.car2.Seattle1.Level3.net (2001:1900:4:1::10A) 92 msec 56 msec 56 msec 7 2001:1900:1B:2::9 60 msec 60 msec 60 msec 8 * * * 9 * * * 10 * * * 11 * * * 12 * * * 13 * * * 14 * * * 15 * * * 16 * * * 17 * * * 18 * * * 19 * * * 20 * * * timeout ! Guess I should open a ticket upstream and maybe have Level3 poke Cloudflare ?
Hmm, access via Cogent appears to be fine for us. [root@bergman ~]# traceroute6 -I 2400:cb00:2048:1::8d65:7fda traceroute to 2400:cb00:2048:1::8d65:7fda (2400:cb00:2048:1::8d65:7fda), 30 hops max, 80 byte packets 1 2620:0:2810:16c::fffd (2620:0:2810:16c::fffd) 0.462 ms 0.544 ms 0.641 ms 2 2620:0:2810:e002::253 (2620:0:2810:e002::253) 0.436 ms 0.562 ms 0.643 ms 3 2620:0:2810:101::fffd (2620:0:2810:101::fffd) 0.394 ms 0.402 ms 0.402 ms 4 2001:550:2:91::1 (2001:550:2:91::1) 0.779 ms 0.778 ms 0.825 ms 5 * * * 6 2001:550::105 (2001:550::105) 81.400 ms 82.737 ms 82.719 ms 7 2001:978::19 (2001:978::19) 75.574 ms 75.596 ms 75.618 ms 8 2001:978:2:46::41 (2001:978:2:46::41) 81.402 ms 81.474 ms 81.391 ms 9 prs-b2-link.telia.net (2001:2000:3080:408::1) 73.976 ms 73.990 ms 73.959 ms 10 prs-b8-v6.telia.net (2001:2000:3018:2a::1) 81.030 ms 80.925 ms 80.837 ms 11 cloudflare-ic-154355-prs-b8.c.telia.net (2001:2000:3080:567::2) 80.217 ms 80.211 ms 80.274 ms 12 2400:cb00:2048:1::8d65:7fda (2400:cb00:2048:1::8d65:7fda) 77.695 ms 77.691 ms 77.723 ms ---- Matthew Huff | 1 Manhattanville Rd Director of Operations | Purchase, NY 10577 OTA Management LLC | Phone: 914-460-4039
-----Original Message----- From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Brandon Applegate Sent: Tuesday, June 11, 2013 1:47 PM To: outages@outages.org Subject: Re: [outages] CloudFlare - ipv6 issues ?
Hmm, there may be MTU issues, but I think I have just routing problems to start. All of my traffic is going to go out Level3 (at least right now). Doing a trace from level3 (Cincinnati - which is where our circuits come from) results in timeouts:
Traceroute results from Cincinnati, OH to 2400:cb00:2048:1::8d65:7fda 1 vl-5.bar1.Cincinatti1.Level3.net (2001:1900:23::1) 0 msec 0 msec 0 msec 2 vl-4066.car2.Chicago1.Level3.net (2001:1900:4:1::27D) 8 msec 8 msec 8 msec 3 vl-4080.car1.Chicago1.Level3.net (2001:1900:4:1::15) 40 msec 60 msec 8 msec 4 vl-4040.edge1.Chicago2.Level3.net (2001:1900:4:1::1E) 8 msec 8 msec 4 msec 5 vl-4042.edge6.Denver1.Level3.net (2001:1900:4:1::35) 32 msec 84 msec 28 msec 6 vl-4060.car2.Seattle1.Level3.net (2001:1900:4:1::10A) 92 msec 56 msec 56 msec 7 2001:1900:1B:2::9 60 msec 60 msec 60 msec 8 * * * 9 * * * 10 * * * 11 * * * 12 * * * 13 * * * 14 * * * 15 * * * 16 * * * 17 * * * 18 * * * 19 * * * 20 * * * timeout !
Guess I should open a ticket upstream and maybe have Level3 poke Cloudflare ? _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
This is fixed now. It was a Level3 specific routing issue - I worked with them and it's resolved. Thanks all.
participants (5)
-
Antonio Prado -
Brandon Applegate -
Frank Bulk (iname.com) -
Graham Beneke -
Matthew Huff