Its been around 30 minutes. Is this the same result from less than 20 hours ago?
Yes, it is down. Let's get back to work! http://downforeveryoneorjustme.com/facebook.com Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 On Thu, Sep 23, 2010 at 3:21 PM, jon <relicwr@gmail.com> wrote:
Its been around 30 minutes. Is this the same result from less than 20 hours ago?
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
On 9/23/2010 12:28, Josh Luthman wrote:
Yes, it is down. Let's get back to work!
I didn't even notice since I *was* working. ;) ~Seth
---- Original Message ----- From: "Seth Mattinen" <sethm@rollernet.us> To: outages@outages.org Sent: Thursday, September 23, 2010 3:29:41 PM Subject: Re: [outages] Facebook
On 9/23/2010 12:28, Josh Luthman wrote:
Yes, it is down. Let's get back to work! http://downforeveryoneorjustme.com/facebook.com
I didn't even notice since I *was* working. ;)
Just to provide the customary reminder: for some of us, "taking calls about why customers can't get to Facebook" *is* working. Cheers, -- jr 'but being snarky *is* fun, ain't it?' a -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com '87 e24 St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274 Start a man a fire, and he'll be warm all night. Set a man on fire, and he'll be warm for the rest of his life.
Maybe Facebook uses XO for connectivity, and thats why they are down! Just had a through an XO dig in there.... -John On Sep 23, 2010, at 3:40 PM, Jay R. Ashworth wrote:
---- Original Message ----- From: "Seth Mattinen" <sethm@rollernet.us> To: outages@outages.org Sent: Thursday, September 23, 2010 3:29:41 PM Subject: Re: [outages] Facebook
On 9/23/2010 12:28, Josh Luthman wrote:
Yes, it is down. Let's get back to work! http://downforeveryoneorjustme.com/facebook.com
I didn't even notice since I *was* working. ;)
Just to provide the customary reminder: for some of us, "taking calls about why customers can't get to Facebook" *is* working.
Cheers, -- jr 'but being snarky *is* fun, ain't it?' a
-- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http:// baylink.pitas.com '87 e24 St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274
Start a man a fire, and he'll be warm all night. Set a man on fire, and he'll be warm for the rest of his life. _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
On 9/23/2010 12:40, Jay R. Ashworth wrote:
---- Original Message ----- From: "Seth Mattinen" <sethm@rollernet.us> To: outages@outages.org Sent: Thursday, September 23, 2010 3:29:41 PM Subject: Re: [outages] Facebook
On 9/23/2010 12:28, Josh Luthman wrote:
Yes, it is down. Let's get back to work! http://downforeveryoneorjustme.com/facebook.com
I didn't even notice since I *was* working. ;)
Just to provide the customary reminder: for some of us, "taking calls about why customers can't get to Facebook" *is* working.
It used to be a long time ago, but after a few years I wanted to stab people over the phone. I paid my dues on the front line. This was long before the social networking era so I can only imagine how much worse it could be now. The most awesome call I ever witnessed was someone arguing to one of my coworkers over speakerphone for 45 minutes that the O'Reilly Factor website had to have the ' in the URL because "it is in his name". The customer absolutely refused to remove the apostrophe, claimed we were censoring, etc. At one point he hit mute, picked up the monitor, pointed it at the phone, and yelled "look! the site works fine!" FWIW I can't reach facebook either as of right now (13:10 PDT) either at work or on my Android via IPv4. However, I *can* access it just fine via IPv6 at www.v6.facebook.com ~Seth
On Thu, 23 Sep 2010, Josh Luthman wrote:
Yes, it is down. Let's get back to work!
Funny, facebook being down == cant work for some of us. ;) -- david raistrick http://www.netmeister.org/news/learn2quote.html drais@icantclick.org http://www.expita.com/nomime.html
The length of the outage could actually affect Sept productivity. Scary how many need their Farmville. On Sep 23, 2010 4:09 PM, "david raistrick" <drais@icantclick.org> wrote:
On Thu, 23 Sep 2010, Josh Luthman wrote:
Yes, it is down. Let's get back to work!
Funny, facebook being down == cant work for some of us. ;)
-- david raistrick http://www.netmeister.org/news/learn2quote.html drais@icantclick.org http://www.expita.com/nomime.html
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Its been around 30 minutes. Is this the same result from less than 20 hours ago?
Seems that is the case. Via Facebook:
API Latency Issues Today, September 23 at 3:07pm We are currently experiencing latency issues with the API, and we are actively investigating. We will provide an update when either the issue is resolved or we have an ETA for resolution.
http://developers.facebook.com/live_status/ -Matt Dodd
---- Original Message ----- From: "jon" <relicwr@gmail.com>
Its been around 30 minutes. Is this the same result from less than 20 hours ago?
Based on Pete Cashmore's comments at Mashable, I think not, but we'll see. It's just come back up and is slowly rolling out, both vertically and horizontally. Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com '87 e24 St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274 Start a man a fire, and he'll be warm all night. Set a man on fire, and he'll be warm for the rest of his life.
Once upon a time, jon <relicwr@gmail.com> said:
Its been around 30 minutes. Is this the same result from less than 20 hours ago?
Interesting, www.facebook.com switched to an Akamai IP for me at about 19:30 UTC. -- Chris Adams <cmadams@hiwaay.net> Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble.
Seeing different things from different places, hasn't fully propagated yet. dig www.facebook.com @8.8.8.8 soa |grep -P '[0-9]{10}' g.akamai.net. 471 IN SOA n0g.akamai.net. hostmaster.akamai.com. 1285274188 1000 1000 1000 1800 dig www.facebook.com @4.2.2.1 soa |grep -P '[0-9]{10}' x.akamai.net. 900 IN SOA n0x.akamai.net. hostmaster.akamai.com. 1285274726 1000 1000 1000 1800 So 8.8.8.8 is definitely the older version, and it shows their IP: dig www.facebook.com @8.8.8.8 +short 69.63.189.11 And 4.2.2.1 is fresher and appears to show a switch to akamai: dig www.facebook.com @4.2.2.1 +short wwws.ak.facebook.com.edgesuite.net. a1254.x.akamai.net. 66.160.206.18 65.49.92.113 Somebody dun goofed. -wil On Sep 23, 2010, at 1:18 PM, Chris Adams wrote:
Once upon a time, jon <relicwr@gmail.com> said:
Its been around 30 minutes. Is this the same result from less than 20 hours ago?
Interesting, www.facebook.com switched to an Akamai IP for me at about 19:30 UTC. -- Chris Adams <cmadams@hiwaay.net> Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble. _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
On 9/23/2010 15:18, Chris Adams wrote:
Once upon a time, jon <relicwr@gmail.com> said:
Its been around 30 minutes. Is this the same result from less than 20 hours ago?
Interesting, www.facebook.com switched to an Akamai IP for me at about 19:30 UTC.
I thought FB always ran throuhg Akamai. I'm now getting quote Service Unavailable - DNS failure The server is temporarily unable to service your request. Please try again later. Reference #11.8397a746.1285274927.3ef1e4f0 unquote. I think that is an Akamai error, isn't it? The browser address is: http://www.facebook.com/home.php?#!/?ref=home -- Somebody should have said: A democracy is two wolves and a lamb voting on what to have for dinner. Freedom under a constitutional republic is a well armed lamb contesting the vote. Requiescas in pace o email Ex turpi causa non oritur actio Eppure si rinfresca ICBM Targeting Information: http://tinyurl.com/4sqczs http://tinyurl.com/7tp8ml
http://developers.facebook.com/live_status -----Original Message----- From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Larry Sheldon Sent: Thursday, September 23, 2010 1:54 PM To: outages@outages.org Subject: Re: [outages] Facebook On 9/23/2010 15:18, Chris Adams wrote:
Once upon a time, jon <relicwr@gmail.com> said:
Its been around 30 minutes. Is this the same result from less than 20 hours ago?
Interesting, www.facebook.com switched to an Akamai IP for me at about 19:30 UTC.
I thought FB always ran throuhg Akamai. I'm now getting quote Service Unavailable - DNS failure The server is temporarily unable to service your request. Please try again later. Reference #11.8397a746.1285274927.3ef1e4f0 unquote. I think that is an Akamai error, isn't it? The browser address is: http://www.facebook.com/home.php?#!/?ref=home -- Somebody should have said: A democracy is two wolves and a lamb voting on what to have for dinner. Freedom under a constitutional republic is a well armed lamb contesting the vote. Requiescas in pace o email Ex turpi causa non oritur actio Eppure si rinfresca ICBM Targeting Information: http://tinyurl.com/4sqczs http://tinyurl.com/7tp8ml _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
On 9/23/2010 15:53, Larry Sheldon wrote:
On 9/23/2010 15:18, Chris Adams wrote:
Once upon a time, jon <relicwr@gmail.com> said:
Its been around 30 minutes. Is this the same result from less than 20 hours ago?
Interesting, www.facebook.com switched to an Akamai IP for me at about 19:30 UTC.
I thought FB always ran throuhg Akamai.
I'm now getting quote Service Unavailable - DNS failure The server is temporarily unable to service your request. Please try again later.
Reference #11.8397a746.1285274927.3ef1e4f0 unquote.
I think that is an Akamai error, isn't it?
The browser address is: http://www.facebook.com/home.php?#!/?ref=home
Daughter in KCMO sez it (FB) was down, now up to her. Still down to me. Trace www.facebook.com goes to a-70-183-191-122.deploy.akamaitechnologies.com [ 70.183.191.122] -- Somebody should have said: A democracy is two wolves and a lamb voting on what to have for dinner. Freedom under a constitutional republic is a well armed lamb contesting the vote. Requiescas in pace o email Ex turpi causa non oritur actio Eppure si rinfresca ICBM Targeting Information: http://tinyurl.com/4sqczs http://tinyurl.com/7tp8ml
Works for me 5:16PM EST C:\Users\jluthman>dig facebook.com ; <<>> DiG 9.3.2 <<>> facebook.com ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1929 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;facebook.com. IN A ;; ANSWER SECTION: facebook.com. 3166 IN A 69.63.189.11 facebook.com. 3166 IN A 69.63.189.16 facebook.com. 3166 IN A 69.63.181.12 ;; Query time: 4 msec ;; SERVER: 192.168.21.1#53(192.168.21.1) ;; WHEN: Thu Sep 23 17:16:12 2010 ;; MSG SIZE rcvd: 78 Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 On Thu, Sep 23, 2010 at 4:59 PM, Larry Sheldon <LarrySheldon@cox.net> wrote:
On 9/23/2010 15:53, Larry Sheldon wrote:
On 9/23/2010 15:18, Chris Adams wrote:
Once upon a time, jon <relicwr@gmail.com> said:
Its been around 30 minutes. Is this the same result from less than 20 hours ago?
Interesting, www.facebook.com switched to an Akamai IP for me at about 19:30 UTC.
I thought FB always ran throuhg Akamai.
I'm now getting quote Service Unavailable - DNS failure The server is temporarily unable to service your request. Please try again later.
Reference #11.8397a746.1285274927.3ef1e4f0 unquote.
I think that is an Akamai error, isn't it?
The browser address is: http://www.facebook.com/home.php?#!/?ref=home
Daughter in KCMO sez it (FB) was down, now up to her.
Still down to me.
Trace www.facebook.com goes to a-70-183-191-122.deploy.akamaitechnologies.com [ 70.183.191.122]
-- Somebody should have said: A democracy is two wolves and a lamb voting on what to have for dinner.
Freedom under a constitutional republic is a well armed lamb contesting the vote.
Requiescas in pace o email Ex turpi causa non oritur actio Eppure si rinfresca
ICBM Targeting Information: http://tinyurl.com/4sqczs http://tinyurl.com/7tp8ml
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
DItto as of 2 min ago: <<>> DiG 9.6.0-APPLE-P2 <<>> facebook.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22921 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;facebook.com. IN A ;; ANSWER SECTION: facebook.com. 3427 IN A 69.63.189.16 facebook.com. 3427 IN A 69.63.189.11 facebook.com. 3427 IN A 69.63.181.12 ;; Query time: 106 msec ;; SERVER: 208.67.220.220#53(208.67.220.220) ;; WHEN: Thu Sep 23 17:33:42 2010 ;; MSG SIZE rcvd: 78 ~jason On Thu, Sep 23, 2010 at 5:16 PM, Josh Luthman <josh@imaginenetworksllc.com>wrote:
Works for me 5:16PM EST
C:\Users\jluthman>dig facebook.com
; <<>> DiG 9.3.2 <<>> facebook.com ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1929 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION: ;facebook.com. IN A
;; ANSWER SECTION: facebook.com. 3166 IN A 69.63.189.11 facebook.com. 3166 IN A 69.63.189.16 facebook.com. 3166 IN A 69.63.181.12
;; Query time: 4 msec ;; SERVER: 192.168.21.1#53(192.168.21.1) ;; WHEN: Thu Sep 23 17:16:12 2010 ;; MSG SIZE rcvd: 78
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
On Thu, Sep 23, 2010 at 4:59 PM, Larry Sheldon <LarrySheldon@cox.net> wrote:
On 9/23/2010 15:53, Larry Sheldon wrote:
On 9/23/2010 15:18, Chris Adams wrote:
Once upon a time, jon <relicwr@gmail.com> said:
Its been around 30 minutes. Is this the same result from less than 20 hours ago?
Interesting, www.facebook.com switched to an Akamai IP for me at about 19:30 UTC.
I thought FB always ran throuhg Akamai.
I'm now getting quote Service Unavailable - DNS failure The server is temporarily unable to service your request. Please try again later.
Reference #11.8397a746.1285274927.3ef1e4f0 unquote.
I think that is an Akamai error, isn't it?
The browser address is: http://www.facebook.com/home.php?#!/?ref=home
Daughter in KCMO sez it (FB) was down, now up to her.
Still down to me.
Trace www.facebook.com goes to a-70-183-191-122.deploy.akamaitechnologies.com [ 70.183.191.122]
-- Somebody should have said: A democracy is two wolves and a lamb voting on what to have for dinner.
Freedom under a constitutional republic is a well armed lamb contesting the vote.
Requiescas in pace o email Ex turpi causa non oritur actio Eppure si rinfresca
ICBM Targeting Information: http://tinyurl.com/4sqczs http://tinyurl.com/7tp8ml
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Its worth noting that facebook.com and www.facebook.com are handled differently. facebook.com is a limited set of RR IPs that redirect to www. www is a gslb'ed domain. On Thu, Sep 23, 2010 at 2:34 PM, Jason L. Sparks <jlsparks@gmail.com> wrote:
DItto as of 2 min ago:
<<>> DiG 9.6.0-APPLE-P2 <<>> facebook.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22921 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION: ;facebook.com. IN A
;; ANSWER SECTION: facebook.com. 3427 IN A 69.63.189.16 facebook.com. 3427 IN A 69.63.189.11 facebook.com. 3427 IN A 69.63.181.12
;; Query time: 106 msec ;; SERVER: 208.67.220.220#53(208.67.220.220) ;; WHEN: Thu Sep 23 17:33:42 2010 ;; MSG SIZE rcvd: 78
~jason
On Thu, Sep 23, 2010 at 5:16 PM, Josh Luthman <josh@imaginenetworksllc.com
wrote:
Works for me 5:16PM EST
C:\Users\jluthman>dig facebook.com
; <<>> DiG 9.3.2 <<>> facebook.com ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1929 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION: ;facebook.com. IN A
;; ANSWER SECTION: facebook.com. 3166 IN A 69.63.189.11 facebook.com. 3166 IN A 69.63.189.16 facebook.com. 3166 IN A 69.63.181.12
;; Query time: 4 msec ;; SERVER: 192.168.21.1#53(192.168.21.1) ;; WHEN: Thu Sep 23 17:16:12 2010 ;; MSG SIZE rcvd: 78
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
On Thu, Sep 23, 2010 at 4:59 PM, Larry Sheldon <LarrySheldon@cox.net> wrote:
On 9/23/2010 15:53, Larry Sheldon wrote:
On 9/23/2010 15:18, Chris Adams wrote:
Once upon a time, jon <relicwr@gmail.com> said:
Its been around 30 minutes. Is this the same result from less than 20 hours ago?
Interesting, www.facebook.com switched to an Akamai IP for me at about 19:30 UTC.
I thought FB always ran throuhg Akamai.
I'm now getting quote Service Unavailable - DNS failure The server is temporarily unable to service your request. Please try again later.
Reference #11.8397a746.1285274927.3ef1e4f0 unquote.
I think that is an Akamai error, isn't it?
The browser address is: http://www.facebook.com/home.php?#!/?ref=home
Daughter in KCMO sez it (FB) was down, now up to her.
Still down to me.
Trace www.facebook.com goes to a-70-183-191-122.deploy.akamaitechnologies.com [ 70.183.191.122]
-- Somebody should have said: A democracy is two wolves and a lamb voting on what to have for dinner.
Freedom under a constitutional republic is a well armed lamb contesting the vote.
Requiescas in pace o email Ex turpi causa non oritur actio Eppure si rinfresca
ICBM Targeting Information: http://tinyurl.com/4sqczs http://tinyurl.com/7tp8ml
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Also up for me in Central Oregon: root@oem-desktop:~# dig facebook.com ; <<>> DiG 9.5.1-P2.1 <<>> facebook.com ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55496 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 5, ADDITIONAL: 5 ;; QUESTION SECTION: ;facebook.com. IN A ;; ANSWER SECTION: facebook.com. 890 IN A 69.63.189.11 facebook.com. 890 IN A 69.63.189.16 facebook.com. 890 IN A 69.63.181.12 ;; AUTHORITY SECTION: facebook.com. 130293 IN NS ns1.facebook.com. facebook.com. 130293 IN NS ns2.facebook.com. facebook.com. 130293 IN NS ns4.facebook.com. facebook.com. 130293 IN NS ns3.facebook.com. facebook.com. 130293 IN NS ns5.facebook.com. ;; ADDITIONAL SECTION: ns1.facebook.com. 1436 IN A 204.74.66.132 ns2.facebook.com. 1436 IN A 204.74.67.132 ns3.facebook.com. 1436 IN A 69.63.178.21 ns4.facebook.com. 1436 IN A 69.63.186.49 ns5.facebook.com. 1436 IN A 69.63.176.200 ;; Query time: 8 msec ;; SERVER: 216.228.160.6#53(216.228.160.6) ;; WHEN: Thu Sep 23 14:46:44 2010 ;; MSG SIZE rcvd: 248 Plus, haven't heard any gripes from over the cube walls, all day. Dave Owens On Thu, Sep 23, 2010 at 2:16 PM, Josh Luthman <josh@imaginenetworksllc.com>wrote:
Works for me 5:16PM EST
C:\Users\jluthman>dig facebook.com
; <<>> DiG 9.3.2 <<>> facebook.com ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1929 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION: ;facebook.com. IN A
;; ANSWER SECTION: facebook.com. 3166 IN A 69.63.189.11 facebook.com. 3166 IN A 69.63.189.16 facebook.com. 3166 IN A 69.63.181.12
;; Query time: 4 msec ;; SERVER: 192.168.21.1#53(192.168.21.1) ;; WHEN: Thu Sep 23 17:16:12 2010 ;; MSG SIZE rcvd: 78
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
On Thu, Sep 23, 2010 at 4:59 PM, Larry Sheldon <LarrySheldon@cox.net> wrote:
On 9/23/2010 15:53, Larry Sheldon wrote:
On 9/23/2010 15:18, Chris Adams wrote:
Once upon a time, jon <relicwr@gmail.com> said:
Its been around 30 minutes. Is this the same result from less than 20 hours ago?
Interesting, www.facebook.com switched to an Akamai IP for me at about 19:30 UTC.
I thought FB always ran throuhg Akamai.
I'm now getting quote Service Unavailable - DNS failure The server is temporarily unable to service your request. Please try again later.
Reference #11.8397a746.1285274927.3ef1e4f0 unquote.
I think that is an Akamai error, isn't it?
The browser address is: http://www.facebook.com/home.php?#!/?ref=home
Daughter in KCMO sez it (FB) was down, now up to her.
Still down to me.
Trace www.facebook.com goes to a-70-183-191-122.deploy.akamaitechnologies.com [ 70.183.191.122]
-- Somebody should have said: A democracy is two wolves and a lamb voting on what to have for dinner.
Freedom under a constitutional republic is a well armed lamb contesting the vote.
Requiescas in pace o email Ex turpi causa non oritur actio Eppure si rinfresca
ICBM Targeting Information: http://tinyurl.com/4sqczs http://tinyurl.com/7tp8ml
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Many of my customers cant reach me from behind FioS. Traceroute shows a problem spot in Newark in Verizon backbone. 256ms latency at the last hop, then dies. It several hops within Verizon, so it cant be an Abovenet peering issue. If anyone from Verizon is listening out there, please look into this. Fios support refuses to troubleshoot. I know at least 20 people who have called in to complain, there are Verizon (DSL or Fios) through Jersey and Delaware. 1 gw-238-225.quonix.net (208.72.238.225) 0.256 ms 0.268 ms 0.270 ms 2 ge-11-1-2.mpr3.phl2.us.above.net (209.249.122.165) 0.174 ms 0.195 ms 0. 166 ms 3 xe-2-3-0.cr1.lga5.us.above.net (64.125.31.34) 2.422 ms 2.357 ms 2.412 m s 4 xe-0-1-0.er1.lga5.us.above.net (64.125.27.61) 2.222 ms 2.200 ms 2.228 m s 5 0.ge-3-2-0.BR3.NYC4.ALTER.NET (204.255.168.25) 2.254 ms 2.297 ms 2.264 ms 6 0.ge-4-2-0.NY5030-BB-RTR2.verizon-gni.net (152.63.10.54) 2.573 ms 2.605 m s 2.566 ms 7 P15-0-0.NWRKNJ-LCR-04.verizon-gni.net (130.81.29.195) 4.767 ms 4.701 ms 4.714 ms 8 P12-0-0.NWRKNJ-LCR-06.verizon-gni.net (130.81.27.7) 5.717 ms 5.348 ms 5 .262 ms 9 P14-0.NWRKNJ-LCR-08.verizon-gni.net (130.81.30.95) 5.265 ms 5.202 ms 5. 259 ms 10 * * * 11 * * * 12 * * * 13 * * * 14 * * *
On Thu, Sep 23, 2010 at 06:24:03PM +0000, john@quonix.net wrote:
Many of my customers cant reach me from behind FioS. Traceroute shows a problem spot in Newark in Verizon backbone. 256ms latency at the last hop, then dies.
It several hops within Verizon, so it cant be an Abovenet peering issue.
If anyone from Verizon is listening out there, please look into this. Fios support refuses to troubleshoot. I know at least 20 people who have called in to complain, there are Verizon (DSL or Fios) through Jersey and Delaware.
1 gw-238-225.quonix.net (208.72.238.225) 0.256 ms 0.268 ms 0.270 ms 2 ge-11-1-2.mpr3.phl2.us.above.net (209.249.122.165) 0.174 ms 0.195 ms 0. 166 ms 3 xe-2-3-0.cr1.lga5.us.above.net (64.125.31.34) 2.422 ms 2.357 ms 2.412 m s 4 xe-0-1-0.er1.lga5.us.above.net (64.125.27.61) 2.222 ms 2.200 ms 2.228 m s 5 0.ge-3-2-0.BR3.NYC4.ALTER.NET (204.255.168.25) 2.254 ms 2.297 ms 2.264 ms 6 0.ge-4-2-0.NY5030-BB-RTR2.verizon-gni.net (152.63.10.54) 2.573 ms 2.605 m s 2.566 ms 7 P15-0-0.NWRKNJ-LCR-04.verizon-gni.net (130.81.29.195) 4.767 ms 4.701 ms 4.714 ms 8 P12-0-0.NWRKNJ-LCR-06.verizon-gni.net (130.81.27.7) 5.717 ms 5.348 ms 5 .262 ms 9 P14-0.NWRKNJ-LCR-08.verizon-gni.net (130.81.30.95) 5.265 ms 5.202 ms 5. 259 ms 10 * * * 11 * * * 12 * * * 13 * * * 14 * * *
This isn't to say there's not a problem, but where is the "256ms latency at the last hop" evidence? All that's shown here is 3 probes at hop #9 with RTTs of 5.265, 5.202, and 5.259ms. The copy-paste was done poorly so the output is formatted badly. You're going to need to provide traceroutes from both directions (src->dst and dst->src), and should probably try UDP, TCP, and ICMP-based traceroutes to rule out filtering at the endpoints. Some traceroute implementations also offer a flag (on BSD it's -e, to be used alongside -p) which causes UDP and TCP destination packets to *not* increment their port number per probe, which can help if the destination uses a firewall (and the user has control over it). When dealing with Verizon, you should probably also indicate if all forms of packets are being filtered (meaning provide actual packet traces on both the source and destination ends showing something like the results of an telnet x.x.x.x yyyy towards you, where yyyy should be a port you're definitely listening on + have permitted on your firewall and/or port forwarded). Basically you want to confirm if the remote end receives a SYN+ACK response (from you) to their initial SYN. Good luck. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |
Once upon a time, Larry Sheldon <LarrySheldon@cox.net> said:
On 9/23/2010 15:18, Chris Adams wrote:
Interesting, www.facebook.com switched to an Akamai IP for me at about 19:30 UTC.
I thought FB always ran throuhg Akamai.
Well, it changed from an external host to our local Akamai servers. Now that it is back up, it is back to external servers. Looks like this might have been an Akamai problem then. -- Chris Adams <cmadams@hiwaay.net> Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble.
Details of the outage case are at http://www.facebook.com/notes/facebook-engineering/more-details-on-todays-ou... In short, code problem which required them "turn off" the site to resolve. Nothing to do with Akamai/etc. Scott.
participants (17)
-
Chris Adams -
Dave Owens -
david raistrick -
Jason L. Sparks -
Jay R. Ashworth -
Jeremy Chadwick -
John Von Essen -
john＠quonix.net
-
jon -
Josh Luthman -
Larry Sheldon -
Matthew Dodd -
Mike Gazzerro -
Robin Pimentel -
Scott Howard -
Seth Mattinen -
Wil Schultz