September 2013 - Outages

Netflix over IPv6
by Frank Bulk 01 Oct '13

01 Oct '13

Friday evening, Sunday morning, and Sunday evening, our monitoring system has seen sporadic HTTPv6 access issues to www.netflix.com. Starting since 10:36 am Central this morning, I've seen sporadic ICMPv6 failures to www.netflix.com. Now www.netflix.com has many AAAA records that are likely different around the world and I don't have our monitoring system set up to individually monitor each AAAA record, but wondering if anyone else has seen the same thing? Regards, Frank ================ Current AAAA's, currently pingable: 2406:da00:ff00::b849:f83b 2406:da00:ff00::b849:fcb3 2406:da00:ff00::6b16:ebed ================= Former AAAA's during non-working state: 2620:108:700f::36f4:7696 2620:108:700f::36f4:dd14 2620:108:700f::36f4:e97a 2620:108:700f::3270:5c77 2620:108:700f::3270:5c7b 2620:108:700f::36d6:752 2620:108:700f::36d6:b87 2620:108:700f::36d6:22b2

2 2

Level3 ICMP Filtering in Chicago?
by Justin Krejci 30 Sep '13

30 Sep '13

At about 3:30PM central time today we started seeing ICMP packet loss going into Level3 networks, we believe it is likely some sort of Level3 filtering of some sort. Two traces below from the same origin to the same destination at the same time, one is ICMP and one is UDP. It is not just traceroute ICMP packets, we cannot ping our own equipment when it takes a Level3 path. Other UDP/TCP traffic so far seems unaffected. Is anyone else seeing this? This appears to be a Chicago-area connection. Using ICMP to trace to: 204.11.209.66 Tracing to host 204.11.209.66 traceroute to 204.11.209.66 (204.11.209.66), 30 hops max, 60 byte packets 1 216.17.34.1 (216.17.34.1) [as10242/AS10242] 0.558 ms 0.554 ms 0.552 ms 2 v103.usi-cr02-mpls.usinternet.com (216.17.36.25) [as10242/AS10242] 1.094 ms 1.199 ms 1.555 ms 3 10gigabitethernet2-6.core1.msp1.he.net (216.66.73.173) [AS6939] 1.203 ms 1.530 ms 1.533 ms 4 100gigabitethernet7-1.core1.chi1.he.net (184.105.223.177) [AS46841] 9.076 ms 9.084 ms 9.154 ms 5 chi-bb1-link.telia.net (213.248.104.213) [AS1299] 9.155 ms 9.163 ms 9.163 ms 6 * * * 7 * * * 8 * * * 9 * * * 10 * * * 11 * * * 12 * * * 13 * * * 14 * * * 15 * * * Using UDP to trace to: 204.11.209.66 Tracing to host 204.11.209.66 traceroute to 204.11.209.66 (204.11.209.66), 30 hops max, 60 byte packets 1 216.17.34.1 (216.17.34.1) [as10242/AS10242] 0.438 ms 0.480 ms 0.495 ms 2 v103.usi-cr02-mpls.usinternet.com (216.17.36.25) [as10242/AS10242] 1.106 ms 1.166 ms 1.235 ms 3 10gigabitethernet2-6.core1.msp1.he.net (216.66.73.173) [AS6939] 4.447 ms 4.445 ms 4.440 ms 4 100gigabitethernet7-1.core1.chi1.he.net (184.105.223.177) [AS46841] 12.821 ms 12.898 ms 12.893 ms 5 chi-bb1-link.telia.net (213.248.104.213) [AS1299] 9.056 ms 9.052 ms 9.054 ms 6 level3-ic-300135-chi-bb1.c.telia.net (213.248.87.238) [AS1299] 23.943 ms 24.000 ms 18.147 ms 7 ae-32-52.ebr2.Chicago1.Level3.net (4.69.138.62) [AS3356] 20.890 ms 18.162 ms 18.189 ms 8 ae-5-5.ebr2.Chicago2.Level3.net (4.69.140.194) [AS3356] 20.993 ms 20.982 ms 22.160 ms 9 ae-22-52.car2.Chicago2.Level3.net (4.69.138.165) [AS3356] 18.164 ms 22.165 ms 18.231 ms 10 RED-ANVIL-L.car2.Chicago2.Level3.net (4.30.14.162) [AS3356] 44.853 ms 44.822 ms 44.810 ms 11 ge-1-0-1-0.cr1.noc.ip.redanvil.net (204.15.100.173) [AS33693] 19.849 ms 19.888 ms 19.797 ms

3 4

www.i3d.net is down (another AAAA fiasco)
by Constantine A. Murenin 30 Sep '13

30 Sep '13

Hello, It would seem like i3d.net web-site is down / half-down. #### telnet i3d.net http Trying 2a00:1630:1::13d... telnet: connect to address 2a00:1630:1::13d: Operation timed out Trying 188.122.94.99... Connected to i3d.net. Escape character is '^]'. #### #### 5 2001:18e8:ffff:54::2 6.568 ms 6.540 ms 6.545 ms 6 10gigabitethernet4-4.core1.chi1.he.net 6.548 ms 10gigabitethernet2.switch2.chi1.he.net 6.538 ms 10gigabitethernet4-4.core1.chi1.he.net 13.932 ms 7 100gigabitethernet7-2.core1.nyc4.he.net 30.890 ms 26.764 ms 26.866 ms 8 10gigabitethernet6-4.core1.lon1.he.net 99.531 ms 102.892 ms 96.918 ms 9 10gigabitethernet1-1.core1.ams1.he.net 107.169 ms 107.118 ms 112.465 ms 10 30ge.ams-ix.i3d.net 107.408 ms 107.452 ms 107.570 ms 11 cr0.i3d.net 117.612 ms 108.926 ms 107.925 ms 12 2a00:1630:1fff::3:5 149.623 ms 118.996 ms 107.560 ms 13 * * * 14 * * * #### P.S. Is anyone keeping any statistics of how many web-sites with AAAA records don't respond on port 80? C.

3 2

LAX routing anomaly: NetworkLayer? CoreSite? Hong Kong ISPs?
by Jeremy Chadwick 30 Sep '13

30 Sep '13

I just came across this on NANOG while troubleshooting something: http://mailman.nanog.org/pipermail/nanog/2013-September/061332.html If the issue described there is what I'm seeing, then it looks like some part of the 'net in the LA area is very very broken -- and worse, *has* been broken since roughly Sep 27 21:16:00 PDT (UTC-7). What I saw (and am still seeing): src IP: 208.79.90.130 (Southern CA, AS 25795 (I think)) dst IP: 67.18.187.25 (Texas, AS 21844 (I think)) === Fri Sep 27 21:14:00 PDT 2013 (1380341640) Start: Fri Sep 27 21:14:00 2013 HOST: omake.koitsu.org Loss% Snt Rcv Last Avg Best Wrst 1.|-- 208.79.90.129 0.0% 40 40 3.7 11.0 0.9 202.9 2.|-- 206.223.143.131 0.0% 40 40 0.6 0.8 0.5 8.5 3.|-- 173.192.18.140 2.5% 40 39 28.4 30.1 28.2 86.8 4.|-- 173.192.18.225 0.0% 40 40 29.0 29.2 28.7 40.9 5.|-- 70.87.255.66 0.0% 40 40 31.6 31.5 31.3 31.9 6.|-- 70.87.254.74 0.0% 40 40 29.0 31.5 28.6 60.3 7.|-- 67.18.7.90 0.0% 40 40 28.7 28.6 28.4 29.0 8.|-- 67.18.187.25 0.0% 40 40 31.9 31.5 31.3 31.9 === END === Fri Sep 27 21:15:00 PDT 2013 (1380341700) Start: Fri Sep 27 21:15:00 2013 HOST: omake.koitsu.org Loss% Snt Rcv Last Avg Best Wrst 1.|-- 208.79.90.129 0.0% 40 40 2.2 51.1 0.9 245.7 2.|-- 206.223.143.131 2.5% 40 39 0.9 12.3 0.5 183.2 3.|-- 173.192.18.140 5.0% 40 38 0.8 29.4 0.7 284.1 4.|-- 173.192.18.225 0.0% 40 40 0.8 17.1 0.5 177.8 5.|-- 70.87.255.66 30.0% 40 28 11.2 39.1 3.4 199.4 6.|-- 70.87.254.74 0.0% 40 40 11.0 21.1 0.5 121.5 7.|-- 67.18.7.90 0.0% 40 40 163.8 122.6 4.6 347.0 8.|-- 67.18.187.25 22.5% 40 31 171.6 81.4 0.5 173.8 === END === Fri Sep 27 21:16:00 PDT 2013 (1380341760) Start: Fri Sep 27 21:16:00 2013 HOST: omake.koitsu.org Loss% Snt Rcv Last Avg Best Wrst 1.|-- 208.79.90.129 0.0% 40 40 2.9 7.4 0.9 95.8 2.|-- 208.79.88.135 0.0% 40 40 0.6 15.8 0.4 182.3 3.|-- 129.250.198.185 0.0% 40 40 0.7 0.8 0.7 1.2 4.|-- 129.250.5.69 0.0% 40 40 0.7 5.1 0.6 34.1 5.|-- 129.250.6.11 45.0% 40 22 17.8 14.6 9.6 47.1 6.|-- 129.250.5.53 0.0% 40 40 11.6 10.8 10.0 12.2 7.|-- 128.241.219.234 0.0% 40 40 162.7 163.1 161.7 178.3 8.|-- 173.192.18.151 2.5% 40 39 172.3 172.3 171.2 176.3 9.|-- 173.192.18.166 12.5% 40 35 157.2 156.3 155.6 157.9 10.|-- 173.192.18.140 7.5% 40 37 198.5 201.1 198.4 234.7 11.|-- 173.192.18.225 0.0% 40 40 183.0 190.1 182.9 355.2 12.|-- 70.87.255.66 0.0% 40 40 183.0 183.8 183.0 185.9 13.|-- 70.87.254.74 0.0% 40 40 193.6 195.8 192.0 284.1 14.|-- 67.18.7.90 2.5% 40 39 197.0 197.4 196.9 198.8 15.|-- 67.18.187.25 2.5% 40 39 192.8 192.8 192.2 194.0 === END FYI: 206.223.143.131 resolves to te2-6.bbr01.cs01.lax01.networklayer.com.any2ix.coresite.com. ARIN states 206.223.143.0/24 is CoreSite. The packet path for the above is: LAX (ARP Networks) -> SFO (NTT/Verio) -> San Jose (NTT/Verio) -> LAX (NetworkLayer) -> Dallas (NetworkLayer and Linode) -> 67.18.187.25 And now for the amusing part -- the return path: src IP: 67.18.187.25 (Texas, AS 21844 (I think)) dst IP: 208.79.90.130 (Southern CA, AS 25795 (I think)) traceroute to omake.koitsu.org (208.79.90.130), 30 hops max, 60 byte packets 1 router2-dal.linode.com (67.18.7.162) 0.604 ms 0.812 ms 0.815 ms 2 xe-2-0-0.car04.dllstx2.networklayer.com (67.18.7.93) 0.415 ms 0.428 ms 0.426 ms 3 po102.dsr01.dllstx2.networklayer.com (70.87.254.81) 0.756 ms 0.764 ms 0.963 ms 4 po21.dsr01.dllstx3.networklayer.com (70.87.255.65) 0.702 ms 0.759 ms 1.103 ms 5 ae16.bbr02.eq01.dal03.networklayer.com (173.192.18.228) 0.438 ms 0.446 ms 0.441 ms 6 ae7.bbr01.eq01.dal03.networklayer.com (173.192.18.208) 1.271 ms 0.709 ms 0.612 ms 7 ae0.bbr01.cs01.lax01.networklayer.com (173.192.18.141) 28.374 ms 28.431 ms 28.411 ms 8 ae7.bbr02.cs01.lax01.networklayer.com (173.192.18.167) 28.773 ms 28.747 ms 28.727 ms 9 * * * 10 ae0.bbr01.eq01.tok01.networklayer.com (50.97.18.161) 137.992 ms 138.001 ms 137.967 ms 11 ae7.bbr02.eq01.tok01.networklayer.com (50.97.18.163) 138.084 ms 138.068 ms 138.064 ms 12 ae0.bbr01.pn01.hkg01.networklayer.com (50.97.18.167) 181.952 ms 182.026 ms 181.990 ms 13 hutchcity21-10G.hkix.net (202.40.160.193) 180.121 ms 179.677 ms 179.631 ms 14 218.189.5.51 (218.189.5.51) 179.553 ms 179.521 ms 179.505 ms 15 d1-34-224-143-118-on-nets.com (118.143.224.34) 186.883 ms 186.829 ms 186.807 ms 16 * * * 17 omake.koitsu.org (208.79.90.130) 192.086 ms 192.973 ms 192.095 ms The packet path for the above is: Dallas (Linode then NetworkLayer) -> LAX (NetworkLayer) -> Tokyo (NetworkLayer) -> Hong Kong (NetworkLayer then via HKIX) -> Hong Kong (Hutchison Global Communications) (AS 9304) -> ? -> 208.79.90.130 Reminder: 208.79.90.130 is not physically in Hong Kong. I haven't spent the time to look at bgplay.routeviews.org yet, but I'm not sure it'll necessarily have insights into any of this. I have reached out to ARP Networks to have them check things, but this is an interesting situation I thought I'd mention as folks more familiar with BGP than myself could probably assist with. I can only speculate at this point, but I wonder if AS 9304 may have announced some kind of preferencing that certain things have picked up (but never withdrew/changed?). There are lots of explanations though, so that's purely hearsay on my part. -- | Jeremy Chadwick jdc(a)koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB |

2 2

tumblr custom domains names (72.32.231.8) are down (for good?)
by Constantine A. Murenin 29 Sep '13

29 Sep '13

Dear outages@, It would seem like there is no more any web-server at 72.32.231.8. This IP address was the IP-address that you had to configure your domain name with to have the "custom domain name" feature within tumblr. Currently if I log-in to tumblr, I'm told: "Please update your domain configurations for <..>. Learn more." With the link from "Learn more" following to http://www.tumblr.com/docs/en/custom_domains. It seems like it's been a couple of weeks (or even months) as this has been the case, since my tumblr is almost completely gone from Google now. However, there was no such message in the dashboard just a couple of months ago earlier this year, so, this seems like a somewhat unannounced and unexplained downtime. Why unexplained and unannounced? There are no mentions of 72.32.231.8 on that "Learn more" page, only 66.6.44.4 is mentioned, and, for completeness sake, no details are provided for why anything has had / has to be updated in the first place, or when the old IP-address will stop / has stopped serving custom domains. And there is not much excuse for such information to be missing here, since 66.6.32.0/20 has been allocated for TUMBLR a couple of years ago (2011-04-25), yet it seems like it was never explained to the users at large that a change was forthcoming. Were people really supposed to change their settings when everything was working and when nowhere was it explained that a downtime would be forthcoming should a change not be made and the custom domain not re-configured anew? Well, just a heads-up and a rant, I guess. (BTW, it seems like the RSS imports feature has suffered a similar fate a couple of years prior, too.) Best regards, Constantine.

2 2

Cogent / IAD / Ashburn
by Randy McAnally 27 Sep '13

27 Sep '13

Anyone noticing random src/dest IP pairs being blackholed at the Cogent peer router(s) since 4AM EST? No rhyme or reason. Even cogent customer IPs. I'm guessing a failing LAG somewhere... mtr x.x.101.98 --address x.x.160.80 Host Loss% Last Avg Best Wrst StDev 1. 209.9.238.161 0.0% 0.4 0.5 0.4 0.6 0.1 2. 38.122.250.x 0.0% 17.7 4.6 0.6 17.7 7.4 3. te1-6.ccr02.iad01.atlas.cogentco.com 0.0% 190.9 120.3 1.1 190.9 74.3 4. te0-8-0-33.ccr41.iad02.atlas.cogentco.com 0.0% 0.8 1.4 0.7 3.8 1.3 5. be2177.ccr22.dca01.atlas.cogentco.com 0.0% 2.0 2.0 1.8 2.3 0.2 be2112.mpd22.dca01.atlas.cogentco.com 6. be2155.mpd22.ord01.atlas.cogentco.com 0.0% 21.1 21.0 20.9 21.1 0.1 be2154.mpd21.ord01.atlas.cogentco.com be2153.ccr22.ord01.atlas.cogentco.com be2152.ccr21.ord01.atlas.cogentco.com 7. te0-4-0-5.ccr22.mci01.atlas.cogentco.com 0.0% 32.8 32.8 32.7 33.0 0.1 te0-4-0-0.mpd22.mci01.atlas.cogentco.com te0-0-0-3.mpd22.mci01.atlas.cogentco.com 8. be2132.ccr21.sfo01.atlas.cogentco.com 0.0% 70.8 70.8 70.5 71.6 0.4 be2133.ccr22.sfo01.atlas.cogentco.com be2135.mpd22.sfo01.atlas.cogentco.com be2134.mpd21.sfo01.atlas.cogentco.com 9. be2166.ccr21.sjc01.atlas.cogentco.com 0.0% 72.0 72.3 72.0 73.3 0.5 be2167.ccr22.sjc01.atlas.cogentco.com 10. be2000.ccr21.sjc03.atlas.cogentco.com 0.0% 73.2 77.6 72.2 97.7 11.3 be2047.ccr21.sjc03.atlas.cogentco.com 11. cox.sjc03.atlas.cogentco.com 0.0% 75.0 75.1 75.0 75.1 0.1 12. elcndsrj01-ae0.0.rd.sd.cox.net 0.0% 82.6 82.7 82.4 83.4 0.4 <snip> mtr x.x.101.98 --address x.x.160.81 Host Loss% Last Avg Best Wrst StDev 1. x.x.238.161 0.0% 0.8 0.6 0.4 0.8 0.2 2. 38.122.250.x 0.0% 0.9 0.8 0.7 0.9 0.1 3. ??? -- Randy M

1 0

Looks like Gmail is having issues.
by Eric Tykwinski 24 Sep '13

24 Sep '13

Just had a call about delays, so figured I would let others know: http://www.google.com/appsstatus#hl=en&v=issue&ts=1379995199000&sid=1&iid=04 3f082bc7cd18e15458318035d9bc7a Sincerely, Eric Tykwinski TrueNet, Inc. P: 610-429-8300 F: 610-429-3222

4 4

Re: [outages] [Outages-discussion] spam to outages folks from qualisystems.com?
by Jay Ashworth 23 Sep '13

23 Sep '13

----- Original Message ----- > From: "Scott Weeks" <surfer(a)mauigateway.com> > It looks like slimeball spammers are scraping the outages lists > foe email addresses. Someone told me they see this scum company > on these lists. Has anyone gotten one of these, so I can validate > the info is correct? I've not myself seen any, Scott, and yours is the first report. <moderator> But, for the record: *IT IS NOT PERMISSIBLE FOR ANYONE TO SCRAPE THIS LIST OR IT'S ARCHIVE FOR EMAIL ADDRESSES FOR COMMERCIAL EMAIL*. If a list member *solicits* suggestions, recommendations, or quotes, that's another thing. The fact that I had to say it concerning these lists should not be considered evidence for exceptio probat regulum: it's *never* OK to do that. Anywhere. </moderator> Cheers, -- jr "you'd think people would get it by now" a -- Jay R. Ashworth Baylink jra(a)baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274

1 0

Re: [outages] OT: Weird problem with Firefox & G+
by Paul Ferguson 23 Sep '13

23 Sep '13

Following up on this... It appears AdBlock Plus is the culprit here: https://plus.google.com/116028828272319130998/posts/doopZKh1G3u So either adding an exception for plus.google.com or simply disabling for that specific property "fixes" the problem. FYI, - ferg On 9/22/2013 11:28 AM, Paul Ferguson wrote: > Pardon the off-topic post, but... > > For some bizarre reason, Google+ started giving me an "invisible" page > after logging in this morning. It starts off fine, but then after > pulling content from ssl.gstatic.com and *.googleusercontent.com, etc., > it then goes "invisible". > > This is with Firefox 24.0 -- I have, one by one, unloaded and restarted > Firefox with various plugins disabled (e.g. NoScript, HTTPS Everywhere, > etc.) and the problem persists. > > There *is* content there, it is just "invisible" (e.g. body { > visibility: hidden; > }) > > ...and seems to work fine in IE (not my browser of choice). > > Just curious if anyone else running Firefox 24.0 has run into this > problem today on G+, or if Google made some sort of "enhancement" which > manifests itself this way. > > All other Google property pages work fine, just G+ has exhibited this > behavior today. > > Thanks, > > - ferg > > -- Paul Ferguson Vice President, Threat Intelligence Internet Identity, Tacoma, Washington USA IID --> "Connect and Collaborate" --> www.internetidentity.com

2 1

gmail.com 550-5.7.1 refusing mtr(8) output; other excessive bounces
by Constantine A. Murenin 22 Sep '13

22 Sep '13

This has happened to me several times recently; has never happened until a couple of weeks ago. >From the https://mail.google.com/mail/ web-interface, Cc an alias on my server, which expands to, amongst other things, my gmail.com email address; a bounce is immediately received in the thread (apparently, gmail doesn't reject the bounces from the very same server from which some messages are rejected). Most likely cause: too many domain names from the mtr(8) output in the message. Same for sending a similar message directly from my own server, directly to my gmail.com address, being in the Cc field: a bounce is received within my own server, no message is received at gmail. The bounce from such an experience is at the end of this message. I consider this a major outage: how many other valid semi-automated emails addressed to myself have been unfairly rejected by gmail on the SMTP level recently? I also now know why my gmail.com address has been unsubscribed from several mailing lists over this summer, citing excessive bouncing. My own mailserver wasn't even involved: <<## Delivered-To: mureninc(a)gmail.com Received: by 10.114.5.70 with SMTP id q6csp13073ldq; Tue, 10 Sep 2013 03:43:00 -0700 (PDT) X-Received: by 10.204.76.203 with SMTP id d11mr18914232bkk.3.1378809779724; Tue, 10 Sep 2013 03:42:59 -0700 (PDT) Return-Path: <lm-sensors-bounces(a)lm-sensors.org> Received: from flocki.atrpms.net (m.at.physik.fu-berlin.de. [160.45.254.30]) by mx.google.com with ESMTPS id zk7si1773373bkb.190.1969.12.31.16.00.00 (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 10 Sep 2013 03:42:59 -0700 (PDT) Received-SPF: neutral (google.com: 160.45.254.30 is neither permitted nor denied by best guess record for domain of lm-sensors-bounces(a)lm-sensors.org) client-ip=160.45.254.30; Authentication-Results: mx.google.com; spf=neutral (google.com: 160.45.254.30 is neither permitted nor denied by best guess record for domain of lm-sensors-bounces(a)lm-sensors.org) smtp.mail=lm-sensors-bounces(a)lm-sensors.org Received: from localhost ([::1] helo=m.at.physik.fu-berlin.de) by flocki.atrpms.net with esmtp (Exim 4.80.1) id 1VJLPJ-0001fB-QJ for mureninc(a)gmail.com; Tue, 10 Sep 2013 12:42:59 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit From: lm-sensors-request(a)lm-sensors.org To: mureninc(a)gmail.com ... ... Your membership in the mailing list lm-sensors has been disabled due to excessive bounces The last bounce received from you was dated 10-Sep-2013. You will not get any more messages from this list until you re-enable your membership. You will receive 3 more reminders like this before your membership in the list is deleted. ... ##>> If you need any further info, feel free to contact me off-list. C. ----- Forwarded message from Mail Delivery Subsystem <MAILER-DAEMON@XXXXXXXXXX> ----- Date: Sun, 22 Sep 2013 11:36:28 -0700 (PDT) From: Mail Delivery Subsystem <MAILER-DAEMON@XXXXXXXXXX> To: XXXXXX@bugmailXXXXXXXX Subject: Returned mail: see transcript for details The original message was received at Sun, 22 Sep 2013 11:36:19 -0700 (PDT) from XXXXXlocalhost [127.0.0.1] ----- The following addresses had permanent fatal errors ----- <mureninc(a)gmail.com> (reason: 550-5.7.1 [2001:470:7240:: 12] Our system has detected that this message is) ----- Transcript of session follows ----- ... while talking to gmail-smtp-in.l.google.com.: >>> DATA <<< 550-5.7.1 [2001:470:7240:: 12] Our system has detected that this message is <<< 550-5.7.1 likely unsolicited mail. To reduce the amount of spam sent to Gmail, <<< 550-5.7.1 this message has been blocked. Please visit <<< 550-5.7.1 http://support.google.com/mail/bin/answer.py?hl=en&answer=188131 for <<< 550 5.7.1 more information. od6si8589779bkb.242 - gsmtp 554 5.0.0 Service unavailable Reporting-MTA: dns; XXXXXXXXXX Received-From-MTA: DNS; localhost Arrival-Date: Sun, 22 Sep 2013 11:36:19 -0700 (PDT) Final-Recipient: RFC822; mureninc(a)gmail.com Action: failed Status: 5.7.1 Remote-MTA: DNS; gmail-smtp-in.l.google.com Diagnostic-Code: SMTP; 550-5.7.1 [2001:470:7240:: 12] Our system has detected that this message is Last-Attempt-Date: Sun, 22 Sep 2013 11:36:20 -0700 (PDT) Return-Path: <XXXXXX@bugmailXXXXXXXX> Received: from XXXXXXXXXX (XXXX@localhost [127.0.0.1]) by XXXXXXXXXX (8.14.5/8.14.5) with ESMTP id r8MIaJGR015280; Sun, 22 Sep 2013 11:36:19 -0700 (PDT) Received: (from XXXX@localhost) by XXXXXXXXXX (8.14.5/8.14.5/Submit) id r8MIaJ9o021840; Sun, 22 Sep 2013 11:36:19 -0700 (PDT) X-Authentication-Warning: XXXXXXXXXX: XXXX set sender to XXXXXX@bugmailXXXXXXXX using -f Date: Sun, 22 Sep 2013 11:36:19 -0700 From: "Constantine A. Murenin" <XXXXXX@bugmailXXXXXXXX> To: noc(a)he.net Cc: mureninc(a)gmail.com Subject: packet loss at 10gigabitethernet9-6.core1.par2.he.net Message-ID: <20130922183619.GA10404@XXXXXXXXXX> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) ----- End forwarded message -----

1 0