I just checked into this. Looks like some sort of transport issue in sfba. The appropriate teams have been working on it for a little while. I haven't heard what the expected time to repair is yet. John On Oct 2, 2013 5:57 PM, "Jeremy Chadwick" <jdc@koitsu.org> wrote:
This just happened -- looks like something in Comcast's network within Oakland broke, reconverged (quickly and seamless), but issues remain.
Ren Provo et al, if you're lurking, might want to make someone aware of this (if they aren't already). :-)
src: 76.102.14.35 dst: 208.79.90.130
=== Wed Oct 2 16:14:00 PDT 2013 (1380755640) Start: Wed Oct 2 16:14:00 2013 HOST: icarus.home.lan Loss% Snt Rcv Last Avg Best Wrst 1.|-- 192.168.1.1 0.0% 40 40 0.4 0.3 0.2 0.4 2.|-- 76.102.12.1 0.0% 40 40 9.6 10.8 8.1 73.5 3.|-- 68.86.249.253 0.0% 40 40 8.9 10.2 8.3 62.2 4.|-- 69.139.199.98 0.0% 40 40 10.6 13.1 9.8 47.4 5.|-- 68.85.155.14 0.0% 40 40 12.0 16.8 11.4 87.7 6.|-- 68.86.90.157 0.0% 40 40 14.6 17.8 12.6 87.5 7.|-- 68.86.89.210 0.0% 40 40 23.7 24.7 20.6 92.9 8.|-- 68.86.87.210 0.0% 40 40 25.9 25.0 20.7 99.0 9.|-- 173.167.57.138 0.0% 40 40 22.1 24.7 21.3 89.0 10.|-- 69.31.127.141 0.0% 40 40 20.5 22.0 19.8 67.1 11.|-- 69.31.124.114 0.0% 40 40 20.6 21.6 20.0 45.9 12.|-- 67.199.135.102 0.0% 40 40 39.9 36.5 21.1 231.0 13.|-- 208.79.90.130 0.0% 40 40 21.2 20.7 20.2 21.7 === END
=== Wed Oct 2 16:15:00 PDT 2013 (1380755700) Start: Wed Oct 2 16:15:00 2013 HOST: icarus.home.lan Loss% Snt Rcv Last Avg Best Wrst 1.|-- 192.168.1.1 0.0% 40 40 0.2 0.3 0.2 0.4 2.|-- 76.102.12.1 0.0% 40 40 8.1 9.4 8.1 13.6 3.|-- 68.86.249.253 0.0% 40 40 8.7 9.7 8.4 14.1 4.|-- 68.86.143.98 0.0% 40 40 13.1 13.1 10.2 21.1 5.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0 6.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0 7.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0 8.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0 9.|-- 173.167.57.138 0.0% 40 40 33.0 33.9 31.6 37.1 10.|-- 69.31.127.141 5.0% 40 38 30.2 31.4 29.7 39.3 11.|-- 69.31.124.114 7.5% 40 37 31.8 38.5 29.8 104.0 12.|-- 67.199.135.102 7.5% 40 37 31.1 36.6 30.6 119.7 13.|-- 208.79.90.130 7.5% 40 37 30.9 31.9 30.4 38.9 === END
68.86.249.253 = te-0-2-0-5-ur06.santaclara.ca.sfba.comcast.net 69.139.199.98 = te-1-1-0-11-ar01.oakland.ca.sfba.comcast.net 68.85.155.14 = be-90-ar01.sfsutro.ca.sfba.comcast.net 68.86.143.98 = te-1-1-0-5-ar01.oakland.ca.sfba.comcast.net
And packet loss has gradually been getting worse (impacting all forms of packets, not just TCP).
=== Wed Oct 2 16:45:00 PDT 2013 (1380757500) Start: Wed Oct 2 16:45:00 2013 HOST: icarus.home.lan Loss% Snt Rcv Last Avg Best Wrst 1.|-- 192.168.1.1 0.0% 40 40 0.2 0.3 0.2 0.4 2.|-- 76.102.12.1 0.0% 40 40 8.6 9.9 8.1 14.3 3.|-- 68.86.249.253 0.0% 40 40 9.2 10.2 8.4 16.7 4.|-- 68.86.143.98 0.0% 40 40 13.9 13.1 10.6 16.6 5.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0 6.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0 7.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0 8.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0 9.|-- 173.167.57.138 15.0% 40 34 34.6 34.8 31.7 41.7 10.|-- 69.31.127.141 25.0% 40 30 31.3 32.6 30.0 41.7 11.|-- 69.31.124.114 5.0% 40 38 31.8 33.5 30.5 58.6 12.|-- 67.199.135.102 12.8% 39 34 30.9 37.1 30.9 77.3 13.|-- 208.79.90.130 7.7% 39 36 32.9 32.3 30.8 37.3 === END
And here's what I see on the return path, for a brief run:
src: 208.79.90.130 dst: 76.102.14.35
Host Loss% Snt Rcv Last Avg Best Wrst 1. 208.79.90.129 0.0% 93 93 1.5 6.1 0.9 163.0 2. s7.lax.arpnetworks.com 0.0% 93 93 0.7 11.2 0.5 150.5 3. ge0-15.as01.lax07.mzima.net 0.0% 93 93 4.1 2.4 0.4 64.6 4. ae3-50g.cr1.lax1.us.nlayer.net 0.0% 93 93 0.6 2.2 0.4 56.0 5. ae1-50g.ar1.lax1.us.nlayer.net 1.1% 93 92 3.1 3.3 1.3 6.4 6. be-102-pe01.600wseventh.ca.ibone.comcast.ne 0.0% 93 93 3.8 3.1 1.1 5.4 7. pos-0-8-0-0-cr01.losangeles.ca.ibone.comcas 0.0% 93 93 3.0 2.9 1.5 6.1 8. he-2-15-0-0-cr01.sanjose.ca.ibone.comcast.n 0.0% 92 92 8.6 10.3 8.1 12.5 9. he-0-9-0-0-ar01.sfsutro.ca.sfba.comcast.net 0.0% 92 92 12.5 11.6 9.8 13.6 10. be-100-ar01.oakland.ca.sfba.comcast.net 3.3% 92 88 22.0 25.2 21.9 49.6 11. te-0-4-0-2-ur06.santaclara.ca.sfba.comcast. 3.3% 92 89 23.8 23.8 23.0 25.4 12. te-6-0-acr03.santaclara.ca.sfba.comcast.net 9.8% 92 83 23.8 24.4 22.6 46.3 13. c-76-102-14-35.hsd1.ca.comcast.net 6.5% 92 86 36.3 31.7 30.5 38.3
-- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB |
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages