I am getting ~50% packet loss to 8.8.8.8, 4.2.2.2, 216.58.216.238, and 172.217.1.46 on two separate Charter (Time Warner) lines in SE Wisconsin. It looks to be affecting just ping requests as DNS and google websites are still working. On a Level 3 line things look normal. ---------------------------------------------------------------------------------------------------------------------------- Information contained in this email transmission is privileged and confidential. If you are not the intended recipient, do not read, distribute or reproduce this transmission (including any attachments). If you have received this email in error, please notify the sender by email reply. ----------------------------------------------------------------------------------------------------------------------------
here in NYC im also having issues to 8.8.4.4 from Time Warner Cable On Thu, Jan 5, 2017 at 10:12 AM, Jesse Curtis via Outages < outages@outages.org> wrote:
I am getting ~50% packet loss to 8.8.8.8, 4.2.2.2, 216.58.216.238, and 172.217.1.46 on two separate Charter (Time Warner) lines in SE Wisconsin. It looks to be affecting just ping requests as DNS and google websites are still working.
On a Level 3 line things look normal.
------------------------------ Information contained in this email transmission is privileged and confidential. If you are not the intended recipient, do not read, distribute or reproduce this transmission (including any attachments). If you have received this email in error, please notify the sender by email reply. ------------------------------
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Izzy Goldstein Chief Technology Officer Main: (212) 477-1000 x 2085 <(212)%20477-1000> Direct: (929) 477-2085 Website: www.telego.com <http://www.telego.net/> <http://www.telego.com/> Confidentiality Notice: This e-mail may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error please notify us immediately by email reply and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of TeleGo (T). Employees of TeleGo are expressly required not to make defamatory statements and not to infringe or authorize any infringement of copyright or any other legal right by email communications. Any such communication is contrary to TeleGo policy and outside the scope of the employment of the individual concerned. TeleGo will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising. TeleGo Hosted PBX <https://youtu.be/DaT8YAZ4V0w>
Google rate limit ICMP – a similar problem came up recently on the London Internet Exchange (LINX). It’s probable that if you are seeing packet loss solely on ICMP that you’re hitting a rate limit somewhere. I suggest trying a TCP or UDP mtr to see if the problem is more than ICMP. Phil From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Jesse Curtis via Outages Sent: 05 January 2017 15:12 To: outages@outages.org Subject: [outages] (WI) Packet Loss to Google on Charter I am getting ~50% packet loss to 8.8.8.8, 4.2.2.2, 216.58.216.238, and 172.217.1.46 on two separate Charter (Time Warner) lines in SE Wisconsin. It looks to be affecting just ping requests as DNS and google websites are still working. On a Level 3 line things look normal. ________________________________ Information contained in this email transmission is privileged and confidential. If you are not the intended recipient, do not read, distribute or reproduce this transmission (including any attachments). If you have received this email in error, please notify the sender by email reply. ________________________________
On Thu Jan 05, 2017 at 03:12:10PM +0000, Jesse Curtis via Outages wrote:
I am getting ~50% packet loss to 8.8.8.8, 4.2.2.2, 216.58.216.238, and 172.217.1.46 on two separate Charter (Time Warner) lines in SE Wisconsin. It looks to be affecting just ping requests as DNS and google websites are still working.
Google have publicly stated that they rate limit ICMP to their public services such as 8.8.8.8 - so if you're only seeing ping requests being dropped but all other functionality operating as normal, then this is what you're seeing. Simon
i dont think the issue is google because from other connection(verizon fios etc) it works, with the same retes etc and when i ping TW Default Gateways from our clients, i also have packet loss so it appears to us, the issue is with Time Warner (surprised ?) On Thu, Jan 5, 2017 at 10:17 AM, Simon Lockhart via Outages < outages@outages.org> wrote:
On Thu Jan 05, 2017 at 03:12:10PM +0000, Jesse Curtis via Outages wrote:
I am getting ~50% packet loss to 8.8.8.8, 4.2.2.2, 216.58.216.238, and 172.217.1.46 on two separate Charter (Time Warner) lines in SE Wisconsin. It looks to be affecting just ping requests as DNS and google websites are still working.
Google have publicly stated that they rate limit ICMP to their public services such as 8.8.8.8 - so if you're only seeing ping requests being dropped but all other functionality operating as normal, then this is what you're seeing.
Simon _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Izzy Goldstein Chief Technology Officer Main: (212) 477-1000 x 2085 <(212)%20477-1000> Direct: (929) 477-2085 Website: www.telego.com <http://www.telego.net/> <http://www.telego.com/> Confidentiality Notice: This e-mail may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error please notify us immediately by email reply and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of TeleGo (T). Employees of TeleGo are expressly required not to make defamatory statements and not to infringe or authorize any infringement of copyright or any other legal right by email communications. Any such communication is contrary to TeleGo policy and outside the scope of the employment of the individual concerned. TeleGo will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising. TeleGo Hosted PBX <https://youtu.be/DaT8YAZ4V0w>
The same discussion was had on the LINX mailing list when people stated that packet loss only occurred on LINX and not on a transit provider. Google piped in and said that rate limiting applies on various parts of their network. The route that your lossy traffic is taking is almost certainly different to the route your non-lossy traffic is taking. Try a UDP MTR to port 53 on 8.8.8.8 – this will prove the problem either way. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Izzy Goldstein - TeleGo via Outages Sent: 05 January 2017 15:23 To: Simon Lockhart <simon@slimey.org> Cc: outages@outages.org Subject: Re: [outages] (WI) Packet Loss to Google on Charter i dont think the issue is google because from other connection(verizon fios etc) it works, with the same retes etc and when i ping TW Default Gateways from our clients, i also have packet loss so it appears to us, the issue is with Time Warner (surprised ?) On Thu, Jan 5, 2017 at 10:17 AM, Simon Lockhart via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: On Thu Jan 05, 2017 at 03:12:10PM +0000, Jesse Curtis via Outages wrote:
I am getting ~50% packet loss to 8.8.8.8, 4.2.2.2, 216.58.216.238, and 172.217.1.46 on two separate Charter (Time Warner) lines in SE Wisconsin. It looks to be affecting just ping requests as DNS and google websites are still working.
Google have publicly stated that they rate limit ICMP to their public services such as 8.8.8.8 - so if you're only seeing ping requests being dropped but all other functionality operating as normal, then this is what you're seeing. Simon _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages -- Izzy Goldstein Chief Technology Officer Main: (212) 477-1000 x 2085<tel:(212)%20477-1000> Direct: (929) 477-2085<tel:(929)%20477-2085> Website: www.telego.com<http://www.telego.net/> [https://lh4.googleusercontent.com/Ze7NkT2QtotXfVWbpWXrpSMRIcAdqb0Ci0_RZsAjQzmRrd-_v02-O86QlqwZnGUE8SlBECXBvJUPXSH7ZRyDvDVeaXQFSS5SiA4gan112DN-0s2D3prRdt2UeUprZQEo1YmUFfZ7]<http://www.telego.com/> Confidentiality Notice: This e-mail may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error please notify us immediately by email reply and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of TeleGo (T). Employees of TeleGo are expressly required not to make defamatory statements and not to infringe or authorize any infringement of copyright or any other legal right by email communications. Any such communication is contrary to TeleGo policy and outside the scope of the employment of the individual concerned. TeleGo will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising. TeleGo Hosted PBX<https://youtu.be/DaT8YAZ4V0w>
We are experiencing the same issues here in Maine via Time Warner Cable, anywhere from 20% to 80% loss, started at 6:00AM EST. We use google's DNS as part of our monitoring to determine our internet connection status. Loss seems to start after a tbone.rr.com switch in NYC. 1 2 ms 2 ms 2 ms [Redacted] 2 <1 ms <1 ms <1 ms [Redacted] 3 1 ms 1 ms <1 ms [Redacted] 4 <1 ms <1 ms <1 ms [Redacted] 5 8 ms 8 ms 11 ms agg15.ptldme0201h.northeast.rr.com [24.58.29.48] 6 9 ms 9 ms 9 ms agg34.ptldmehx02r.northeast.rr.com [24.58.42.136] 7 12 ms 19 ms 14 ms be25.albynyyf01r.northeast.rr.com [24.58.32.64] 8 20 ms 23 ms 23 ms bu-ether16.nycmny837aw-bcr00.tbone.rr.com [66.109.6.74] 9 * * * Request timed out. 10 17 ms 17 ms * 72.14.232.109 11 * * 18 ms 108.170.235.81 12 * * * Request timed out. 13 * 18 ms 18 ms google-public-dns-a.google.com [8.8.8.8] ~Jason Robinson On Thu, Jan 5, 2017 at 10:28 AM, Phil Lavin via Outages <outages@outages.org
wrote:
The same discussion was had on the LINX mailing list when people stated that packet loss only occurred on LINX and not on a transit provider. Google piped in and said that rate limiting applies on various parts of their network. The route that your lossy traffic is taking is almost certainly different to the route your non-lossy traffic is taking.
Try a UDP MTR to port 53 on 8.8.8.8 – this will prove the problem either way.
*From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Izzy Goldstein - TeleGo via Outages *Sent:* 05 January 2017 15:23 *To:* Simon Lockhart <simon@slimey.org> *Cc:* outages@outages.org *Subject:* Re: [outages] (WI) Packet Loss to Google on Charter
i dont think the issue is google
because from other connection(verizon fios etc) it works, with the same retes etc
and when i ping TW Default Gateways from our clients, i also have packet loss
so it appears to us, the issue is with Time Warner (surprised ?)
On Thu, Jan 5, 2017 at 10:17 AM, Simon Lockhart via Outages < outages@outages.org> wrote:
On Thu Jan 05, 2017 at 03:12:10PM +0000, Jesse Curtis via Outages wrote:
I am getting ~50% packet loss to 8.8.8.8, 4.2.2.2, 216.58.216.238, and 172.217.1.46 on two separate Charter (Time Warner) lines in SE Wisconsin. It looks to be affecting just ping requests as DNS and google websites are still working.
Google have publicly stated that they rate limit ICMP to their public services such as 8.8.8.8 - so if you're only seeing ping requests being dropped but all other functionality operating as normal, then this is what you're seeing.
Simon
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
--
*Izzy Goldstein *
*Chief Technology Officer *
Main: *(212) 477-1000 x 2085 <(212)%20477-1000>*
Direct: *(929) 477-2085 <(929)%20477-2085>*
Website: www.telego.com <http://www.telego.net/>
*Confidentiality Notice: This e-mail may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error please notify us immediately by email reply and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of TeleGo (T). Employees of TeleGo are expressly required not to make defamatory statements and not to infringe or authorize any infringement of copyright or any other legal right by email communications. Any such communication is contrary to TeleGo policy and outside the scope of the employment of the individual concerned. TeleGo will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising.*
TeleGo Hosted PBX <https://youtu.be/DaT8YAZ4V0w>
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
seems ping to www.foxnews.com / and opendns IP 208.67.222.222 is not loosing any packets but 8.8.8.8 and 8.8.4.4 is all 4 pings are from a TW connection, so i assume issue is with TW and google? [image: Inline image 1] On Thu, Jan 5, 2017 at 10:46 AM, Jason Robinson <jasonbluefire@gmail.com> wrote:
We are experiencing the same issues here in Maine via Time Warner Cable, anywhere from 20% to 80% loss, started at 6:00AM EST.
We use google's DNS as part of our monitoring to determine our internet connection status.
Loss seems to start after a tbone.rr.com switch in NYC.
1 2 ms 2 ms 2 ms [Redacted] 2 <1 ms <1 ms <1 ms [Redacted] 3 1 ms 1 ms <1 ms [Redacted] 4 <1 ms <1 ms <1 ms [Redacted] 5 8 ms 8 ms 11 ms agg15.ptldme0201h.northeast.rr.com [24.58.29.48] 6 9 ms 9 ms 9 ms agg34.ptldmehx02r.northeast.rr.com [24.58.42.136] 7 12 ms 19 ms 14 ms be25.albynyyf01r.northeast.rr.com [24.58.32.64] 8 20 ms 23 ms 23 ms bu-ether16.nycmny837aw-bcr00.tbone.rr.com [66.109.6.74] 9 * * * Request timed out. 10 17 ms 17 ms * 72.14.232.109 11 * * 18 ms 108.170.235.81 12 * * * Request timed out. 13 * 18 ms 18 ms google-public-dns-a.google.com [8.8.8.8]
~Jason Robinson
On Thu, Jan 5, 2017 at 10:28 AM, Phil Lavin via Outages < outages@outages.org> wrote:
The same discussion was had on the LINX mailing list when people stated that packet loss only occurred on LINX and not on a transit provider. Google piped in and said that rate limiting applies on various parts of their network. The route that your lossy traffic is taking is almost certainly different to the route your non-lossy traffic is taking.
Try a UDP MTR to port 53 on 8.8.8.8 – this will prove the problem either way.
*From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Izzy Goldstein - TeleGo via Outages *Sent:* 05 January 2017 15:23 *To:* Simon Lockhart <simon@slimey.org> *Cc:* outages@outages.org *Subject:* Re: [outages] (WI) Packet Loss to Google on Charter
i dont think the issue is google
because from other connection(verizon fios etc) it works, with the same retes etc
and when i ping TW Default Gateways from our clients, i also have packet loss
so it appears to us, the issue is with Time Warner (surprised ?)
On Thu, Jan 5, 2017 at 10:17 AM, Simon Lockhart via Outages < outages@outages.org> wrote:
On Thu Jan 05, 2017 at 03:12:10PM +0000, Jesse Curtis via Outages wrote:
I am getting ~50% packet loss to 8.8.8.8, 4.2.2.2, 216.58.216.238, and 172.217.1.46 on two separate Charter (Time Warner) lines in SE Wisconsin. It looks to be affecting just ping requests as DNS and google websites are still working.
Google have publicly stated that they rate limit ICMP to their public services such as 8.8.8.8 - so if you're only seeing ping requests being dropped but all other functionality operating as normal, then this is what you're seeing.
Simon
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
--
*Izzy Goldstein *
*Chief Technology Officer *
Main: *(212) 477-1000 x 2085 <(212)%20477-1000>*
Direct: *(929) 477-2085 <(929)%20477-2085>*
Website: www.telego.com <http://www.telego.net/>
*Confidentiality Notice: This e-mail may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error please notify us immediately by email reply and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of TeleGo (T). Employees of TeleGo are expressly required not to make defamatory statements and not to infringe or authorize any infringement of copyright or any other legal right by email communications. Any such communication is contrary to TeleGo policy and outside the scope of the employment of the individual concerned. TeleGo will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising.*
TeleGo Hosted PBX <https://youtu.be/DaT8YAZ4V0w>
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Izzy Goldstein Chief Technology Officer Main: (212) 477-1000 x 2085 <(212)%20477-1000> Direct: (929) 477-2085 Website: www.telego.com <http://www.telego.net/> <http://www.telego.com/> Confidentiality Notice: This e-mail may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error please notify us immediately by email reply and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of TeleGo (T). Employees of TeleGo are expressly required not to make defamatory statements and not to infringe or authorize any infringement of copyright or any other legal right by email communications. Any such communication is contrary to TeleGo policy and outside the scope of the employment of the individual concerned. TeleGo will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising. TeleGo Hosted PBX <https://youtu.be/DaT8YAZ4V0w>
participants (5)
-
Izzy Goldstein - TeleGo -
Jason Robinson -
Jesse Curtis -
Phil Lavin -
Simon Lockhart