I'm seeing consistent packet loss getting to Comcast endpoints in the northeast US. If you can, please forward to someone w/ a clue at Comcast? $ mtr --report-wide 50.232.27.1 ... 3.|-- e1-5.er01.ral01.alwaysdedicated.net 0.0% 10 0.7 0.7 0.6 0.8 0.0 4.|-- 208.79.80.254 0.0% 10 7.0 7.1 7.0 7.2 0.0 5.|-- 246.10.225.104.rdns.hostvirtual.com 0.0% 10 7.0 7.0 6.8 7.2 0.0 6.|-- 1200.ae8.er4.iad10.us.zip.zayo.com 0.0% 10 7.1 7.2 6.9 7.5 0.0 7.|-- ae16.er5.iad10.us.zip.zayo.com 0.0% 10 7.4 7.4 7.3 7.5 0.0 8.|-- ae15.cr2.dca2.us.zip.zayo.com 0.0% 10 8.5 8.5 8.2 9.0 0.0 9.|-- ae1.er2.iad10.us.zip.zayo.com 0.0% 10 9.6 11.3 9.2 29.3 6.3 10.|-- zayo-comcast.iad10.us.zip.zayo.com 0.0% 10 11.1 9.1 8.5 11.1 0.7 11.|-- hu-0-3-0-6-cr02.ashburn.va.ibone.comcast.net 0.0% 10 10.1 10.5 10.1 11.4 0.0 12.|-- be-10102-cr01.newark.nj.ibone.comcast.net 20.0% 10 16.2 15.8 15.3 16.2 0.0 13.|-- be-10203-cr02.newyork.ny.ibone.comcast.net 30.0% 10 16.3 15.8 15.0 16.3 0.0 14.|-- be-7922-ar01.needham.ma.boston.comcast.net 20.0% 10 21.6 21.9 21.3 23.0 0.4 15.|-- te-0-7-0-0-sur02.foxboro.ma.boston.comcast.net 10.0% 10 22.0 21.9 21.8 22.1 0.0 16.|-- 68.85.88.253 30.0% 10 21.9 21.9 21.8 22.0 0.0 17.|-- 50-232-27-1-static.hfc.comcastbusiness.net 30.0% 10 21.6 21.5 21.4 21.7 0.0
Not my department but I get the incident alerts. There is a fiber cut in the area that is being worked on. Jeremy Neumann, Network Engineer, Service Assurance, Enterprise Managed Services Comcast Business Services [cid:image002.png@01D2875C.DFB92700] From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Mark Price via Outages Sent: Wednesday, August 2, 2017 8:43 AM To: outages@outages.org Subject: [outages] Comcast NYC area I'm seeing consistent packet loss getting to Comcast endpoints in the northeast US. If you can, please forward to someone w/ a clue at Comcast? $ mtr --report-wide 50.232.27.1 ... 3.|-- e1-5.er01.ral01.alwaysdedicated.net<http://e1-5.er01.ral01.alwaysdedicated.net> 0.0% 10 0.7 0.7 0.6 0.8 0.0 4.|-- 208.79.80.254 0.0% 10 7.0 7.1 7.0 7.2 0.0 5.|-- 246.10.225.104.rdns.hostvirtual.com<http://246.10.225.104.rdns.hostvirtual.com> 0.0% 10 7.0 7.0 6.8 7.2 0.0 6.|-- 1200.ae8.er4.iad10.us.zip.zayo.com<http://1200.ae8.er4.iad10.us.zip.zayo.com> 0.0% 10 7.1 7.2 6.9 7.5 0.0 7.|-- ae16.er5.iad10.us.zip.zayo.com<http://ae16.er5.iad10.us.zip.zayo.com> 0.0% 10 7.4 7.4 7.3 7.5 0.0 8.|-- ae15.cr2.dca2.us.zip.zayo.com<http://ae15.cr2.dca2.us.zip.zayo.com> 0.0% 10 8.5 8.5 8.2 9.0 0.0 9.|-- ae1.er2.iad10.us.zip.zayo.com<http://ae1.er2.iad10.us.zip.zayo.com> 0.0% 10 9.6 11.3 9.2 29.3 6.3 10.|-- zayo-comcast.iad10.us.zip.zayo.com<http://zayo-comcast.iad10.us.zip.zayo.com> 0.0% 10 11.1 9.1 8.5 11.1 0.7 11.|-- hu-0-3-0-6-cr02.ashburn.va.ibone.comcast.net<http://hu-0-3-0-6-cr02.ashburn.va.ibone.comcast.net> 0.0% 10 10.1 10.5 10.1 11.4 0.0 12.|-- be-10102-cr01.newark.nj.ibone.comcast.net<http://be-10102-cr01.newark.nj.ibone.comcast.net> 20.0% 10 16.2 15.8 15.3 16.2 0.0 13.|-- be-10203-cr02.newyork.ny.ibone.comcast.net<http://be-10203-cr02.newyork.ny.ibone.comcast.net> 30.0% 10 16.3 15.8 15.0 16.3 0.0 14.|-- be-7922-ar01.needham.ma.boston.comcast.net<http://be-7922-ar01.needham.ma.boston.comcast.net> 20.0% 10 21.6 21.9 21.3 23.0 0.4 15.|-- te-0-7-0-0-sur02.foxboro.ma.boston.comcast.net<http://te-0-7-0-0-sur02.foxboro.ma.boston.comcast.net> 10.0% 10 22.0 21.9 21.8 22.1 0.0 16.|-- 68.85.88.253 30.0% 10 21.9 21.9 21.8 22.0 0.0 17.|-- 50-232-27-1-static.hfc.comcastbusiness.net<http://50-232-27-1-static.hfc.comcastbusiness.net> 30.0% 10 21.6 21.5 21.4 21.7 0.0
Just confirming we see this too -Guillermo On 2017-08-02 08:42, Mark Price via Outages wrote:
I'm seeing consistent packet loss getting to Comcast endpoints in the northeast US.
If you can, please forward to someone w/ a clue at Comcast?
$ mtr --report-wide 50.232.27.1 ... 3.|-- e1-5.er01.ral01.alwaysdedicated.net [1] 0.0% 10 0.7 0.7 0.6 0.8 0.0 4.|-- 208.79.80.254 0.0% 10 7.0 7.1 7.0 7.2 0.0 5.|-- 246.10.225.104.rdns.hostvirtual.com [2] 0.0% 10 7.0 7.0 6.8 7.2 0.0 6.|-- 1200.ae8.er4.iad10.us.zip.zayo.com [3] 0.0% 10 7.1 7.2 6.9 7.5 0.0 7.|-- ae16.er5.iad10.us.zip.zayo.com [4] 0.0% 10 7.4 7.4 7.3 7.5 0.0 8.|-- ae15.cr2.dca2.us.zip.zayo.com [5] 0.0% 10 8.5 8.5 8.2 9.0 0.0 9.|-- ae1.er2.iad10.us.zip.zayo.com [6] 0.0% 10 9.6 11.3 9.2 29.3 6.3 10.|-- zayo-comcast.iad10.us.zip.zayo.com [7] 0.0% 10 11.1 9.1 8.5 11.1 0.7 11.|-- hu-0-3-0-6-cr02.ashburn.va.ibone.comcast.net [8] 0.0% 10 10.1 10.5 10.1 11.4 0.0 12.|-- be-10102-cr01.newark.nj.ibone.comcast.net [9] 20.0% 10 16.2 15.8 15.3 16.2 0.0 13.|-- be-10203-cr02.newyork.ny.ibone.comcast.net [10] 30.0% 10 16.3 15.8 15.0 16.3 0.0 14.|-- be-7922-ar01.needham.ma.boston.comcast.net [11] 20.0% 10 21.6 21.9 21.3 23.0 0.4 15.|-- te-0-7-0-0-sur02.foxboro.ma.boston.comcast.net [12] 10.0% 10 22.0 21.9 21.8 22.1 0.0 16.|-- 68.85.88.253 30.0% 10 21.9 21.9 21.8 22.0 0.0 17.|-- 50-232-27-1-static.hfc.comcastbusiness.net [13] 30.0% 10 21.6 21.5 21.4 21.7 0.0
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Links: ------ [1] http://e1-5.er01.ral01.alwaysdedicated.net [2] http://246.10.225.104.rdns.hostvirtual.com [3] http://1200.ae8.er4.iad10.us.zip.zayo.com [4] http://ae16.er5.iad10.us.zip.zayo.com [5] http://ae15.cr2.dca2.us.zip.zayo.com [6] http://ae1.er2.iad10.us.zip.zayo.com [7] http://zayo-comcast.iad10.us.zip.zayo.com [8] http://hu-0-3-0-6-cr02.ashburn.va.ibone.comcast.net [9] http://be-10102-cr01.newark.nj.ibone.comcast.net [10] http://be-10203-cr02.newyork.ny.ibone.comcast.net [11] http://be-7922-ar01.needham.ma.boston.comcast.net [12] http://te-0-7-0-0-sur02.foxboro.ma.boston.comcast.net [13] http://50-232-27-1-static.hfc.comcastbusiness.net
Comcast has stated that this is related to a pole fire which caused damage to their cable. The damage is in Baltimore, MD. I assume related to this: http://baltimore.cbslocal.com/2017/08/02/2-alarm-warehouse-fire/ Currently no ETR - Jeremiah On 2017-08-02 09:33, Guillermo via Outages wrote:
Just confirming we see this too
-Guillermo
On 2017-08-02 08:42, Mark Price via Outages wrote:
I'm seeing consistent packet loss getting to Comcast endpoints in the northeast US.
If you can, please forward to someone w/ a clue at Comcast?
$ mtr --report-wide 50.232.27.1 ... 3.|-- e1-5.er01.ral01.alwaysdedicated.net [1] 0.0% 10 0.7 0.7 0.6 0.8 0.0 4.|-- 208.79.80.254 0.0% 10 7.0 7.1 7.0 7.2 0.0 5.|-- 246.10.225.104.rdns.hostvirtual.com [2] 0.0% 10 7.0 7.0 6.8 7.2 0.0 6.|-- 1200.ae8.er4.iad10.us.zip.zayo.com [3] 0.0% 10 7.1 7.2 6.9 7.5 0.0 7.|-- ae16.er5.iad10.us.zip.zayo.com [4] 0.0% 10 7.4 7.4 7.3 7.5 0.0 8.|-- ae15.cr2.dca2.us.zip.zayo.com [5] 0.0% 10 8.5 8.5 8.2 9.0 0.0
9.|-- ae1.er2.iad10.us.zip.zayo.com [6] 0.0% 10 9.6 11.3 9.2 29.3 6.3 10.|-- zayo-comcast.iad10.us.zip.zayo.com [7] 0.0% 10 11.1 9.1 8.5 11.1 0.7 11.|-- hu-0-3-0-6-cr02.ashburn.va.ibone.comcast.net [8] 0.0% 10 10.1 10.5 10.1 11.4 0.0 12.|-- be-10102-cr01.newark.nj.ibone.comcast.net [9] 20.0% 10 16.2 15.8 15.3 16.2 0.0 13.|-- be-10203-cr02.newyork.ny.ibone.comcast.net [10] 30.0% 10 16.3 15.8 15.0 16.3 0.0 14.|-- be-7922-ar01.needham.ma.boston.comcast.net [11] 20.0% 10 21.6 21.9 21.3 23.0 0.4 15.|-- te-0-7-0-0-sur02.foxboro.ma.boston.comcast.net [12] 10.0% 10 22.0 21.9 21.8 22.1 0.0 16.|-- 68.85.88.253 30.0% 10 21.9 21.9 21.8 22.0 0.0 17.|-- 50-232-27-1-static.hfc.comcastbusiness.net [13] 30.0% 10 21.6 21.5 21.4 21.7 0.0
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages [14]
Links: ------ [1] http://e1-5.er01.ral01.alwaysdedicated.net [2] http://246.10.225.104.rdns.hostvirtual.com [3] http://1200.ae8.er4.iad10.us.zip.zayo.com [4] http://ae16.er5.iad10.us.zip.zayo.com [5] http://ae15.cr2.dca2.us.zip.zayo.com [6] http://ae1.er2.iad10.us.zip.zayo.com [7] http://zayo-comcast.iad10.us.zip.zayo.com [8] http://hu-0-3-0-6-cr02.ashburn.va.ibone.comcast.net [9] http://be-10102-cr01.newark.nj.ibone.comcast.net [10] http://be-10203-cr02.newyork.ny.ibone.comcast.net [11] http://be-7922-ar01.needham.ma.boston.comcast.net [12] http://te-0-7-0-0-sur02.foxboro.ma.boston.comcast.net [13] http://50-232-27-1-static.hfc.comcastbusiness.net [14] https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
I'll take a look and see what's going on. Thanks, John On Wed, Aug 2, 2017 at 6:42 AM, Mark Price via Outages <outages@outages.org> wrote:
I'm seeing consistent packet loss getting to Comcast endpoints in the northeast US.
If you can, please forward to someone w/ a clue at Comcast?
$ mtr --report-wide 50.232.27.1 ... 3.|-- e1-5.er01.ral01.alwaysdedicated.net 0.0% 10 0.7 0.7 0.6 0.8 0.0 4.|-- 208.79.80.254 0.0% 10 7.0 7.1 7.0 7.2 0.0 5.|-- 246.10.225.104.rdns.hostvirtual.com 0.0% 10 7.0 7.0 6.8 7.2 0.0 6.|-- 1200.ae8.er4.iad10.us.zip.zayo.com 0.0% 10 7.1 7.2 6.9 7.5 0.0 7.|-- ae16.er5.iad10.us.zip.zayo.com 0.0% 10 7.4 7.4 7.3 7.5 0.0 8.|-- ae15.cr2.dca2.us.zip.zayo.com 0.0% 10 8.5 8.5 8.2 9.0 0.0 9.|-- ae1.er2.iad10.us.zip.zayo.com 0.0% 10 9.6 11.3 9.2 29.3 6.3 10.|-- zayo-comcast.iad10.us.zip.zayo.com 0.0% 10 11.1 9.1 8.5 11.1 0.7 11.|-- hu-0-3-0-6-cr02.ashburn.va.ibone.comcast.net 0.0% 10 10.1 10.5 10.1 11.4 0.0 12.|-- be-10102-cr01.newark.nj.ibone.comcast.net 20.0% 10 16.2 15.8 15.3 16.2 0.0 13.|-- be-10203-cr02.newyork.ny.ibone.comcast.net 30.0% 10 16.3 15.8 15.0 16.3 0.0 14.|-- be-7922-ar01.needham.ma.boston.comcast.net 20.0% 10 21.6 21.9 21.3 23.0 0.4 15.|-- te-0-7-0-0-sur02.foxboro.ma.boston.comcast.net 10.0% 10 22.0 21.9 21.8 22.1 0.0 16.|-- 68.85.88.253 30.0% 10 21.9 21.9 21.8 22.0 0.0 17.|-- 50-232-27-1-static.hfc.comcastbusiness.net 30.0% 10 21.6 21.5 21.4 21.7 0.0
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Additional remediation was required. Should be good now. Traffic is fully routed around the problem area. Thanks, John On Wed, Aug 2, 2017 at 7:58 AM, John Neiberger <jneiberger@gmail.com> wrote:
I'll take a look and see what's going on.
Thanks, John
On Wed, Aug 2, 2017 at 6:42 AM, Mark Price via Outages < outages@outages.org> wrote:
I'm seeing consistent packet loss getting to Comcast endpoints in the northeast US.
If you can, please forward to someone w/ a clue at Comcast?
$ mtr --report-wide 50.232.27.1 ... 3.|-- e1-5.er01.ral01.alwaysdedicated.net 0.0% 10 0.7 0.7 0.6 0.8 0.0 4.|-- 208.79.80.254 0.0% 10 7.0 7.1 7.0 7.2 0.0 5.|-- 246.10.225.104.rdns.hostvirtual.com 0.0% 10 7.0 7.0 6.8 7.2 0.0 6.|-- 1200.ae8.er4.iad10.us.zip.zayo.com 0.0% 10 7.1 7.2 6.9 7.5 0.0 7.|-- ae16.er5.iad10.us.zip.zayo.com 0.0% 10 7.4 7.4 7.3 7.5 0.0 8.|-- ae15.cr2.dca2.us.zip.zayo.com 0.0% 10 8.5 8.5 8.2 9.0 0.0 9.|-- ae1.er2.iad10.us.zip.zayo.com 0.0% 10 9.6 11.3 9.2 29.3 6.3 10.|-- zayo-comcast.iad10.us.zip.zayo.com 0.0% 10 11.1 9.1 8.5 11.1 0.7 11.|-- hu-0-3-0-6-cr02.ashburn.va.ibone.comcast.net 0.0% 10 10.1 10.5 10.1 11.4 0.0 12.|-- be-10102-cr01.newark.nj.ibone.comcast.net 20.0% 10 16.2 15.8 15.3 16.2 0.0 13.|-- be-10203-cr02.newyork.ny.ibone.comcast.net 30.0% 10 16.3 15.8 15.0 16.3 0.0 14.|-- be-7922-ar01.needham.ma.boston.comcast.net 20.0% 10 21.6 21.9 21.3 23.0 0.4 15.|-- te-0-7-0-0-sur02.foxboro.ma.boston.comcast.net 10.0% 10 22.0 21.9 21.8 22.1 0.0 16.|-- 68.85.88.253 30.0% 10 21.9 21.9 21.8 22.0 0.0 17.|-- 50-232-27-1-static.hfc.comcastbusiness.net 30.0% 10 21.6 21.5 21.4 21.7 0.0
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Thank you John! I see a new path from Ashburn to the Northeast via Marietta, GA and Chicago. All is working well with no loss; confirmed resolved on this side. All 50+ of my sites are back up. Thanks again! - Cary Wiedemann On Wed, Aug 2, 2017 at 10:12 AM, John Neiberger via Outages < outages@outages.org> wrote:
Additional remediation was required. Should be good now. Traffic is fully routed around the problem area.
Thanks, John
On Wed, Aug 2, 2017 at 7:58 AM, John Neiberger <jneiberger@gmail.com> wrote:
I'll take a look and see what's going on.
Thanks, John
On Wed, Aug 2, 2017 at 6:42 AM, Mark Price via Outages < outages@outages.org> wrote:
I'm seeing consistent packet loss getting to Comcast endpoints in the northeast US.
If you can, please forward to someone w/ a clue at Comcast?
$ mtr --report-wide 50.232.27.1 ... 3.|-- e1-5.er01.ral01.alwaysdedicated.net 0.0% 10 0.7 0.7 0.6 0.8 0.0 4.|-- 208.79.80.254 0.0% 10 7.0 7.1 7.0 7.2 0.0 5.|-- 246.10.225.104.rdns.hostvirtual.com 0.0% 10 7.0 7.0 6.8 7.2 0.0 6.|-- 1200.ae8.er4.iad10.us.zip.zayo.com 0.0% 10 7.1 7.2 6.9 7.5 0.0 7.|-- ae16.er5.iad10.us.zip.zayo.com 0.0% 10 7.4 7.4 7.3 7.5 0.0 8.|-- ae15.cr2.dca2.us.zip.zayo.com 0.0% 10 8.5 8.5 8.2 9.0 0.0 9.|-- ae1.er2.iad10.us.zip.zayo.com 0.0% 10 9.6 11.3 9.2 29.3 6.3 10.|-- zayo-comcast.iad10.us.zip.zayo.com 0.0% 10 11.1 9.1 8.5 11.1 0.7 11.|-- hu-0-3-0-6-cr02.ashburn.va.ibone.comcast.net 0.0% 10 10.1 10.5 10.1 11.4 0.0 12.|-- be-10102-cr01.newark.nj.ibone.comcast.net 20.0% 10 16.2 15.8 15.3 16.2 0.0 13.|-- be-10203-cr02.newyork.ny.ibone.comcast.net 30.0% 10 16.3 15.8 15.0 16.3 0.0 14.|-- be-7922-ar01.needham.ma.boston.comcast.net 20.0% 10 21.6 21.9 21.3 23.0 0.4 15.|-- te-0-7-0-0-sur02.foxboro.ma.boston.comcast.net 10.0% 10 22.0 21.9 21.8 22.1 0.0 16.|-- 68.85.88.253 30.0% 10 21.9 21.9 21.8 22.0 0.0 17.|-- 50-232-27-1-static.hfc.comcastbusiness.net 30.0% 10 21.6 21.5 21.4 21.7 0.0
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
On 08/02/2017 06:58 AM, John Neiberger via Outages wrote:
I'll take a look and see what's going on.
Thanks, John
On Wed, Aug 2, 2017 at 6:42 AM, Mark Price via Outages <outages@outages.org <mailto:outages@outages.org>> wrote:
I'm seeing consistent packet loss getting to Comcast endpoints in the northeast US.
Welcome to my world. I see this from Comcast just about any time I run mtr against just about any destination you'd care to name. Retail "high speed Internet" at its ... worst. - John --
I'll take a look. It goes without saying that a fiber cut out east has nothing to do with packet loss in the Seattle regional network, so it's off-topic for this thread. I'll look and see what I can find. That does look like there is some real loss between that AR and CR. John On Wed, Aug 2, 2017 at 8:25 AM, John Sage <jsage@finchhaven.com> wrote:
On 08/02/2017 06:58 AM, John Neiberger via Outages wrote:
I'll take a look and see what's going on.
Thanks, John
On Wed, Aug 2, 2017 at 6:42 AM, Mark Price via Outages <outages@outages.org <mailto:outages@outages.org>> wrote:
I'm seeing consistent packet loss getting to Comcast endpoints in the northeast US.
Welcome to my world. I see this from Comcast just about any time I run mtr against just about any destination you'd care to name.
Retail "high speed Internet" at its ... worst.
- John --
participants (7)
-
Cary Wiedemann -
Guillermo -
Jeremiah Millay -
John Neiberger -
John Sage -
Mark Price -
Neumann, Jeremy