The circuits bouncing is most likely Comcast doing rolling DOCSIS firmware updates, assuming you all are seeing it on coax circuits. I've seen it as well over in the VA region. Thank you, Braden McGrath IT Systems Engineer Saber Healthcare Group From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Rob Hall via Outages Sent: Friday, May 12, 2017 7:05 AM To: outages@outages.org Subject: Re: [outages] Comcast/Level 3 Issues - Detroit/Chicago Areas -- Now CenturyLink today too? I've been seeing this same behavior. Connection bouncing, sometimes multiple times, in the middle of the night. Had one branch office in IL bounce 3 times night before last. Thanks! -- Rob Hall, CCNP R&S, CCP-V I.T. Manager @Work Personnel Service 3815 Hwy 66 S, Suite 4 Rogersville, TN 37857 Office: (423)272-7897, x10113 Mobile: (423)312-6003 [cisco-certified-ccnp][CCP-V] From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Dave Wagner via Outages Sent: Thursday, May 11, 2017 2:22 PM To: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Comcast/Level 3 Issues - Detroit/Chicago Areas -- Now CenturyLink today too? We have been seeing more issues in the last few weeks with our Comcast Business customers also. Lots of offline alerts at night. Most in the same geographical area and it seems like they are doing different areas each week. No evidence though from them. When we call them there like what are you talking about? David Wagner | Director of Technology Xpert Technologies, Inc. Office | 248.524.0101 x310 Support | 248.524.0101 Opt 2 Support | support@xxpert.com<mailto:support@xxpert.com> Web | xxpert.com <https://linkprotect.cudasvc.com/url?a=http://www.xxpert.com/&c=E,1,o_Nqnliwg1g46dbFIlq64oO9mXdW-STJuk-2S3hR8ZsS_ylXZFEWlsHLy8YQaWmBxjpTJfZ4Vic-FcZg7vG3ddGJPNNeg9c1cS23SYNbfcU,&typo=1> [X_logo_tag (3)] From: Outages [mailto:outages-bounces@outages.org] On Behalf Of B.J. Ash via Outages Sent: Thursday, May 11, 2017 10:07 AM To: Outages Mailing List <outages@outages.org<mailto:outages@outages.org>> Subject: [outages] Comcast/Level 3 Issues - Detroit/Chicago Areas -- Now CenturyLink today too? Does anyone have any information on some ongoing issues with Comcast in the Michigan area since end of April that involve intermittent packet loss, latency spikes (from normal 20 ms to 500+ ms), and just overall cruddy paths? We're seeing issues with 3 of our sites in West/Central Michigan that use Comcast Business for Internet as they route from Comcast to Level 3. Traceroutes seem to die somewhere in Chicago or leaving Detroit either in Level 3 or at the edge of Comcast: Tracing the route to 172.87.34.1 VRF info: (vrf in name/id, vrf out name/id) 1 50-77-204-206-static.hfc.comcastbusiness.net<https://linkprotect.cudasvc.com/url?a=https://50-77-204-206-static.hfc.comcastbusiness.net&c=E,1,MesRXXX-ZpHhLB33CeAxAXaukAm9dMf3Fh0yMfozobuQGQxj71pIOS9sqQQhWWdrwMNdiWO1PJYRrJO1hmyPQlWjZbF1vS0HE5gDskwxNyvFlmTV&typo=1> (50.77.204.206) 52 msec 40 msec 32 msec 2 96.120.42.37 32 msec 44 msec 40 msec 3 xe-7-0-6-sur02.muskegon.mi.michigan.comcast.net (68.85.189.25) 52 msec 52 msec 56 msec 4 xe-1-3-1-0-sur01.lyndon.mi.michigan.comcast.net (68.87.189.177) 60 msec 68 msec 64 msec 5 ae2.bar1.detroit1.level3.net<https://linkprotect.cudasvc.com/url?a=https://ae2.bar1.detroit1.level3.net&c=E,1,VX1Wtis8mOU_HmkSZzDyqRAzH8VxM31xi8BamAQacC92kIgCHjrUZEqIpVd2L20v8HRTRJ4FQDgFH1AEXGVN3um02afezL5hpr3HkDPQU7e1Ft--&typo=1> (4.68.71.33) 72 msec 64 msec 64 msec 6 * * * 7 lynx-networ.ear3.chicago2.level3.net<https://linkprotect.cudasvc.com/url?a=https://lynx-networ.ear3.chicago2.level3.net&c=E,1,mTHzCTVbGRpda9786-dOlc-yLOkIB66HjJCxrLXuEQEdu2xhWIuXr6_SvQCB4kFn1GiRVSEifzVjZsKgz7Xy7RdIeSc6qS_pN_Oi8yk,&typo=1> (4.35.108.250) 136 msec 144 msec 152 msec 8 66.35.87.172.in-addr.arpa.lynxnetworkgroup.com<https://linkprotect.cudasvc.com/url?a=https://66.35.87.172.in-addr.arpa.lynxnetworkgroup.com&c=E,1,p32JgU9qvfSYIqYng6cxDawsSW4AhPwqOOoN6BSHDrYOQgmtkO34kJVoBBdPP7DCxCyUs-coNMm70xgBJg64M6MJK1ZsgA3Njpkmsj_3V4bzAvIB5gHfEbQ,&typo=1> (172.87.34.1) 148 msec * 220 msec Tracing the route to 172.87.34.1 VRF info: (vrf in name/id, vrf out name/id) 1 96-95-79-62-static.hfc.comcastbusiness.net<https://linkprotect.cudasvc.com/url?a=https://96-95-79-62-static.hfc.comcastbusiness.net&c=E,1,sNqVce2uAPhqogMSJ6ICjEZ6oKeaT4J5vdHj7qQAhqiwuH6y8-hjv_LWM1O9zW4ueqVKkOIs8FtVEky5IOTiu0A0aCG86dQWZoRZKTYP6LqIEiG-r968jDU,&typo=1> (96.95.79.62) 0 msec 4 msec 0 msec 2 96.120.28.97 8 msec 8 msec 8 msec 3 be92-sur02.stjoseph.mi.sbend.comcast.net (68.85.204.29) 8 msec 12 msec 12 msec 4 68.85.183.237 12 msec 8 msec 8 msec 5 be-126-ar01.area4.il.chicago.comcast.net (68.87.230.249) 12 msec 16 msec 16 msec 6 * * * 7 * * * 8 LYNX-NETWOR.ear3.Chicago2.Level3.net<https://linkprotect.cudasvc.com/url?a=https://LYNX-NETWOR.ear3.Chicago2.Level3.net&c=E,1,A9YvfSwg2cc4y0prRHCtTjFkNpW74klxPfBui53WumpjBfHJPRyCj3BJFpyYkNhXPH5QvewGFBrsfhMvJXjiL5IwzP3Ezds5BUuaug,,&typo=1> (4.35.108.250) 20 msec 16 msec 16 msec 9 66.35.87.172.in-addr.arpa.lynxnetworkgroup.com<https://linkprotect.cudasvc.com/url?a=https://66.35.87.172.in-addr.arpa.lynxnetworkgroup.com&c=E,1,kXLeVSEgqyqPak2JP8c8bh3tml2ISK-ir5XBPpebVWMaxSSbAGbxEBhR-9kiXnIVq7Kv0KVdk8VpF3o30p3Cn4yScA2NqVelosXMU4Y,&typo=1> (172.87.35.66) 16 msec 28 msec 20 msec 10 * * * 11 * * * Tracing the route to 172.87.34.1 VRF info: (vrf in name/id, vrf out name/id) 1 173-15-238-74-static.hfc.comcastbusiness.net<https://linkprotect.cudasvc.com/url?a=https://173-15-238-74-static.hfc.comcastbusiness.net&c=E,1,O_E4PxXmKYuMcAhMXD6gjUWOFtgCNWVrSL-umAYMAurWyzGEdh30knO6P_wUXHaTZj1_x0r-9eaTjlrtbZiHUw30WoVgpFoQp3Fcp1eV95FGt6lcZHlgyJHX&typo=1> (173.15.238.74) 0 msec 4 msec 4 msec 2 96.120.42.165 16 msec 16 msec 16 msec 3 68.85.49.133 16 msec 12 msec 12 msec 4 xe-0-0-0-0-sur01.stclairshors.mi.michigan.comcast.net (68.87.189.38) 16 msec 12 msec 12 msec 5 te-0-3-0-15-ar02.pontiac.mi.michigan.comcast.net (68.85.235.249) 20 msec 20 msec 16 msec 6 ae2.bar1.detroit1.level3.net<https://linkprotect.cudasvc.com/url?a=https://ae2.bar1.detroit1.level3.net&c=E,1,C1WUZjUdFRQMEUCj_27wiWn8zTmrQDk_Iq60VqJSR5AqCFCRkfCCgEEHZ5TYYyBOlxAkjSrLMZgL_a0PNhiP7KMAL_BNin_r49EaA9OW90Vr&typo=1> (4.68.71.33) 44 msec 20 msec 16 msec 7 * * * 8 lynx-networ.ear3.chicago2.level3.net<https://linkprotect.cudasvc.com/url?a=https://lynx-networ.ear3.chicago2.level3.net&c=E,1,lqBRUMWD4B3q8U8gCPCJQhLWJbT6xXYNMp1JYoaffXZgtHcoyoGyf0epxcudVIcdv6aapj8mrVyRr2W1jJ2yxlW7A-uIgOYWdYWiHLxxGprwRTXFSE7GGA,,&typo=1> (4.35.108.250) 24 msec 32 msec 28 msec 9 66.35.87.172.in-addr.arpa.lynxnetworkgroup.com<https://linkprotect.cudasvc.com/url?a=https://66.35.87.172.in-addr.arpa.lynxnetworkgroup.com&c=E,1,hmmq0QFsFnVzOxps-7iTgl9pVzFXuFsNpp4zqxoEO7wn1oUWmQJzYR7WRPBOXusFsVW2YBJAuyHg8KnGU-w318ioyuXR8a7WB-UqicpeR7p_WBFq5aZ_xpE,&typo=1> (172.87.34.1) 32 msec * 32 msec Now, this morning we're seeing an inability to pass two-way traffic over CenturyLink through Chicago: 1 23ms 23ms 24ms 63.149.1.22 2 * * * 3 23ms 23ms 23ms 63.146.27.18 4 * * * 5 28ms 29ms 29ms 4.35.108.250 6 35ms 28ms 29ms 172.87.34.1 Tracing the route to 63.149.1.22 VRF info: (vrf in name/id, vrf out name/id) 1 172.87.35.66 0 msec 0 msec 0 msec 2 6-2-40.ear3.chicago2.level3.net<https://linkprotect.cudasvc.com/url?a=https://6-2-40.ear3.chicago2.level3.net&c=E,1,bQwKL6_R3ZpzT-p5vodvIDshnb4XzpklrQ7IiX-fmze32vrAHYOkbgg_0ELRriLm13c62OP67INYb1H_4Hu-q2o6ZHUUh54IgvFbHEMpL1k,&typo=1> (4.35.108.249) 34 msec 0 msec 0 msec 3 ae-14-3604.edge4.chicago3.level3.net<https://linkprotect.cudasvc.com/url?a=https://ae-14-3604.edge4.chicago3.level3.net&c=E,1,DJtHkO_-u9lL4VH9EiXgJk2mR-OU-2sAxZ_hjy5LAVBAlQ83gahXNjGR5jsLAjdCG9XffqbSvv4DURg83jKb_f2s-vH67btLLI5YvNfwAMUcYRs1zIk1lFU2&typo=1> (4.69.204.166) 8 msec 9 msec 8 msec 4 chp-brdr-03.inet.qwest.net<https://linkprotect.cudasvc.com/url?a=https://chp-brdr-03.inet.qwest.net&c=E,1,1U4xzmJx3Dnr_UZ7SFMLrpNcjToJu0EbPy4QINiRxdVEYNby6-cqx8lYm0c0dgtWt1aLPQLNuWFtCcD5p4qn0iw2534t314Au8CNWTMuhJ2S&typo=1> (63.146.27.17) 9 msec 8 msec 8 msec 5 cer-edge-15.inet.qwest.net<https://linkprotect.cudasvc.com/url?a=https://cer-edge-15.inet.qwest.net&c=E,1,RCbzcVO0dNWlB3zVOEg357IMxGJyB0ZtY8gVoH3_5J1POuIE54yF2qEE3bkmW_SSCihpHQljVsxfr1yveXKR38vuqefruWFYb5l3JP0ZyaFFNpY-xhE,&typo=1> (205.171.93.26) 9 msec 8 msec 9 msec 6 63.149.1.22 33 msec 34 msec * Comcast appears blind to the issues we've been seeing at their edge. Seems more like a Level 3 issue. What's going on in Chicago lately? -- B.J. Ash bjash@kentrecords.com<mailto:bjash@kentrecords.com> IT Supervisor | Kent Record Management Inc. 1950 Waldorf St. NW, Ste. A | Grand Rapids, MI 49544 W:616-459-6681 | www.kentrecords.com<https://linkprotect.cudasvc.com/url?a=http://www.kentrecords.com/&c=E,1,i-ve63w-IWSVEG5iYFh3J5uOmr1LVSJi2Lz60l-0iRyE-S5x5Db_2xTjfIHFk1mDILTwqJosmdo7IW1Z1794xwzxy1OIXVO312dNPVaUVh22NSmN8MY,&typo=1> Disclaimer: This email and its attachments may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Kent Record Management Inc. If you are not the intended recipient of this email and its attachments, you must take no action based upon them, nor must you copy or show them to anyone. Please contact the sender if you believe you have received this email in error. ________________________________ NOTICE: This email may contain PRIVILEGED and CONFIDENTIAL information and is intended only for the use of the specific individual(s) to which it is addressed. If you are not the intended recipient of this email, you are hereby notified that any unauthorized use, dissemination or copying of this email or the information contained in it or attached to it is strictly prohibited. You may be subject to penalties under law for any improper use or further disclosure of any Protected Health Information in this email. If you have received this email in error, please delete it and immediately notify the sender of this email by reply mail. Thank you. [SHG-PHI-TXT]
