Comcast/Level3 Transport Issues?
seeing multiple customers reporting substantial packet loss 10.|-- lag-12.ear2.Washington12. 90.0% 10 7309. 7309. 7309. 7309. 0.0 11.|-- ae-3-3.bar1.Philadelphia1 30.0% 10 122.5 126.6 121.6 134.3 4.4 12.|-- Comcast-level3-50G.Philad 60.0% 10 130.6 131.3 129.2 133.8 1.7 13.|-- be-900-ar03.norristown.pa 80.0% 10 130.2 130.3 130.2 130.3 0.0 14.|-- ae-1-0-sur01.levittown.pa 40.0% 10 126.0 129.7 126.0 135.3 3.5 15.|-- te-7-0-acr01.levittown.pa 70.0% 10 125.5 127.8 125.5 130.6 2.5 16.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 another example 10.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 11.|-- ae-3-3.bar1.Philadelphia1 50.0% 10 120.8 125.5 120.8 133.9 5.1 12.|-- Comcast-level3-50G.Philad 50.0% 10 136.5 131.8 126.4 137.2 4.8 13.|-- be-100-ar03.plainfield.nj 60.0% 10 129.3 132.5 127.8 137.2 4.6 14.|-- 68.85.62.18 60.0% 10 122.1 130.0 122.1 135.2 5.7 15.|-- te-6-0-acr02.jerseycity.n 30.0% 10 129.0 131.1 118.1 139.7 7.2 16.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 just got confirmation from one comcast contact that this may be related to known event going on starting about 30 mins ago. curious to see if anyone else is seeing this or any other technical info chris
looks a little stormy in the cloud http://downdetector.com/status/comcast-xfinity/map/ On Mon, Nov 6, 2017 at 1:39 PM, chris <tknchris@gmail.com> wrote:
seeing multiple customers reporting substantial packet loss
10.|-- lag-12.ear2.Washington12. 90.0% 10 7309. 7309. 7309. 7309. 0.0 11.|-- ae-3-3.bar1.Philadelphia1 30.0% 10 122.5 126.6 121.6 134.3 4.4 12.|-- Comcast-level3-50G.Philad 60.0% 10 130.6 131.3 129.2 133.8 1.7 13.|-- be-900-ar03.norristown.pa 80.0% 10 130.2 130.3 130.2 130.3 0.0 14.|-- ae-1-0-sur01.levittown.pa 40.0% 10 126.0 129.7 126.0 135.3 3.5 15.|-- te-7-0-acr01.levittown.pa 70.0% 10 125.5 127.8 125.5 130.6 2.5 16.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
another example
10.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 11.|-- ae-3-3.bar1.Philadelphia1 50.0% 10 120.8 125.5 120.8 133.9 5.1 12.|-- Comcast-level3-50G.Philad 50.0% 10 136.5 131.8 126.4 137.2 4.8 13.|-- be-100-ar03.plainfield.nj 60.0% 10 129.3 132.5 127.8 137.2 4.6 14.|-- 68.85.62.18 60.0% 10 122.1 130.0 122.1 135.2 5.7 15.|-- te-6-0-acr02.jerseycity.n 30.0% 10 129.0 131.1 118.1 139.7 7.2 16.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
just got confirmation from one comcast contact that this may be related to known event going on starting about 30 mins ago.
curious to see if anyone else is seeing this or any other technical info
chris
On 11/06/2017 10:40 AM, chris via Outages wrote:
looks a little stormy in the cloud http://downdetector.com/status/comcast-xfinity/map/
ah... the irony. Or something. - John
Yes, lots of people having this problem right now. It appears to be an issue with Level 3, but we're trying to gather data and confirm. Whatever is happening is widespread. On Mon, Nov 6, 2017 at 11:39 AM, chris via Outages <outages@outages.org> wrote:
seeing multiple customers reporting substantial packet loss
10.|-- lag-12.ear2.Washington12. 90.0% 10 7309. 7309. 7309. 7309. 0.0 11.|-- ae-3-3.bar1.Philadelphia1 30.0% 10 122.5 126.6 121.6 134.3 4.4 12.|-- Comcast-level3-50G.Philad 60.0% 10 130.6 131.3 129.2 133.8 1.7 13.|-- be-900-ar03.norristown.pa 80.0% 10 130.2 130.3 130.2 130.3 0.0 14.|-- ae-1-0-sur01.levittown.pa 40.0% 10 126.0 129.7 126.0 135.3 3.5 15.|-- te-7-0-acr01.levittown.pa 70.0% 10 125.5 127.8 125.5 130.6 2.5 16.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
another example
10.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 11.|-- ae-3-3.bar1.Philadelphia1 50.0% 10 120.8 125.5 120.8 133.9 5.1 12.|-- Comcast-level3-50G.Philad 50.0% 10 136.5 131.8 126.4 137.2 4.8 13.|-- be-100-ar03.plainfield.nj 60.0% 10 129.3 132.5 127.8 137.2 4.6 14.|-- 68.85.62.18 60.0% 10 122.1 130.0 122.1 135.2 5.7 15.|-- te-6-0-acr02.jerseycity.n 30.0% 10 129.0 131.1 118.1 139.7 7.2 16.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
just got confirmation from one comcast contact that this may be related to known event going on starting about 30 mins ago.
curious to see if anyone else is seeing this or any other technical info
chris
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Does seem related to Level3. Peerings down? From Seattle, it's quite scenic to get back to Seattle: Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. internal.e4.router.seattle.nfoservers.com 0.5% 1690 0.0 0.1 0.0 14.1 0.6 2. border8.te7-2.nuclearfallout-55.sef.pnap.net 0.0% 1690 0.0 0.4 0.0 30.0 2.0 3. core1.t6-2-bbnet2.sea.pnap.net 0.0% 1690 1.8 0.8 0.0 20.8 0.9 4. ae-18.a00.sttlwa01.us.bb.gin.ntt.net 0.3% 1690 1.4 1.8 0.0 45.3 3.3 5. ae-14.r05.sttlwa01.us.bb.gin.ntt.net 0.0% 1690 2.0 0.6 0.0 14.1 0.7 6. ae-1.r21.sttlwa01.us.bb.gin.ntt.net 0.1% 1690 1.1 3.1 0.0 88.2 8.2 7. ae-3.r23.snjsca04.us.bb.gin.ntt.net 0.1% 1690 18.7 18.3 16.3 215.6 5.3 8. ae-0.r22.snjsca04.us.bb.gin.ntt.net 1.2% 1690 18.6 17.9 16.4 36.0 1.8 9. ae-7.r23.asbnva02.us.bb.gin.ntt.net 2.1% 1690 80.0 80.5 78.0 114.6 2.8 10. ae-2.r05.asbnva02.us.bb.gin.ntt.net 0.0% 1689 80.3 81.8 78.9 97.3 3.1 11. ae-0.a02.asbnva02.us.bb.gin.ntt.net 2.5% 1689 84.3 85.8 81.7 128.6 5.1 12. lag-12.ear2.Washington12.Level3.net 99.1% 1689 6992. 7236. 6992. 7729. 199.2 13. ae-2-3609.ear1.Seattle1.Level3.net 89.3% 1689 200.0 200.7 183.6 346.4 16.4 14. 4.68.71.78 44.6% 1689 186.2 195.6 179.8 209.2 4.6 15. po-1-rur101.seattle.wa.seattle.comcast.net 41.6% 1689 190.6 195.6 179.1 206.5 4.4 16. 68.86.177.242 44.2% 1689 187.0 195.5 179.0 212.9 4.6 -John -John On 11/6/2017 10:50 AM, John Neiberger via Outages wrote:
Yes, lots of people having this problem right now. It appears to be an issue with Level 3, but we're trying to gather data and confirm. Whatever is happening is widespread.
On Mon, Nov 6, 2017 at 11:39 AM, chris via Outages <outages@outages.org <mailto:outages@outages.org>> wrote:
seeing multiple customers reporting substantial packet loss
10.|-- lag-12.ear2.Washington12. 90.0% 10 7309. 7309. 7309. 7309. 0.0 11.|-- ae-3-3.bar1.Philadelphia1 30.0% 10 122.5 126.6 121.6 134.3 4.4 12.|-- Comcast-level3-50G.Philad 60.0% 10 130.6 131.3 129.2 133.8 1.7 13.|-- be-900-ar03.norristown.pa <http://be-900-ar03.norristown.pa> 80.0% 10 130.2 130.3 130.2 130.3 0.0 14.|-- ae-1-0-sur01.levittown.pa <http://ae-1-0-sur01.levittown.pa> 40.0% 10 126.0 129.7 126.0 135.3 3.5 15.|-- te-7-0-acr01.levittown.pa <http://te-7-0-acr01.levittown.pa> 70.0% 10 125.5 127.8 125.5 130.6 2.5 16.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
another example
10.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 11.|-- ae-3-3.bar1.Philadelphia1 50.0% 10 120.8 125.5 120.8 133.9 5.1 12.|-- Comcast-level3-50G.Philad 50.0% 10 136.5 131.8 126.4 137.2 4.8 13.|-- be-100-ar03.plainfield.nj 60.0% 10 129.3 132.5 127.8 137.2 4.6 14.|-- 68.85.62.18 60.0% 10 122.1 130.0 122.1 135.2 5.7 15.|-- te-6-0-acr02.jerseycity.n 30.0% 10 129.0 131.1 118.1 139.7 7.2 16.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
just got confirmation from one comcast contact that this may be related to known event going on starting about 30 mins ago.
curious to see if anyone else is seeing this or any other technical info
chris
_______________________________________________ Outages mailing list Outages@outages.org <mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages <https://puck.nether.net/mailman/listinfo/outages>
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
As of a minute ago, this appears resolved in the Everett, WA, US (Seattle-ish) area. ~Nate
Hi all, Speculating... I posted this in the other thread too: https://puck.nether.net/pipermail/outages/2017-November/010947.html It appears that a bunch of more-specifics belonging to regional comcast networks are now visible in the default-free zone. This path (which I assume was not designed to carry all the world's comcast traffic) probably shouldn't be globally visible. https://stat.ripe.net/174.54.0.0%2F15#tabId=routing (normally the DFZ would only see a /12 instead of this /15) Kind regards, Job
participants (6)
-
chris -
Job Snijders -
John -
John Neiberger -
John Sage -
Nate Schmoll