We are seeing Level 3 connectivity issues in multiple city's can anyone else confirm
Yes from SLC I'm seeing packet loss when level3 hands off to zayo. Some packet loss through LA too From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Jordan Morris via Outages Sent: Wednesday, April 25, 2018 1:30 PM To: outages@outages.org Subject: [outages] Level3 Connectivity Issues We are seeing Level 3 connectivity issues in multiple city's can anyone else confirm
On Wed, Apr 25, 2018 at 2:29 PM, Jordan Morris via Outages < outages@outages.org> wrote:
We are seeing Level 3 connectivity issues in multiple city’s can anyone else confirm
Hey Jordan, Yeah, I noticed something earlier with IP transit over Level3, looked to be in the Los Angeles, CA area and/or impacting a Level3 / Cox connection, but not sure if it's more widespread than that. I ended up temporarily dropping Cox routes from that transit and everything was fine after that, so my suspicion is that it's a fairly localized event. Take care, Matt
Yes, there is a significant Level 3 network event in Orange County, CA. We're affected via a downstream provider (Cox), unfortunately we do not have access to any additional detail at this time. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Jordan Morris via Outages Sent: Wednesday, April 25, 2018 3:30 PM To: outages@outages.org Subject: [outages] Level3 Connectivity Issues We are seeing Level 3 connectivity issues in multiple city's can anyone else confirm
The ticket we have with Level3 has been frustrating to say the least. They keep blaming it on other peers. First zayo now cox. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Wyn Bryant via Outages Sent: Wednesday, April 25, 2018 2:35 PM To: outages@outages.org Subject: Re: [outages] Level3 Connectivity Issues Yes, there is a significant Level 3 network event in Orange County, CA. We're affected via a downstream provider (Cox), unfortunately we do not have access to any additional detail at this time. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Jordan Morris via Outages Sent: Wednesday, April 25, 2018 3:30 PM To: outages@outages.org Subject: [outages] Level3 Connectivity Issues We are seeing Level 3 connectivity issues in multiple city's can anyone else confirm
So according to Level3, is it a peering issue? On 04/25/2018 12:41 PM, Joseph Jackson via Outages wrote:
The ticket we have with Level3 has been frustrating to say the least. They keep blaming it on other peers. First zayo now cox.
*From:*Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Wyn Bryant via Outages *Sent:* Wednesday, April 25, 2018 2:35 PM *To:* outages@outages.org *Subject:* Re: [outages] Level3 Connectivity Issues
Yes, there is a significant Level 3 network event in Orange County, CA. We’re affected via a downstream provider (Cox), unfortunately we do not have access to any additional detail at this time.
*From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Jordan Morris via Outages *Sent:* Wednesday, April 25, 2018 3:30 PM *To:* outages@outages.org *Subject:* [outages] Level3 Connectivity Issues
We are seeing Level 3 connectivity issues in multiple city’s can anyone else confirm
**
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
I wonder how. We have equipment with ndchost out in CA. They peer with both L3 and Cox. As soon as they dropped L3 everything came back to normal. We also had the exact same issue with them last week. On Wed, Apr 25, 2018 at 3:56 PM, Tim Choe via Outages <outages@outages.org> wrote:
So according to Level3, is it a peering issue?
On 04/25/2018 12:41 PM, Joseph Jackson via Outages wrote:
The ticket we have with Level3 has been frustrating to say the least. They keep blaming it on other peers. First zayo now cox.
*From:* Outages [mailto:outages-bounces@outages.org <outages-bounces@outages.org>] *On Behalf Of *Wyn Bryant via Outages *Sent:* Wednesday, April 25, 2018 2:35 PM *To:* outages@outages.org *Subject:* Re: [outages] Level3 Connectivity Issues
Yes, there is a significant Level 3 network event in Orange County, CA. We’re affected via a downstream provider (Cox), unfortunately we do not have access to any additional detail at this time.
*From:* Outages [mailto:outages-bounces@outages.org <outages-bounces@outages.org>] * On Behalf Of *Jordan Morris via Outages *Sent:* Wednesday, April 25, 2018 3:30 PM *To:* outages@outages.org *Subject:* [outages] Level3 Connectivity Issues
We are seeing Level 3 connectivity issues in multiple city’s can anyone else confirm
_______________________________________________ Outages mailing listOutages@outages.orghttps://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
According to a CL ticket summary sent ~15 minutes ago they've found a misbehaving router in LA and are working on it. ________________________________ From: Outages <outages-bounces@outages.org> on behalf of Dovid Bender via Outages <outages@outages.org> Sent: Wednesday, April 25, 2018 1:35 PM To: Tim Choe Cc: outages@outages.org Subject: Re: [outages] Level3 Connectivity Issues I wonder how. We have equipment with ndchost out in CA. They peer with both L3 and Cox. As soon as they dropped L3 everything came back to normal. We also had the exact same issue with them last week. On Wed, Apr 25, 2018 at 3:56 PM, Tim Choe via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: So according to Level3, is it a peering issue? On 04/25/2018 12:41 PM, Joseph Jackson via Outages wrote: The ticket we have with Level3 has been frustrating to say the least. They keep blaming it on other peers. First zayo now cox. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Wyn Bryant via Outages Sent: Wednesday, April 25, 2018 2:35 PM To: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Level3 Connectivity Issues Yes, there is a significant Level 3 network event in Orange County, CA. We’re affected via a downstream provider (Cox), unfortunately we do not have access to any additional detail at this time. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Jordan Morris via Outages Sent: Wednesday, April 25, 2018 3:30 PM To: outages@outages.org<mailto:outages@outages.org> Subject: [outages] Level3 Connectivity Issues We are seeing Level 3 connectivity issues in multiple city’s can anyone else confirm _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Foutages&data=02%7C01%7Candyt%40zumasys.com%7C37f498a854014cb8b14908d5aaec3017%7Cbfd62fcabd384e07a0785e9943d1f9c0%7C0%7C0%7C636602853767043922&sdata=DiqLjAZyaoHjk2S%2FxV%2BVTl27d3pyAIQpnPwWJEa6YaE%3D&reserved=0> _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Foutages&data=02%7C01%7Candyt%40zumasys.com%7C37f498a854014cb8b14908d5aaec3017%7Cbfd62fcabd384e07a0785e9943d1f9c0%7C0%7C0%7C636602853767043922&sdata=DiqLjAZyaoHjk2S%2FxV%2BVTl27d3pyAIQpnPwWJEa6YaE%3D&reserved=0>
Looks like things are starting to stabilize From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Andy Takacs via Outages Sent: Wednesday, April 25, 2018 1:52 PM To: Outages <outages@outages.org> Subject: Re: [outages] Level3 Connectivity Issues According to a CL ticket summary sent ~15 minutes ago they've found a misbehaving router in LA and are working on it. ________________________________ From: Outages <outages-bounces@outages.org<mailto:outages-bounces@outages.org>> on behalf of Dovid Bender via Outages <outages@outages.org<mailto:outages@outages.org>> Sent: Wednesday, April 25, 2018 1:35 PM To: Tim Choe Cc: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Level3 Connectivity Issues I wonder how. We have equipment with ndchost out in CA. They peer with both L3 and Cox. As soon as they dropped L3 everything came back to normal. We also had the exact same issue with them last week. On Wed, Apr 25, 2018 at 3:56 PM, Tim Choe via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: So according to Level3, is it a peering issue? On 04/25/2018 12:41 PM, Joseph Jackson via Outages wrote: The ticket we have with Level3 has been frustrating to say the least. They keep blaming it on other peers. First zayo now cox. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Wyn Bryant via Outages Sent: Wednesday, April 25, 2018 2:35 PM To: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Level3 Connectivity Issues Yes, there is a significant Level 3 network event in Orange County, CA. We're affected via a downstream provider (Cox), unfortunately we do not have access to any additional detail at this time. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Jordan Morris via Outages Sent: Wednesday, April 25, 2018 3:30 PM To: outages@outages.org<mailto:outages@outages.org> Subject: [outages] Level3 Connectivity Issues We are seeing Level 3 connectivity issues in multiple city's can anyone else confirm _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Foutages&data=02%7C01%7Candyt%40zumasys.com%7C37f498a854014cb8b14908d5aaec3017%7Cbfd62fcabd384e07a0785e9943d1f9c0%7C0%7C0%7C636602853767043922&sdata=DiqLjAZyaoHjk2S%2FxV%2BVTl27d3pyAIQpnPwWJEa6YaE%3D&reserved=0> _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Foutages&data=02%7C01%7Candyt%40zumasys.com%7C37f498a854014cb8b14908d5aaec3017%7Cbfd62fcabd384e07a0785e9943d1f9c0%7C0%7C0%7C636602853767043922&sdata=DiqLjAZyaoHjk2S%2FxV%2BVTl27d3pyAIQpnPwWJEa6YaE%3D&reserved=0>
We are having issues with Level3 in Long Beach CA near LA. One of these days Level3 is going to get their act together..... *Jesse Nowlin* IT Manager P: 310-579-6665 jesse.n@westlandreg.com Westland Real Estate Group 520 W. Willow St., Long Beach, CA 90806 www.westlandreg.com On Wed, Apr 25, 2018 at 12:30 PM Jordan Morris via Outages < outages@outages.org> wrote:
We are seeing Level 3 connectivity issues in multiple city’s can anyone else confirm
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
There is definitely something going on in southern California, packet loss and latency. No official information yet. There was also an issue in the Raleigh NC area that appears to be ongoing. - Mike Bolitho On Wed, Apr 25, 2018 at 12:29 PM, Jordan Morris via Outages < outages@outages.org> wrote:
We are seeing Level 3 connectivity issues in multiple city’s can anyone else confirm
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Looks like it runs all the way to Vegas on Cox for us. *Jesse Nowlin* IT Manager P: 310-579-6665 jesse.n@westlandreg.com Westland Real Estate Group 520 W. Willow St., Long Beach, CA 90806 www.westlandreg.com On Wed, Apr 25, 2018 at 12:42 PM Mike Bolitho via Outages < outages@outages.org> wrote:
There is definitely something going on in southern California, packet loss and latency. No official information yet. There was also an issue in the Raleigh NC area that appears to be ongoing.
- Mike Bolitho
On Wed, Apr 25, 2018 at 12:29 PM, Jordan Morris via Outages < outages@outages.org> wrote:
We are seeing Level 3 connectivity issues in multiple city’s can anyone else confirm
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Thanks! -Mike -- Michael T. Voity Network Engineer University of Vermont (802)656-8112 [/Users/mvoity/Library/Containers/com.microsoft.Outlook/Data/Library/Caches/Signatures/signature_1816602405]<https://communities.cisco.com/groups/cisco-champions> Cisco Champions are passionate about Cisco and happy to share what we know. From: Outages <outages-bounces@outages.org> on behalf of Jesse Nowlin via Outages <outages@outages.org> Reply-To: Jesse Nowlin <jesse.n@westlandreg.com> Date: Wednesday, April 25, 2018 at 15:52 To: "mikebolitho@gmail.com" <mikebolitho@gmail.com> Cc: "outages@outages.org" <outages@outages.org> Subject: Re: [outages] Level3 Connectivity Issues Looks like it runs all the way to Vegas on Cox for us. [http://westlandrealestategroup.com/images/CORPLogoEmailSig150px.jpg] Jesse Nowlin IT Manager P: 310-579-6665 jesse.n@westlandreg.com Westland Real Estate Group 520 W. Willow St., Long Beach, CA 90806 www.westlandreg.com On Wed, Apr 25, 2018 at 12:42 PM Mike Bolitho via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: There is definitely something going on in southern California, packet loss and latency. No official information yet. There was also an issue in the Raleigh NC area that appears to be ongoing. - Mike Bolitho On Wed, Apr 25, 2018 at 12:29 PM, Jordan Morris via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: We are seeing Level 3 connectivity issues in multiple city’s can anyone else confirm _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages
Seeing lots of packet loss of level3 too. Looks internal to them, even routing away if they are in other downstream routes probably can’t avoid it.
On Apr 25, 2018, at 12:51 PM, Jesse Nowlin via Outages <outages@outages.org> wrote:
Looks like it runs all the way to Vegas on Cox for us.
Jesse Nowlin IT Manager P: 310-579-6665 jesse.n@westlandreg.com
Westland Real Estate Group 520 W. Willow St., Long Beach, CA 90806 www.westlandreg.com
On Wed, Apr 25, 2018 at 12:42 PM Mike Bolitho via Outages <outages@outages.org> wrote: There is definitely something going on in southern California, packet loss and latency. No official information yet. There was also an issue in the Raleigh NC area that appears to be ongoing.
- Mike Bolitho
On Wed, Apr 25, 2018 at 12:29 PM, Jordan Morris via Outages <outages@outages.org> wrote: We are seeing Level 3 connectivity issues in multiple city’s can anyone else confirm
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
We are routing around Level3 were possible currently to correct this issue for our users. Seem mostly on West coast impact at this point. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Logan Broitman via Outages Sent: Wednesday, April 25, 2018 1:10 PM To: outages@outages.org Subject: Re: [outages] Level3 Connectivity Issues Seeing lots of packet loss of level3 too. Looks internal to them, even routing away if they are in other downstream routes probably can’t avoid it. On Apr 25, 2018, at 12:51 PM, Jesse Nowlin via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: Looks like it runs all the way to Vegas on Cox for us. [http://westlandrealestategroup.com/images/CORPLogoEmailSig150px.jpg] Jesse Nowlin IT Manager P: 310-579-6665 jesse.n@westlandreg.com<mailto:jesse.n@westlandreg.com> Westland Real Estate Group 520 W. Willow St., Long Beach, CA 90806 www.westlandreg.com<http://www.westlandreg.com> On Wed, Apr 25, 2018 at 12:42 PM Mike Bolitho via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: There is definitely something going on in southern California, packet loss and latency. No official information yet. There was also an issue in the Raleigh NC area that appears to be ongoing. - Mike Bolitho On Wed, Apr 25, 2018 at 12:29 PM, Jordan Morris via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: We are seeing Level 3 connectivity issues in multiple city’s can anyone else confirm _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages
participants (12)
-
Andy Takacs -
Dovid Bender -
Jesse Nowlin -
Jordan Morris -
Joseph Jackson -
Logan Broitman -
Matt Harris -
Michael T. Voity -
Mike Bolitho -
Nathan Harward -
Tim Choe -
Wyn Bryant