I have a customer that, last friday at 2AM, they lost VPN connectivity to about 6-10 of their sites in North Carolina Raleigh/Charlotte area. The only common thing with these sites is that they are all Time Warner Cable and have ARRIS modems. When we look at our VPN end-points, we only see IPSEC sent and not received. Their head-end has 600 other VPN tunnels on it with no problems, it is just the ones on TWC in that area. Is anyone else having issues or know of an issue there? Thank you, Jeff
TWC had a residential outage in South Charlotte last night, it might be related to your issues? ./r *My email has changed, please update my email to me@rileychilds.me <me@rileychilds.me>* -- Riley Childs House Electrician CPCC Summer Theatre phone: (704) 497-2086 web: www.rileychilds.me On Mon, Aug 1, 2016 at 8:47 AM, Jeffrey Fry via Outages <outages@outages.org
wrote:
I have a customer that, last friday at 2AM, they lost VPN connectivity to about 6-10 of their sites in North Carolina Raleigh/Charlotte area. The only common thing with these sites is that they are all Time Warner Cable and have ARRIS modems.
When we look at our VPN end-points, we only see IPSEC sent and not received. Their head-end has 600 other VPN tunnels on it with no problems, it is just the ones on TWC in that area.
Is anyone else having issues or know of an issue there?
Thank you,
Jeff
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
We experienced something similar with a customer in North Carolina, also on TWC. Traffic from their site via VPN dropped about 6 AM, and the VPN dropped. The tunnel was restored but no traffic could traverse it. Traces were inconclusive but it was apparent that there was an issue with transmit/receive or somewhere a route had changed. Finally routed them to another carrier on our side to resolve the issue. This was a commercial site, I'll check to see if they use ARRIS modems. David From: Outages <outages-bounces@outages.org<mailto:outages-bounces@outages.org>> on behalf of Jeffrey Fry via Outages <outages@outages.org<mailto:outages@outages.org>> Reply-To: Jeffrey Fry <jeff@fryguy.net<mailto:jeff@fryguy.net>> Date: Monday, August 1, 2016 at 4:47 AM To: "outages@outages.org<mailto:outages@outages.org>" <outages@outages.org<mailto:outages@outages.org>> Subject: [outages] North Carolina TWC IPSEC VPN Issues I have a customer that, last friday at 2AM, they lost VPN connectivity to about 6-10 of their sites in North Carolina Raleigh/Charlotte area. The only common thing with these sites is that they are all Time Warner Cable and have ARRIS modems. When we look at our VPN end-points, we only see IPSEC sent and not received. Their head-end has 600 other VPN tunnels on it with no problems, it is just the ones on TWC in that area. Is anyone else having issues or know of an issue there? Thank you, Jeff
Great - that is what I was hoping to hear (but not wanting to hear). Wonder what happened to cause this - perhaps someone from TWC will see this thread. On Mon, Aug 1, 2016 at 9:23 AM, David Palmerton <david@palmertons.com> wrote:
We experienced something similar with a customer in North Carolina, also on TWC.
Traffic from their site via VPN dropped about 6 AM, and the VPN dropped. The tunnel was restored but no traffic could traverse it. Traces were inconclusive but it was apparent that there was an issue with transmit/receive or somewhere a route had changed.
Finally routed them to another carrier on our side to resolve the issue.
This was a commercial site, I’ll check to see if they use ARRIS modems.
David
From: Outages <outages-bounces@outages.org> on behalf of Jeffrey Fry via Outages <outages@outages.org> Reply-To: Jeffrey Fry <jeff@fryguy.net> Date: Monday, August 1, 2016 at 4:47 AM To: "outages@outages.org" <outages@outages.org> Subject: [outages] North Carolina TWC IPSEC VPN Issues
I have a customer that, last friday at 2AM, they lost VPN connectivity to about 6-10 of their sites in North Carolina Raleigh/Charlotte area. The only common thing with these sites is that they are all Time Warner Cable and have ARRIS modems.
When we look at our VPN end-points, we only see IPSEC sent and not received. Their head-end has 600 other VPN tunnels on it with no problems, it is just the ones on TWC in that area.
Is anyone else having issues or know of an issue there?
Thank you,
Jeff
On Aug 1, 2016, at 9:36 AM, Jeffrey Fry via Outages <outages@outages.org> wrote:
Great - that is what I was hoping to hear (but not wanting to hear).
Wonder what happened to cause this - perhaps someone from TWC will see this thread.
FWIW - TWC has a ‘direct support’ forum on dslreports.com <http://dslreports.com/> - I’ve had luck getting prompt responses from clued humans there. -- Brandon Applegate - CCIE 10273 PGP Key fingerprint: 830B 4802 1DD4 F4F9 63FE B966 C0A7 189E 9EC0 3A74 "SH1-0151. This is the serial number, of our orbital gun."
This site is a commercial site. No modems in service. Sent from my iPhone On Aug 1, 2016, at 9:37 AM, Jeffrey Fry <jeff@fryguy.net<mailto:jeff@fryguy.net>> wrote: Great - that is what I was hoping to hear (but not wanting to hear). Wonder what happened to cause this - perhaps someone from TWC will see this thread. On Mon, Aug 1, 2016 at 9:23 AM, David Palmerton <david@palmertons.com<mailto:david@palmertons.com>> wrote: We experienced something similar with a customer in North Carolina, also on TWC. Traffic from their site via VPN dropped about 6 AM, and the VPN dropped. The tunnel was restored but no traffic could traverse it. Traces were inconclusive but it was apparent that there was an issue with transmit/receive or somewhere a route had changed. Finally routed them to another carrier on our side to resolve the issue. This was a commercial site, I'll check to see if they use ARRIS modems. David From: Outages <outages-bounces@outages.org<mailto:outages-bounces@outages.org>> on behalf of Jeffrey Fry via Outages <outages@outages.org<mailto:outages@outages.org>> Reply-To: Jeffrey Fry <jeff@fryguy.net<mailto:jeff@fryguy.net>> Date: Monday, August 1, 2016 at 4:47 AM To: "outages@outages.org<mailto:outages@outages.org>" <outages@outages.org<mailto:outages@outages.org>> Subject: [outages] North Carolina TWC IPSEC VPN Issues I have a customer that, last friday at 2AM, they lost VPN connectivity to about 6-10 of their sites in North Carolina Raleigh/Charlotte area. The only common thing with these sites is that they are all Time Warner Cable and have ARRIS modems. When we look at our VPN end-points, we only see IPSEC sent and not received. Their head-end has 600 other VPN tunnels on it with no problems, it is just the ones on TWC in that area. Is anyone else having issues or know of an issue there? Thank you, Jeff
We're experiencing the same issue with VPN tunnels in North Carolina when using TWC upstream. All services are business class. We're seeing multiple sites affected. We've opened a carrier ticket. Has anyone else gotten an update from them? Thanks. -Raymond Beaudoin On Mon, Aug 1, 2016 at 9:51 AM, David Palmerton via Outages < outages@outages.org> wrote:
This site is a commercial site. No modems in service.
Sent from my iPhone
On Aug 1, 2016, at 9:37 AM, Jeffrey Fry <jeff@fryguy.net> wrote:
Great - that is what I was hoping to hear (but not wanting to hear).
Wonder what happened to cause this - perhaps someone from TWC will see this thread.
On Mon, Aug 1, 2016 at 9:23 AM, David Palmerton <david@palmertons.com> wrote:
We experienced something similar with a customer in North Carolina, also on TWC.
Traffic from their site via VPN dropped about 6 AM, and the VPN dropped. The tunnel was restored but no traffic could traverse it. Traces were inconclusive but it was apparent that there was an issue with transmit/receive or somewhere a route had changed.
Finally routed them to another carrier on our side to resolve the issue.
This was a commercial site, I’ll check to see if they use ARRIS modems.
David
From: Outages <outages-bounces@outages.org> on behalf of Jeffrey Fry via Outages <outages@outages.org> Reply-To: Jeffrey Fry <jeff@fryguy.net> Date: Monday, August 1, 2016 at 4:47 AM To: "outages@outages.org" <outages@outages.org> Subject: [outages] North Carolina TWC IPSEC VPN Issues
I have a customer that, last friday at 2AM, they lost VPN connectivity to about 6-10 of their sites in North Carolina Raleigh/Charlotte area. The only common thing with these sites is that they are all Time Warner Cable and have ARRIS modems.
When we look at our VPN end-points, we only see IPSEC sent and not received. Their head-end has 600 other VPN tunnels on it with no problems, it is just the ones on TWC in that area.
Is anyone else having issues or know of an issue there?
Thank you,
Jeff
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
We have seen the TWC impacted sites self-restore around 11 AM today. Or at least that is when our alerting system noticed them coming back on-line to our Atlanta DC. Just wanted to share. Jeff On Mon, Aug 1, 2016 at 2:24 PM, Raymond Beaudoin < raymond.beaudoin@icarustech.com> wrote:
We're experiencing the same issue with VPN tunnels in North Carolina when using TWC upstream. All services are business class. We're seeing multiple sites affected. We've opened a carrier ticket. Has anyone else gotten an update from them? Thanks.
-Raymond Beaudoin
On Mon, Aug 1, 2016 at 9:51 AM, David Palmerton via Outages < outages@outages.org> wrote:
This site is a commercial site. No modems in service.
Sent from my iPhone
On Aug 1, 2016, at 9:37 AM, Jeffrey Fry <jeff@fryguy.net> wrote:
Great - that is what I was hoping to hear (but not wanting to hear).
Wonder what happened to cause this - perhaps someone from TWC will see this thread.
On Mon, Aug 1, 2016 at 9:23 AM, David Palmerton <david@palmertons.com> wrote:
We experienced something similar with a customer in North Carolina, also on TWC.
Traffic from their site via VPN dropped about 6 AM, and the VPN dropped. The tunnel was restored but no traffic could traverse it. Traces were inconclusive but it was apparent that there was an issue with transmit/receive or somewhere a route had changed.
Finally routed them to another carrier on our side to resolve the issue.
This was a commercial site, I’ll check to see if they use ARRIS modems.
David
From: Outages <outages-bounces@outages.org> on behalf of Jeffrey Fry via Outages <outages@outages.org> Reply-To: Jeffrey Fry <jeff@fryguy.net> Date: Monday, August 1, 2016 at 4:47 AM To: "outages@outages.org" <outages@outages.org> Subject: [outages] North Carolina TWC IPSEC VPN Issues
I have a customer that, last friday at 2AM, they lost VPN connectivity to about 6-10 of their sites in North Carolina Raleigh/Charlotte area. The only common thing with these sites is that they are all Time Warner Cable and have ARRIS modems.
When we look at our VPN end-points, we only see IPSEC sent and not received. Their head-end has 600 other VPN tunnels on it with no problems, it is just the ones on TWC in that area.
Is anyone else having issues or know of an issue there?
Thank you,
Jeff
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Jeff, Thanks for the heads up. We see the same with similar times. -Raymond Beaudoin On Tue, Aug 2, 2016 at 11:58 AM, Jeffrey Fry <jeff@fryguy.net> wrote:
We have seen the TWC impacted sites self-restore around 11 AM today. Or at least that is when our alerting system noticed them coming back on-line to our Atlanta DC.
Just wanted to share.
Jeff
On Mon, Aug 1, 2016 at 2:24 PM, Raymond Beaudoin < raymond.beaudoin@icarustech.com> wrote:
We're experiencing the same issue with VPN tunnels in North Carolina when using TWC upstream. All services are business class. We're seeing multiple sites affected. We've opened a carrier ticket. Has anyone else gotten an update from them? Thanks.
-Raymond Beaudoin
On Mon, Aug 1, 2016 at 9:51 AM, David Palmerton via Outages < outages@outages.org> wrote:
This site is a commercial site. No modems in service.
Sent from my iPhone
On Aug 1, 2016, at 9:37 AM, Jeffrey Fry <jeff@fryguy.net> wrote:
Great - that is what I was hoping to hear (but not wanting to hear).
Wonder what happened to cause this - perhaps someone from TWC will see this thread.
On Mon, Aug 1, 2016 at 9:23 AM, David Palmerton <david@palmertons.com> wrote:
We experienced something similar with a customer in North Carolina, also on TWC.
Traffic from their site via VPN dropped about 6 AM, and the VPN dropped. The tunnel was restored but no traffic could traverse it. Traces were inconclusive but it was apparent that there was an issue with transmit/receive or somewhere a route had changed.
Finally routed them to another carrier on our side to resolve the issue.
This was a commercial site, I’ll check to see if they use ARRIS modems.
David
From: Outages <outages-bounces@outages.org> on behalf of Jeffrey Fry via Outages <outages@outages.org> Reply-To: Jeffrey Fry <jeff@fryguy.net> Date: Monday, August 1, 2016 at 4:47 AM To: "outages@outages.org" <outages@outages.org> Subject: [outages] North Carolina TWC IPSEC VPN Issues
I have a customer that, last friday at 2AM, they lost VPN connectivity to about 6-10 of their sites in North Carolina Raleigh/Charlotte area. The only common thing with these sites is that they are all Time Warner Cable and have ARRIS modems.
When we look at our VPN end-points, we only see IPSEC sent and not received. Their head-end has 600 other VPN tunnels on it with no problems, it is just the ones on TWC in that area.
Is anyone else having issues or know of an issue there?
Thank you,
Jeff
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
participants (5)
-
Brandon Applegate -
David Palmerton -
Jeffrey Fry -
Raymond Beaudoin -
Riley Childs