[347-2944E457-001E] Lumen / Century Link / Legacy Level3

Anyone seeing issues across the legacy Level3 network today? We've been seeing a number of strange issues especially in the northeast and crossing the Atlantic over the past few hours but haven't been able to nail anything down. Sincerely, Charles Henry Total Uptime Technologies, LLC Support Portal: https://support.totaluptime.com +1 800-584-1514 - TEL (US & Canada) +44 330 808 0228 - TEL (United Kingdom) +1 888-584-1194 - FAX Privileged/Confidential Information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), you may not copy or deliver this message to anyone. In such case, you should destroy this message and kindly notify the sender by reply email. Please advise immediately if you or your employer does not consent to email or messages of this kind. Opinions, conclusions and other information in this message that do not relate to the official business of the firm or its subsidiaries or business units shall be understood as neither given nor endorsed by it.

Seeing high latency with packet loss from LA to Russia over level3 From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Total Uptime Netops via Outages Sent: Thursday, December 09, 2021 7:59 AM To: outages@outages.org Subject: [outages] [347-2944E457-001E] Lumen / Century Link / Legacy Level3 Anyone seeing issues across the legacy Level3 network today? We've been seeing a number of strange issues especially in the northeast and crossing the Atlantic over the past few hours but haven't been able to nail anything down. Sincerely, Charles Henry Total Uptime Technologies, LLC Support Portal: https://support.totaluptime.com +1 800-584-1514 - TEL (US & Canada) +44 330 808 0228 - TEL (United Kingdom) +1 888-584-1194 - FAX Privileged/Confidential Information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), you may not copy or deliver this message to anyone. In such case, you should destroy this message and kindly notify the sender by reply email. Please advise immediately if you or your employer does not consent to email or messages of this kind. Opinions, conclusions and other information in this message that do not relate to the official business of the firm or its subsidiaries or business units shall be understood as neither given nor endorsed by it.

We've had numerous users unable to VPN here in PA, all of them routing through Level3 handoffs from Verizon. eApps is also reporting Level3 issues - https://twitter.com/eApps_Status/status/1468976793885491205 --Shane From: Outages <outages-bounces@outages.org> On Behalf Of jd via Outages Sent: Thursday, December 9, 2021 11:30 To: Total Uptime Netops <netops@totaluptime.com> Cc: outages@outages.org Subject: Re: [outages] [347-2944E457-001E] Lumen / Century Link / Legacy Level3 CAUTION! This email originated from outside Clearview. Do not reply, click links, or open attachments unless you are certain the sender is legitimate and the content is safe. Seeing high latency with packet loss from LA to Russia over level3 From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Total Uptime Netops via Outages Sent: Thursday, December 09, 2021 7:59 AM To: outages@outages.org<mailto:outages@outages.org> Subject: [outages] [347-2944E457-001E] Lumen / Century Link / Legacy Level3 Anyone seeing issues across the legacy Level3 network today? We've been seeing a number of strange issues especially in the northeast and crossing the Atlantic over the past few hours but haven't been able to nail anything down. Sincerely, Charles Henry Total Uptime Technologies, LLC Support Portal: https://support.totaluptime.com +1 800-584-1514 - TEL (US & Canada) +44 330 808 0228 - TEL (United Kingdom) +1 888-584-1194 - FAX Privileged/Confidential Information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), you may not copy or deliver this message to anyone. In such case, you should destroy this message and kindly notify the sender by reply email. Please advise immediately if you or your employer does not consent to email or messages of this kind. Opinions, conclusions and other information in this message that do not relate to the official business of the firm or its subsidiaries or business units shall be understood as neither given nor endorsed by it. NOTICE: This message may contain confidential or legally privileged information and is intended solely for the addressee(s). Any review, disclosure, re-transmission or copying of this message or its contents other than by the named recipient(s) is prohibited. If you received this message in error, please reply to inform us of the mistake then delete this communication from your computer system.

We have been receiving multiple reports from users with Verizon FIOS and Verizon Wireless mobile hotspot unable to connect to VPN hosted on legacy Level 3 in PA. Packet captures on both sides confirmed that the packet loss was in the direction of Level 3 to Verizon. The problem was first reported to us around 6:50am EST. At 2:02pm several Verizon VPN users dropped for 5-10 seconds. On Thu, Dec 9, 2021 at 12:02 PM Shane Simmons via Outages < outages@outages.org> wrote:
We’ve had numerous users unable to VPN here in PA, all of them routing through Level3 handoffs from Verizon.
eApps is also reporting Level3 issues - https://twitter.com/eApps_Status/status/1468976793885491205
--Shane
*From:* Outages <outages-bounces@outages.org> *On Behalf Of *jd via Outages *Sent:* Thursday, December 9, 2021 11:30 *To:* Total Uptime Netops <netops@totaluptime.com> *Cc:* outages@outages.org *Subject:* Re: [outages] [347-2944E457-001E] Lumen / Century Link / Legacy Level3
*CAUTION!* This email originated from outside Clearview. Do not reply, click links, or open attachments unless you are certain the sender is legitimate and the content is safe.
Seeing high latency with packet loss from LA to Russia over level3
*From:* Outages [mailto:outages-bounces@outages.org <outages-bounces@outages.org>] *On Behalf Of *Total Uptime Netops via Outages *Sent:* Thursday, December 09, 2021 7:59 AM *To:* outages@outages.org *Subject:* [outages] [347-2944E457-001E] Lumen / Century Link / Legacy Level3
Anyone seeing issues across the legacy Level3 network today? We've been seeing a number of strange issues especially in the northeast and crossing the Atlantic over the past few hours but haven't been able to nail anything down.
Sincerely,
Charles Henry
Total Uptime Technologies, LLC Support Portal: https://support.totaluptime.com
+1 800-584-1514 - TEL (US & Canada) +44 330 808 0228 - TEL (United Kingdom) +1 888-584-1194 - FAX
Privileged/Confidential Information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), you may not copy or deliver this message to anyone. In such case, you should destroy this message and kindly notify the sender by reply email. Please advise immediately if you or your employer does not consent to email or messages of this kind. Opinions, conclusions and other information in this message that do not relate to the official business of the firm or its subsidiaries or business units shall be understood as neither given nor endorsed by it. NOTICE: This message may contain confidential or legally privileged information and is intended solely for the addressee(s). Any review, disclosure, re-transmission or copying of this message or its contents other than by the named recipient(s) is prohibited. If you received this message in error, please reply to inform us of the mistake then delete this communication from your computer system. _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

We have been having various issues passing different types of traffic from hosts outside the Verizon FIOS network to Verizon FIOS IPs through Level3 since about 0130 EST today. We’re located in the Washington, DC area. For a given host pair, sometimes ICMP was blackholed while other traffic types passed fine, and other times UDP was affected instead. Very inconsistent—seemed flow-by-flow. We just (1401 EST) saw a reroute of our level3 traffic from the DC Verizon handoff to Philly. Now everything seems back to normal. Would be nice to know what’s going on behind the scenes at Verizon today. Patrick Shoemaker From: Outages <outages-bounces@outages.org> on behalf of "outages@outages.org" <outages@outages.org> Reply-To: Shane Simmons <SSimmons@clearviewfcu.org> Date: Thursday, December 9, 2021 at 12:01 PM To: "outages@outages.org" <outages@outages.org> Subject: Re: [outages] [347-2944E457-001E] Lumen / Century Link / Legacy Level3 We’ve had numerous users unable to VPN here in PA, all of them routing through Level3 handoffs from Verizon. eApps is also reporting Level3 issues - https://twitter.com/eApps_Status/status/1468976793885491205 --Shane From: Outages <outages-bounces@outages.org> On Behalf Of jd via Outages Sent: Thursday, December 9, 2021 11:30 To: Total Uptime Netops <netops@totaluptime.com> Cc: outages@outages.org Subject: Re: [outages] [347-2944E457-001E] Lumen / Century Link / Legacy Level3 CAUTION! This email originated from outside Clearview. Do not reply, click links, or open attachments unless you are certain the sender is legitimate and the content is safe. Seeing high latency with packet loss from LA to Russia over level3 From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Total Uptime Netops via Outages Sent: Thursday, December 09, 2021 7:59 AM To: outages@outages.org<mailto:outages@outages.org> Subject: [outages] [347-2944E457-001E] Lumen / Century Link / Legacy Level3 Anyone seeing issues across the legacy Level3 network today? We've been seeing a number of strange issues especially in the northeast and crossing the Atlantic over the past few hours but haven't been able to nail anything down. Sincerely, Charles Henry Total Uptime Technologies, LLC Support Portal: https://support.totaluptime.com +1 800-584-1514 - TEL (US & Canada) +44 330 808 0228 - TEL (United Kingdom) +1 888-584-1194 - FAX Privileged/Confidential Information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), you may not copy or deliver this message to anyone. In such case, you should destroy this message and kindly notify the sender by reply email. Please advise immediately if you or your employer does not consent to email or messages of this kind. Opinions, conclusions and other information in this message that do not relate to the official business of the firm or its subsidiaries or business units shall be understood as neither given nor endorsed by it. NOTICE: This message may contain confidential or legally privileged information and is intended solely for the addressee(s). Any review, disclosure, re-transmission or copying of this message or its contents other than by the named recipient(s) is prohibited. If you received this message in error, please reply to inform us of the mistake then delete this communication from your computer system.
participants (5)
-
Bill Sucevic
-
jd
-
Patrick Shoemaker
-
Shane Simmons
-
Total Uptime Netops