Level 3 LA router failure

Still down as of this morning. I guess it came back online, then went down again. They are waiting on a routing engine. ETR, they say, is 1830GMT. --John On 03/11/2014 10:53 PM, Tri Tran wrote:
Confirmed with Level3 tech. No ETR.
-- Tri Tran
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

We are seeing 20 - 25% packet loss on our Lightower Internet connection in New York City to multiple destinations. This is a legacy Sidera connection. Is anyone else experiencing the same issue? A traceroute shows: 1 2 ms 2 ms 3 ms 69.64.217.17.nyc.electricfiber.net [69.64.217.17] 2 1 ms 1 ms 1 ms 172.16.0.190 3 * * * Request timed out. 4 1 ms 2 ms 1 ms 166.63.159.161 5 2 ms * 2 ms cr1-te-0-7-3-0.newyork.savvis.net [204.70.224.254] 6 * * * Request timed out. 7 * * * Request timed out. 8 2 ms 2 ms 2 ms 206.132.7.195 Thanks. --Vincent

Seeing the same, about 40% packet loss here. Regards, Frank Fleming, Network Engineer (405) 543 1703 Direct (405) 820 4664 Mobile (877) 405 2446 Agio Support Frank.fleming@agio.com<mailto:Frank.fleming@agio.com> Agio | agio.com<https://east.exch021.serverdata.net/owa/redir.aspx?C=2YkGgLaZyESPRYLjmIXg66_KT148Y9AIY4h-KXzi0Jm-mON9ublQfkvtRSInQN7T2rLSpgzVJQU.&URL=http%3a%2f%2fwww.agio.com%2f> From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Vincent Aniello Sent: Wednesday, March 12, 2014 8:58 AM To: outages@outages.org Subject: [outages] Lightower (legacy Sidera) packet loss in New York area We are seeing 20 - 25% packet loss on our Lightower Internet connection in New York City to multiple destinations. This is a legacy Sidera connection. Is anyone else experiencing the same issue? A traceroute shows: 1 2 ms 2 ms 3 ms 69.64.217.17.nyc.electricfiber.net [69.64.217.17] 2 1 ms 1 ms 1 ms 172.16.0.190 3 * * * Request timed out. 4 1 ms 2 ms 1 ms 166.63.159.161 5 2 ms * 2 ms cr1-te-0-7-3-0.newyork.savvis.net [204.70.224.254] 6 * * * Request timed out. 7 * * * Request timed out. 8 2 ms 2 ms 2 ms 206.132.7.195 Thanks. --Vincent The information in this transmittal may be legally privileged, confidential, and/or otherwise protected by law from disclosure, and is intended only for the recipient(s) listed above. If you are neither the intended recipient(s) nor a person responsible for the delivery of this transmittal to the intended recipient(s), you are hereby notified that any distribution or copying of this transmittal is prohibited. If you have received this transmittal in error, please notify Agio LLC immediately at (405) 217-3800 or by return e-mail and take the steps necessary to delete it completely from your computer system. Thank you.

Level3 has sourced a nearby spare routing engine (being borrowed from another router) with no ETA. Original replacement is still en-route via air and will replace the borrowed equipment. On Wed, Mar 12, 2014 at 10:10 AM, John Barbieri <tenpin784@gmail.com> wrote:
Still down as of this morning.
I guess it came back online, then went down again. They are waiting on a routing engine. ETR, they say, is 1830GMT.
--John
On 03/11/2014 10:53 PM, Tri Tran wrote:
Confirmed with Level3 tech. No ETR.
-- Tri Tran
_______________________________________________ Outages mailing listOutages@outages.orghttps://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Devern Adams devernadams@gmail.com

ETA was just added of 14:30 GMT with the routing engine in hand On Wed, Mar 12, 2014 at 11:11 AM, Devern Adams <devernadams@gmail.com>wrote:
Level3 has sourced a nearby spare routing engine (being borrowed from another router) with no ETA. Original replacement is still en-route via air and will replace the borrowed equipment.
On Wed, Mar 12, 2014 at 10:10 AM, John Barbieri <tenpin784@gmail.com>wrote:
Still down as of this morning.
I guess it came back online, then went down again. They are waiting on a routing engine. ETR, they say, is 1830GMT.
--John
On 03/11/2014 10:53 PM, Tri Tran wrote:
Confirmed with Level3 tech. No ETR.
-- Tri Tran
_______________________________________________ Outages mailing listOutages@outages.orghttps://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Devern Adams devernadams@gmail.com
-- Devern Adams devernadams@gmail.com

Sorry for spamming the mailing list repeatedly, but our ticket with Level3 was just updated stating that the replacement is complete and service is stable. Cheers On Wed, Mar 12, 2014 at 11:24 AM, Devern Adams <devernadams@gmail.com>wrote:
ETA was just added of 14:30 GMT with the routing engine in hand
On Wed, Mar 12, 2014 at 11:11 AM, Devern Adams <devernadams@gmail.com>wrote:
Level3 has sourced a nearby spare routing engine (being borrowed from another router) with no ETA. Original replacement is still en-route via air and will replace the borrowed equipment.
On Wed, Mar 12, 2014 at 10:10 AM, John Barbieri <tenpin784@gmail.com>wrote:
Still down as of this morning.
I guess it came back online, then went down again. They are waiting on a routing engine. ETR, they say, is 1830GMT.
--John
On 03/11/2014 10:53 PM, Tri Tran wrote:
Confirmed with Level3 tech. No ETR.
-- Tri Tran
_______________________________________________ Outages mailing listOutages@outages.orghttps://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Devern Adams devernadams@gmail.com
-- Devern Adams devernadams@gmail.com
-- Devern Adams devernadams@gmail.com

As did ours, but three minutes later, it went back down again, or at least, access to our customers whom were affected before. On 03/12/2014 10:17 AM, Devern Adams wrote:
Sorry for spamming the mailing list repeatedly, but our ticket with Level3 was just updated stating that the replacement is complete and service is stable.
Cheers
On Wed, Mar 12, 2014 at 11:24 AM, Devern Adams <devernadams@gmail.com>wrote:
ETA was just added of 14:30 GMT with the routing engine in hand
On Wed, Mar 12, 2014 at 11:11 AM, Devern Adams <devernadams@gmail.com>wrote:
Level3 has sourced a nearby spare routing engine (being borrowed from another router) with no ETA. Original replacement is still en-route via air and will replace the borrowed equipment.
On Wed, Mar 12, 2014 at 10:10 AM, John Barbieri <tenpin784@gmail.com>wrote:
Still down as of this morning.
I guess it came back online, then went down again. They are waiting on a routing engine. ETR, they say, is 1830GMT.
--John
On 03/11/2014 10:53 PM, Tri Tran wrote:
Confirmed with Level3 tech. No ETR.
-- Tri Tran
_______________________________________________ Outages mailing listOutages@outages.orghttps://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Devern Adams devernadams@gmail.com
-- Devern Adams devernadams@gmail.com
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
participants (5)
-
Devern Adams
-
Fleming, Frank
-
John Barbieri
-
Tri Tran
-
Vincent Aniello