Re: [outages] Level 3 issues in Ashburn
Also affects our Hibernia Atlantic transport to NYC. I've received the following update at 13:22 EST: "Please be advised that Hibernia Networks is currently experiencing a network outage between Ashburn and Pittsburgh, USA since 16:40 GMT. We are currently investigating the issue. Update1 - Please be informed that, our partner is currently investigating the issue and has dispatched techs to site, there is currently no ETA available, further updates to follow." Eric Dugas edugas@zerofail.com Network Management Group ZEROFAIL / AS40191
Ouch 1/16/14 8:00:49 PM GMTField Services are on site performing excavation of the impacted fiber, which is fairly deep. When the boring crew hit the fiber, approximately 25 feet of fiber was pulled out and will need to be repaired. Restoration efforts are underway, but no estimated restoral time is currently available. On Thu, Jan 16, 2014 at 11:12 AM, Eric Dugas <EDugas@zerofail.com> wrote:
Also affects our Hibernia Atlantic transport to NYC. I've received the following update at 13:22 EST:
"Please be advised that Hibernia Networks is currently experiencing a network outage between Ashburn and Pittsburgh, USA since 16:40 GMT. We are currently investigating the issue.
Update1 - Please be informed that, our partner is currently investigating the issue and has dispatched techs to site, there is currently no ETA available, further updates to follow."
Eric Dugas edugas@zerofail.com Network Management Group ZEROFAIL / AS40191
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
1/16/14 9:05:48 PM GMT The Transport NOC advised that Field Services are currently working to replace the damaged conduit so that the replacement fiber can be placed in. An estimated start time for splicing is unavailable at this time. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Luke Rockwell Sent: Thursday, January 16, 2014 3:07 PM To: outages@outages.org Subject: Re: [outages] Level 3 issues in Ashburn Ouch 1/16/14 8:00:49 PM GMT Field Services are on site performing excavation of the impacted fiber, which is fairly deep. When the boring crew hit the fiber, approximately 25 feet of fiber was pulled out and will need to be repaired. Restoration efforts are underway, but no estimated restoral time is currently available. On Thu, Jan 16, 2014 at 11:12 AM, Eric Dugas <EDugas@zerofail.com<mailto:EDugas@zerofail.com>> wrote: Also affects our Hibernia Atlantic transport to NYC. I've received the following update at 13:22 EST: "Please be advised that Hibernia Networks is currently experiencing a network outage between Ashburn and Pittsburgh, USA since 16:40 GMT. We are currently investigating the issue. Update1 - Please be informed that, our partner is currently investigating the issue and has dispatched techs to site, there is currently no ETA available, further updates to follow." Eric Dugas edugas@zerofail.com<mailto:edugas@zerofail.com> Network Management Group ZEROFAIL / AS40191 _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages
1/16/14 9:44:58 PM GMTThe Transport NOC informed that Field Services are anticipating splicing preparations will begin at approximately 22:00 GMT. It has been informed that this process will take approximately two hours to complete, and splicing will commence thereafter. On Thu, Jan 16, 2014 at 1:29 PM, Tim Glen <tglen@asicentral.com> wrote:
1/16/14 9:05:48 PM GMT
The Transport NOC advised that Field Services are currently working to replace the damaged conduit so that the replacement fiber can be placed in. An estimated start time for splicing is unavailable at this time.
*From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Luke Rockwell *Sent:* Thursday, January 16, 2014 3:07 PM
*To:* outages@outages.org *Subject:* Re: [outages] Level 3 issues in Ashburn
Ouch
1/16/14 8:00:49 PM GMT
Field Services are on site performing excavation of the impacted fiber, which is fairly deep. When the boring crew hit the fiber, approximately 25 feet of fiber was pulled out and will need to be repaired. Restoration efforts are underway, but no estimated restoral time is currently available.
On Thu, Jan 16, 2014 at 11:12 AM, Eric Dugas <EDugas@zerofail.com> wrote:
Also affects our Hibernia Atlantic transport to NYC. I've received the following update at 13:22 EST:
"Please be advised that Hibernia Networks is currently experiencing a network outage between Ashburn and Pittsburgh, USA since 16:40 GMT. We are currently investigating the issue.
Update1 - Please be informed that, our partner is currently investigating the issue and has dispatched techs to site, there is currently no ETA available, further updates to follow."
Eric Dugas edugas@zerofail.com Network Management Group ZEROFAIL / AS40191
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
1/16/14 10:52:27 PM GMT (5:52:27 PM EST) The Transport NOC informed that Field Services are continuing their splicing preparations, which are expected to be completed by 00:00 GMT (7:00 PM EST) and splicing will begin subsequently. _____ From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Luke Rockwell Sent: Thursday, January 16, 2014 4:46 PM To: outages@outages.org Subject: Re: [outages] Level 3 issues in Ashburn 1/16/14 9:44:58 PM GMT The Transport NOC informed that Field Services are anticipating splicing preparations will begin at approximately 22:00 GMT. It has been informed that this process will take approximately two hours to complete, and splicing will commence thereafter. On Thu, Jan 16, 2014 at 1:29 PM, Tim Glen <tglen@asicentral.com> wrote: 1/16/14 9:05:48 PM GMT The Transport NOC advised that Field Services are currently working to replace the damaged conduit so that the replacement fiber can be placed in. An estimated start time for splicing is unavailable at this time. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Luke Rockwell Sent: Thursday, January 16, 2014 3:07 PM To: outages@outages.org Subject: Re: [outages] Level 3 issues in Ashburn Ouch 1/16/14 8:00:49 PM GMT Field Services are on site performing excavation of the impacted fiber, which is fairly deep. When the boring crew hit the fiber, approximately 25 feet of fiber was pulled out and will need to be repaired. Restoration efforts are underway, but no estimated restoral time is currently available. On Thu, Jan 16, 2014 at 11:12 AM, Eric Dugas <EDugas@zerofail.com> wrote: Also affects our Hibernia Atlantic transport to NYC. I've received the following update at 13:22 EST: "Please be advised that Hibernia Networks is currently experiencing a network outage between Ashburn and Pittsburgh, USA since 16:40 GMT. We are currently investigating the issue. Update1 - Please be informed that, our partner is currently investigating the issue and has dispatched techs to site, there is currently no ETA available, further updates to follow." Eric Dugas edugas@zerofail.com Network Management Group ZEROFAIL / AS40191 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
1/16/14 11:41:42 PM GMTField Services have completed preparation work for splicing on one end, but the other end is still being prepped as unforeseen issues caused delays. It has been informed that splicing is expected to begin at 00:30 GMT. On Thu, Jan 16, 2014 at 3:31 PM, Tim Glen <tglen@asicentral.com> wrote:
1/16/14 10:52:27 PM GMT (5:52:27 PM EST)
The Transport NOC informed that Field Services are continuing their splicing preparations, which are expected to be completed by 00:00 GMT (7:00 PM EST) and splicing will begin subsequently.
------------------------------ *From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Luke Rockwell *Sent:* Thursday, January 16, 2014 4:46 PM
*To:* outages@outages.org *Subject:* Re: [outages] Level 3 issues in Ashburn
1/16/14 9:44:58 PM GMT The Transport NOC informed that Field Services are anticipating splicing preparations will begin at approximately 22:00 GMT. It has been informed that this process will take approximately two hours to complete, and splicing will commence thereafter.
On Thu, Jan 16, 2014 at 1:29 PM, Tim Glen <tglen@asicentral.com> wrote:
1/16/14 9:05:48 PM GMT
The Transport NOC advised that Field Services are currently working to replace the damaged conduit so that the replacement fiber can be placed in. An estimated start time for splicing is unavailable at this time.
*From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Luke Rockwell *Sent:* Thursday, January 16, 2014 3:07 PM
*To:* outages@outages.org *Subject:* Re: [outages] Level 3 issues in Ashburn
Ouch
1/16/14 8:00:49 PM GMT
Field Services are on site performing excavation of the impacted fiber, which is fairly deep. When the boring crew hit the fiber, approximately 25 feet of fiber was pulled out and will need to be repaired. Restoration efforts are underway, but no estimated restoral time is currently available.
On Thu, Jan 16, 2014 at 11:12 AM, Eric Dugas <EDugas@zerofail.com> wrote:
Also affects our Hibernia Atlantic transport to NYC. I've received the following update at 13:22 EST:
"Please be advised that Hibernia Networks is currently experiencing a network outage between Ashburn and Pittsburgh, USA since 16:40 GMT. We are currently investigating the issue.
Update1 - Please be informed that, our partner is currently investigating the issue and has dispatched techs to site, there is currently no ETA available, further updates to follow."
Eric Dugas edugas@zerofail.com Network Management Group ZEROFAIL / AS40191
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
1/17/14 1:19:44 AM GMTSplicing has commenced on the damaged fiber and as splicing progresses, alarms are beginning to clear. It has been prevised that this splice work will be extensive and an estimated restoral time of 05:00 GMT has been provided. ..................................... *Luke Rockwell* Systems and Support Analyst Information Technology ..................................... Cal Alumni Association | UC Berkeley 1 Alumni House, Berkeley, CA 94720 *T* 510.900.8196 *F* 510.642.6252 ..................................... *Commitment, Support, Passion* _____________________________________ *alumni.berkeley.edu <http://alumni.berkeley.edu/>* _____________________________________ *Facebook <https://www.facebook.com/CalAlumni> | LinkedIn <http://www.linkedin.com/groups?gid=70245>* On Thu, Jan 16, 2014 at 3:44 PM, Luke Rockwell < luke.rockwell@alumni.berkeley.edu> wrote:
1/16/14 11:41:42 PM GMT Field Services have completed preparation work for splicing on one end, but the other end is still being prepped as unforeseen issues caused delays. It has been informed that splicing is expected to begin at 00:30 GMT.
On Thu, Jan 16, 2014 at 3:31 PM, Tim Glen <tglen@asicentral.com> wrote:
1/16/14 10:52:27 PM GMT (5:52:27 PM EST)
The Transport NOC informed that Field Services are continuing their splicing preparations, which are expected to be completed by 00:00 GMT (7:00 PM EST) and splicing will begin subsequently.
------------------------------ *From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Luke Rockwell *Sent:* Thursday, January 16, 2014 4:46 PM
*To:* outages@outages.org *Subject:* Re: [outages] Level 3 issues in Ashburn
1/16/14 9:44:58 PM GMT The Transport NOC informed that Field Services are anticipating splicing preparations will begin at approximately 22:00 GMT. It has been informed that this process will take approximately two hours to complete, and splicing will commence thereafter.
On Thu, Jan 16, 2014 at 1:29 PM, Tim Glen <tglen@asicentral.com> wrote:
1/16/14 9:05:48 PM GMT
The Transport NOC advised that Field Services are currently working to replace the damaged conduit so that the replacement fiber can be placed in. An estimated start time for splicing is unavailable at this time.
*From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Luke Rockwell *Sent:* Thursday, January 16, 2014 3:07 PM
*To:* outages@outages.org *Subject:* Re: [outages] Level 3 issues in Ashburn
Ouch
1/16/14 8:00:49 PM GMT
Field Services are on site performing excavation of the impacted fiber, which is fairly deep. When the boring crew hit the fiber, approximately 25 feet of fiber was pulled out and will need to be repaired. Restoration efforts are underway, but no estimated restoral time is currently available.
On Thu, Jan 16, 2014 at 11:12 AM, Eric Dugas <EDugas@zerofail.com> wrote:
Also affects our Hibernia Atlantic transport to NYC. I've received the following update at 13:22 EST:
"Please be advised that Hibernia Networks is currently experiencing a network outage between Ashburn and Pittsburgh, USA since 16:40 GMT. We are currently investigating the issue.
Update1 - Please be informed that, our partner is currently investigating the issue and has dispatched techs to site, there is currently no ETA available, further updates to follow."
Eric Dugas edugas@zerofail.com Network Management Group ZEROFAIL / AS40191
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
1/17/14 2:51:29 AM GMTThe Transport NOC informed that splicing is ongoing to restore services and as splicing progresses, alarms are continuing to clear. The estimated restoral time of 05:00 GMT remains unchanged at this time.1/17/14 1:19:44 AM GMTSplicing has commenced on the damaged fiber and as splicing progresses, alarms are beginning to clear. It has been prevised that this splice work will be extensive and an estimated restoral time of 05:00 GMT has been provided. ..................................... *Luke Rockwell* Systems and Support Analyst Information Technology ..................................... Cal Alumni Association | UC Berkeley 1 Alumni House, Berkeley, CA 94720 *T* 510.900.8196 *F* 510.642.6252 ..................................... *Commitment, Support, Passion* _____________________________________ *alumni.berkeley.edu <http://alumni.berkeley.edu/>* _____________________________________ *Facebook <https://www.facebook.com/CalAlumni> | LinkedIn <http://www.linkedin.com/groups?gid=70245>* On Thu, Jan 16, 2014 at 5:20 PM, Luke Rockwell < luke.rockwell@alumni.berkeley.edu> wrote:
1/17/14 1:19:44 AM GMT Splicing has commenced on the damaged fiber and as splicing progresses, alarms are beginning to clear. It has been prevised that this splice work will be extensive and an estimated restoral time of 05:00 GMT has been provided.
..................................... *Luke Rockwell* Systems and Support Analyst Information Technology
.....................................
Cal Alumni Association | UC Berkeley 1 Alumni House, Berkeley, CA 94720 *T* 510.900.8196 *F* 510.642.6252
.....................................
*Commitment, Support, Passion*
_____________________________________ *alumni.berkeley.edu <http://alumni.berkeley.edu/>*
_____________________________________ *Facebook <https://www.facebook.com/CalAlumni> | LinkedIn <http://www.linkedin.com/groups?gid=70245>*
On Thu, Jan 16, 2014 at 3:44 PM, Luke Rockwell < luke.rockwell@alumni.berkeley.edu> wrote:
1/16/14 11:41:42 PM GMT Field Services have completed preparation work for splicing on one end, but the other end is still being prepped as unforeseen issues caused delays. It has been informed that splicing is expected to begin at 00:30 GMT.
On Thu, Jan 16, 2014 at 3:31 PM, Tim Glen <tglen@asicentral.com> wrote:
1/16/14 10:52:27 PM GMT (5:52:27 PM EST)
The Transport NOC informed that Field Services are continuing their splicing preparations, which are expected to be completed by 00:00 GMT (7:00 PM EST) and splicing will begin subsequently.
------------------------------ *From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Luke Rockwell *Sent:* Thursday, January 16, 2014 4:46 PM
*To:* outages@outages.org *Subject:* Re: [outages] Level 3 issues in Ashburn
1/16/14 9:44:58 PM GMT The Transport NOC informed that Field Services are anticipating splicing preparations will begin at approximately 22:00 GMT. It has been informed that this process will take approximately two hours to complete, and splicing will commence thereafter.
On Thu, Jan 16, 2014 at 1:29 PM, Tim Glen <tglen@asicentral.com> wrote:
1/16/14 9:05:48 PM GMT
The Transport NOC advised that Field Services are currently working to replace the damaged conduit so that the replacement fiber can be placed in. An estimated start time for splicing is unavailable at this time.
*From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Luke Rockwell *Sent:* Thursday, January 16, 2014 3:07 PM
*To:* outages@outages.org *Subject:* Re: [outages] Level 3 issues in Ashburn
Ouch
1/16/14 8:00:49 PM GMT
Field Services are on site performing excavation of the impacted fiber, which is fairly deep. When the boring crew hit the fiber, approximately 25 feet of fiber was pulled out and will need to be repaired. Restoration efforts are underway, but no estimated restoral time is currently available.
On Thu, Jan 16, 2014 at 11:12 AM, Eric Dugas <EDugas@zerofail.com> wrote:
Also affects our Hibernia Atlantic transport to NYC. I've received the following update at 13:22 EST:
"Please be advised that Hibernia Networks is currently experiencing a network outage between Ashburn and Pittsburgh, USA since 16:40 GMT. We are currently investigating the issue.
Update1 - Please be informed that, our partner is currently investigating the issue and has dispatched techs to site, there is currently no ETA available, further updates to follow."
Eric Dugas edugas@zerofail.com Network Management Group ZEROFAIL / AS40191
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
here is the final outage info. it has been cleared 1/17/14 10:46:07 AM GMTThe Technical Services Center has confirmed that all services are up and active. If additional issues are experienced, please contact the Level 3 Technical Service Center at 1-877-4LEVEL3.1/17/14 10:36:53 AM GMTThe Technical Services Center is still confirming the restoration of all customers.1/17/14 9:17:13 AM GMTThe Technical Services Center is still confirming the restoration of all customers.1/17/14 8:17:03 AM GMTTransport NOC reports splicing is complete. The technical service center has been engaged to confirm restoration with customers.1/17/14 7:52:36 AM GMTField crews report splicing continues and there are 2 tubes remaining to be spliced. Transport NOC is monitoring alarms.1/17/14 6:58:51 AM GMTTransport NOC reports field teams continue splicing and are reporting 6 tubes remaining to be spliced. An ETTR of 1 hour has been provided.1/17/14 5:46:03 AM GMTTransport NOC advises the field crews have completed splicing in the West pit and have 8 tubes remaining in the East pit. Teams continue splicing and expect to be complete in 1 hour.1/17/14 5:04:06 AM GMTTransport NOC advises field crews are reporting splicing is just over half way complete. It is estimated that splicing should be complete in about 2 hours or so.1/17/14 4:31:56 AM GMTTransport NOC reports splicing continues in the field. The West pit reports 18 tubes spliced while the East is reporting 12 tubes spliced. Alarms are clearing as splicing continues.1/17/14 3:34:35 AM GMTTransport NOC reports the field advises they are almost half way complete with splicing and expect the remaining splicing to be complete in 2 to 3 hours. Transport NOC is monitoring alarms as they clear. ..................................... *Luke Rockwell* Systems and Support Analyst Information Technology ..................................... Cal Alumni Association | UC Berkeley 1 Alumni House, Berkeley, CA 94720 *T* 510.900.8196 *F* 510.642.6252 ..................................... *Commitment, Support, Passion* _____________________________________ *alumni.berkeley.edu <http://alumni.berkeley.edu/>* _____________________________________ *Facebook <https://www.facebook.com/CalAlumni> | LinkedIn <http://www.linkedin.com/groups?gid=70245>* On Thu, Jan 16, 2014 at 7:25 PM, Luke Rockwell < luke.rockwell@alumni.berkeley.edu> wrote:
1/17/14 2:51:29 AM GMT The Transport NOC informed that splicing is ongoing to restore services and as splicing progresses, alarms are continuing to clear. The estimated restoral time of 05:00 GMT remains unchanged at this time. 1/17/14 1:19:44 AM GMT Splicing has commenced on the damaged fiber and as splicing progresses, alarms are beginning to clear. It has been prevised that this splice work will be extensive and an estimated restoral time of 05:00 GMT has been provided.
..................................... *Luke Rockwell* Systems and Support Analyst Information Technology
.....................................
Cal Alumni Association | UC Berkeley 1 Alumni House, Berkeley, CA 94720 *T* 510.900.8196 *F* 510.642.6252
.....................................
*Commitment, Support, Passion*
_____________________________________ *alumni.berkeley.edu <http://alumni.berkeley.edu/>*
_____________________________________ *Facebook <https://www.facebook.com/CalAlumni> | LinkedIn <http://www.linkedin.com/groups?gid=70245>*
On Thu, Jan 16, 2014 at 5:20 PM, Luke Rockwell < luke.rockwell@alumni.berkeley.edu> wrote:
1/17/14 1:19:44 AM GMT Splicing has commenced on the damaged fiber and as splicing progresses, alarms are beginning to clear. It has been prevised that this splice work will be extensive and an estimated restoral time of 05:00 GMT has been provided.
..................................... *Luke Rockwell* Systems and Support Analyst Information Technology
.....................................
Cal Alumni Association | UC Berkeley 1 Alumni House, Berkeley, CA 94720 *T* 510.900.8196 *F* 510.642.6252
.....................................
*Commitment, Support, Passion*
_____________________________________ *alumni.berkeley.edu <http://alumni.berkeley.edu/>*
_____________________________________ *Facebook <https://www.facebook.com/CalAlumni> | LinkedIn <http://www.linkedin.com/groups?gid=70245>*
On Thu, Jan 16, 2014 at 3:44 PM, Luke Rockwell < luke.rockwell@alumni.berkeley.edu> wrote:
1/16/14 11:41:42 PM GMT Field Services have completed preparation work for splicing on one end, but the other end is still being prepped as unforeseen issues caused delays. It has been informed that splicing is expected to begin at 00:30 GMT.
On Thu, Jan 16, 2014 at 3:31 PM, Tim Glen <tglen@asicentral.com> wrote:
1/16/14 10:52:27 PM GMT (5:52:27 PM EST)
The Transport NOC informed that Field Services are continuing their splicing preparations, which are expected to be completed by 00:00 GMT (7:00 PM EST) and splicing will begin subsequently.
------------------------------ *From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Luke Rockwell *Sent:* Thursday, January 16, 2014 4:46 PM
*To:* outages@outages.org *Subject:* Re: [outages] Level 3 issues in Ashburn
1/16/14 9:44:58 PM GMT The Transport NOC informed that Field Services are anticipating splicing preparations will begin at approximately 22:00 GMT. It has been informed that this process will take approximately two hours to complete, and splicing will commence thereafter.
On Thu, Jan 16, 2014 at 1:29 PM, Tim Glen <tglen@asicentral.com> wrote:
1/16/14 9:05:48 PM GMT
The Transport NOC advised that Field Services are currently working to replace the damaged conduit so that the replacement fiber can be placed in. An estimated start time for splicing is unavailable at this time.
*From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Luke Rockwell *Sent:* Thursday, January 16, 2014 3:07 PM
*To:* outages@outages.org *Subject:* Re: [outages] Level 3 issues in Ashburn
Ouch
1/16/14 8:00:49 PM GMT
Field Services are on site performing excavation of the impacted fiber, which is fairly deep. When the boring crew hit the fiber, approximately 25 feet of fiber was pulled out and will need to be repaired. Restoration efforts are underway, but no estimated restoral time is currently available.
On Thu, Jan 16, 2014 at 11:12 AM, Eric Dugas <EDugas@zerofail.com> wrote:
Also affects our Hibernia Atlantic transport to NYC. I've received the following update at 13:22 EST:
"Please be advised that Hibernia Networks is currently experiencing a network outage between Ashburn and Pittsburgh, USA since 16:40 GMT. We are currently investigating the issue.
Update1 - Please be informed that, our partner is currently investigating the issue and has dispatched techs to site, there is currently no ETA available, further updates to follow."
Eric Dugas edugas@zerofail.com Network Management Group ZEROFAIL / AS40191
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
participants (3)
-
Eric Dugas -
Luke Rockwell -
Tim Glen