Anyone else have any outages on Zayo's network starting 08:34:55.656 UTC. I was seeing a impact in DFW and NYC.
On Mon, 18 May 2015, Joseph Jackson via Outages wrote:
Anyone else have any outages on Zayo's network starting 08:34:55.656 UTC. I was seeing a impact in DFW and NYC.
Closer to 8:45AM EDT for us...but looks like something went boom in the Ashburn area. ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On Mon, May 18, 2015 at 6:34 AM, Jon Lewis via Outages <outages@outages.org> wrote:
On Mon, 18 May 2015, Joseph Jackson via Outages wrote:
Anyone else have any outages on Zayo's network starting 08:34:55.656
UTC. I was seeing a impact in DFW and NYC.
Closer to 8:45AM EDT for us...but looks like something went boom in the Ashburn area.
I see a Level3 between PHL and LGA down at that time. Still down.
---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
I got this from our NOC. "Zayo is observing a possible fiber cut between Reidsville, NC and Chatham, VA. We have reported the outage to our third party fiber provider. We will keep you updated as isolation and restoration progresses." Thanks, James Milko Architect, Network Engineering 900 Main Campus Drive Raleigh, NC 27606 Bandwidth <http://www.bandwidth.com/business/> o 919.439.1935 On Mon, May 18, 2015 at 10:00 AM, Ca By via Outages <outages@outages.org> wrote:
On Mon, May 18, 2015 at 6:34 AM, Jon Lewis via Outages < outages@outages.org> wrote:
On Mon, 18 May 2015, Joseph Jackson via Outages wrote:
Anyone else have any outages on Zayo's network starting 08:34:55.656
UTC. I was seeing a impact in DFW and NYC.
Closer to 8:45AM EDT for us...but looks like something went boom in the Ashburn area.
I see a Level3 between PHL and LGA down at that time. Still down.
---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Latest update I got from Zayo is as follows: Our Third Party fiber provider has advised that this issue is impacting protected services as well. Field Services have advised that the construction crew and splicing crew are expected to arrive onsite simultaneously at approximately 16:45 GMT. We will continue to provide updates until your services have been restored. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of James Milko via Outages Sent: Monday, May 18, 2015 10:44 AM To: Ca By Cc: outages@outages.org Subject: Re: [outages] Zayo I got this from our NOC. "Zayo is observing a possible fiber cut between Reidsville, NC and Chatham, VA. We have reported the outage to our third party fiber provider. We will keep you updated as isolation and restoration progresses." Thanks, James Milko Architect, Network Engineering 900 Main Campus Drive Raleigh, NC 27606 Bandwidth<http://www.bandwidth.com/business/> o 919.439.1935 On Mon, May 18, 2015 at 10:00 AM, Ca By via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: On Mon, May 18, 2015 at 6:34 AM, Jon Lewis via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: On Mon, 18 May 2015, Joseph Jackson via Outages wrote: Anyone else have any outages on Zayo's network starting 08:34:55.656 UTC. I was seeing a impact in DFW and NYC. Closer to 8:45AM EDT for us...but looks like something went boom in the Ashburn area. I see a Level3 between PHL and LGA down at that time. Still down. ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages
I’m told fiber damage is in a remote area near a manhole on a mountain in Kernersville, NC. Update @1:45PM EDT: “Our Third Party fiber provider has advised that outside plant engineer have determined that working in the manhole closest to the break point is not possible. The manholes proximity to an access road would position the repair point in the middle of the road and would be unsafe. The Field Crew is now working to excavate the fiber at point on the other side of the creek onsite. The Field Crew expects to begin preparing the affected cable for the repair in 90 minutes.” -Vinny From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Vinny Abello via Outages Sent: Monday, May 18, 2015 12:53 PM To: outages@outages.org Subject: Re: [outages] Zayo Latest update I got from Zayo is as follows: Our Third Party fiber provider has advised that this issue is impacting protected services as well. Field Services have advised that the construction crew and splicing crew are expected to arrive onsite simultaneously at approximately 16:45 GMT. We will continue to provide updates until your services have been restored. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of James Milko via Outages Sent: Monday, May 18, 2015 10:44 AM To: Ca By Cc: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Zayo I got this from our NOC. "Zayo is observing a possible fiber cut between Reidsville, NC and Chatham, VA. We have reported the outage to our third party fiber provider. We will keep you updated as isolation and restoration progresses." Thanks, James Milko Architect, Network Engineering 900 Main Campus Drive Raleigh, NC 27606 Bandwidth<http://www.bandwidth.com/business/> o 919.439.1935 On Mon, May 18, 2015 at 10:00 AM, Ca By via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: On Mon, May 18, 2015 at 6:34 AM, Jon Lewis via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: On Mon, 18 May 2015, Joseph Jackson via Outages wrote: Anyone else have any outages on Zayo's network starting 08:34:55.656 UTC. I was seeing a impact in DFW and NYC. Closer to 8:45AM EDT for us...but looks like something went boom in the Ashburn area. I see a Level3 between PHL and LGA down at that time. Still down. ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages
Latest update I received @3:37PM EDT for anyone following or affected: “Our Third Party fiber provider advised that Field Services have laid the necessary ductwork and will begin pulling the new section of fiber through the duct once both sides of the fiber are prepared for the cable repair. Field Services expect to begin splicing in approximately 2 hours. The Field Crew is still unable to provide an estimate for the completion of the fiber repair at this time. We will continue to provide updates until your services have been restored.” From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Vinny Abello via Outages Sent: Monday, May 18, 2015 2:45 PM To: outages@outages.org Subject: Re: [outages] Zayo I’m told fiber damage is in a remote area near a manhole on a mountain in Kernersville, NC. Update @1:45PM EDT: “Our Third Party fiber provider has advised that outside plant engineer have determined that working in the manhole closest to the break point is not possible. The manholes proximity to an access road would position the repair point in the middle of the road and would be unsafe. The Field Crew is now working to excavate the fiber at point on the other side of the creek onsite. The Field Crew expects to begin preparing the affected cable for the repair in 90 minutes.” -Vinny From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Vinny Abello via Outages Sent: Monday, May 18, 2015 12:53 PM To: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Zayo Latest update I got from Zayo is as follows: Our Third Party fiber provider has advised that this issue is impacting protected services as well. Field Services have advised that the construction crew and splicing crew are expected to arrive onsite simultaneously at approximately 16:45 GMT. We will continue to provide updates until your services have been restored. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of James Milko via Outages Sent: Monday, May 18, 2015 10:44 AM To: Ca By Cc: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Zayo I got this from our NOC. "Zayo is observing a possible fiber cut between Reidsville, NC and Chatham, VA. We have reported the outage to our third party fiber provider. We will keep you updated as isolation and restoration progresses." Thanks, James Milko Architect, Network Engineering 900 Main Campus Drive Raleigh, NC 27606 Bandwidth<http://www.bandwidth.com/business/> o 919.439.1935 On Mon, May 18, 2015 at 10:00 AM, Ca By via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: On Mon, May 18, 2015 at 6:34 AM, Jon Lewis via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: On Mon, 18 May 2015, Joseph Jackson via Outages wrote: Anyone else have any outages on Zayo's network starting 08:34:55.656 UTC. I was seeing a impact in DFW and NYC. Closer to 8:45AM EDT for us...but looks like something went boom in the Ashburn area. I see a Level3 between PHL and LGA down at that time. Still down. ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages
Just a final follow-up for informational purposes. Service restored and final update from Zayo at 10:102PM EDT last night on the issue: “At this time our third-party provider has confirmed that they are hands off. They have confirmed that there will be a permanent fix at an undetermined later date. We will be notified of this work in advance and will schedule a maintenance window to notify you as appropriate. It has also been determined that a construction crew bored in to the cable while laying pipe. We sincerely appreciate your patience in this matter.” From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Vinny Abello via Outages Sent: Monday, May 18, 2015 3:57 PM To: outages@outages.org Subject: Re: [outages] Zayo Latest update I received @3:37PM EDT for anyone following or affected: “Our Third Party fiber provider advised that Field Services have laid the necessary ductwork and will begin pulling the new section of fiber through the duct once both sides of the fiber are prepared for the cable repair. Field Services expect to begin splicing in approximately 2 hours. The Field Crew is still unable to provide an estimate for the completion of the fiber repair at this time. We will continue to provide updates until your services have been restored.” From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Vinny Abello via Outages Sent: Monday, May 18, 2015 2:45 PM To: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Zayo I’m told fiber damage is in a remote area near a manhole on a mountain in Kernersville, NC. Update @1:45PM EDT: “Our Third Party fiber provider has advised that outside plant engineer have determined that working in the manhole closest to the break point is not possible. The manholes proximity to an access road would position the repair point in the middle of the road and would be unsafe. The Field Crew is now working to excavate the fiber at point on the other side of the creek onsite. The Field Crew expects to begin preparing the affected cable for the repair in 90 minutes.” -Vinny From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Vinny Abello via Outages Sent: Monday, May 18, 2015 12:53 PM To: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Zayo Latest update I got from Zayo is as follows: Our Third Party fiber provider has advised that this issue is impacting protected services as well. Field Services have advised that the construction crew and splicing crew are expected to arrive onsite simultaneously at approximately 16:45 GMT. We will continue to provide updates until your services have been restored. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of James Milko via Outages Sent: Monday, May 18, 2015 10:44 AM To: Ca By Cc: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Zayo I got this from our NOC. "Zayo is observing a possible fiber cut between Reidsville, NC and Chatham, VA. We have reported the outage to our third party fiber provider. We will keep you updated as isolation and restoration progresses." Thanks, James Milko Architect, Network Engineering 900 Main Campus Drive Raleigh, NC 27606 Bandwidth<http://www.bandwidth.com/business/> o 919.439.1935 On Mon, May 18, 2015 at 10:00 AM, Ca By via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: On Mon, May 18, 2015 at 6:34 AM, Jon Lewis via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: On Mon, 18 May 2015, Joseph Jackson via Outages wrote: Anyone else have any outages on Zayo's network starting 08:34:55.656 UTC. I was seeing a impact in DFW and NYC. Closer to 8:45AM EDT for us...but looks like something went boom in the Ashburn area. I see a Level3 between PHL and LGA down at that time. Still down. ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages
Any updates from Zayo since yesterday? Robbie Trencheny (@robbie <http://twitter.com/robbie>) 925-884-3728 robbie.io On Mon, May 18, 2015 at 12:57 PM, Vinny Abello via Outages < outages@outages.org> wrote:
Latest update I received @3:37PM EDT for anyone following or affected:
“Our Third Party fiber provider advised that Field Services have laid the necessary ductwork and will begin pulling the new section of fiber through the duct once both sides of the fiber are prepared for the cable repair. Field Services expect to begin splicing in approximately 2 hours. The Field Crew is still unable to provide an estimate for the completion of the fiber repair at this time. We will continue to provide updates until your services have been restored.”
*From:* Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Vinny Abello via Outages *Sent:* Monday, May 18, 2015 2:45 PM
*To:* outages@outages.org *Subject:* Re: [outages] Zayo
I’m told fiber damage is in a remote area near a manhole on a mountain in Kernersville, NC.
Update @1:45PM EDT:
“Our Third Party fiber provider has advised that outside plant engineer have determined that working in the manhole closest to the break point is not possible. The manholes proximity to an access road would position the repair point in the middle of the road and would be unsafe. The Field Crew is now working to excavate the fiber at point on the other side of the creek onsite. The Field Crew expects to begin preparing the affected cable for the repair in 90 minutes.”
-Vinny
*From:* Outages [mailto:outages-bounces@outages.org <outages-bounces@outages.org>] *On Behalf Of *Vinny Abello via Outages *Sent:* Monday, May 18, 2015 12:53 PM *To:* outages@outages.org *Subject:* Re: [outages] Zayo
Latest update I got from Zayo is as follows:
Our Third Party fiber provider has advised that this issue is impacting protected services as well. Field Services have advised that the construction crew and splicing crew are expected to arrive onsite simultaneously at approximately 16:45 GMT. We will continue to provide updates until your services have been restored.
*From:* Outages [mailto:outages-bounces@outages.org <outages-bounces@outages.org>] *On Behalf Of *James Milko via Outages *Sent:* Monday, May 18, 2015 10:44 AM *To:* Ca By *Cc:* outages@outages.org *Subject:* Re: [outages] Zayo
I got this from our NOC.
"Zayo is observing a possible fiber cut between Reidsville, NC and Chatham, VA. We have reported the outage to our third party fiber provider. We will keep you updated as isolation and restoration progresses."
Thanks,
*James Milko*
*Architect, Network Engineering*
900 Main Campus Drive
Raleigh, NC 27606
Bandwidth <http://www.bandwidth.com/business/>
*o* 919.439.1935
On Mon, May 18, 2015 at 10:00 AM, Ca By via Outages <outages@outages.org> wrote:
On Mon, May 18, 2015 at 6:34 AM, Jon Lewis via Outages < outages@outages.org> wrote:
On Mon, 18 May 2015, Joseph Jackson via Outages wrote:
Anyone else have any outages on Zayo's network starting 08:34:55.656 UTC. I was seeing a impact in DFW and NYC.
Closer to 8:45AM EDT for us...but looks like something went boom in the Ashburn area.
I see a Level3 between PHL and LGA down at that time. Still down.
---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
My company was contracted to make this repair. There is a 144ct that was cut near Klamath Falls, Oregon belonging to Zayo. Crews dug through the night to uncover the relevant vaults, removal of damaged section happened approx 0545 PST and splicers are currently reassembling connections. ETR is 14:00PST at this time. Phil Lee, Network Engineer Direct: 541.414.1428 hunterfiber.com [download] From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Vinny Abello via Outages Sent: Monday, May 18, 2015 12:57 PM To: outages@outages.org Subject: Re: [outages] Zayo Latest update I received @3:37PM EDT for anyone following or affected: “Our Third Party fiber provider advised that Field Services have laid the necessary ductwork and will begin pulling the new section of fiber through the duct once both sides of the fiber are prepared for the cable repair. Field Services expect to begin splicing in approximately 2 hours. The Field Crew is still unable to provide an estimate for the completion of the fiber repair at this time. We will continue to provide updates until your services have been restored.” From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Vinny Abello via Outages Sent: Monday, May 18, 2015 2:45 PM To: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Zayo I’m told fiber damage is in a remote area near a manhole on a mountain in Kernersville, NC. Update @1:45PM EDT: “Our Third Party fiber provider has advised that outside plant engineer have determined that working in the manhole closest to the break point is not possible. The manholes proximity to an access road would position the repair point in the middle of the road and would be unsafe. The Field Crew is now working to excavate the fiber at point on the other side of the creek onsite. The Field Crew expects to begin preparing the affected cable for the repair in 90 minutes.” -Vinny From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Vinny Abello via Outages Sent: Monday, May 18, 2015 12:53 PM To: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Zayo Latest update I got from Zayo is as follows: Our Third Party fiber provider has advised that this issue is impacting protected services as well. Field Services have advised that the construction crew and splicing crew are expected to arrive onsite simultaneously at approximately 16:45 GMT. We will continue to provide updates until your services have been restored. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of James Milko via Outages Sent: Monday, May 18, 2015 10:44 AM To: Ca By Cc: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Zayo I got this from our NOC. "Zayo is observing a possible fiber cut between Reidsville, NC and Chatham, VA. We have reported the outage to our third party fiber provider. We will keep you updated as isolation and restoration progresses." Thanks, James Milko Architect, Network Engineering 900 Main Campus Drive Raleigh, NC 27606 Bandwidth<http://www.bandwidth.com/business/> o 919.439.1935 On Mon, May 18, 2015 at 10:00 AM, Ca By via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: On Mon, May 18, 2015 at 6:34 AM, Jon Lewis via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: On Mon, 18 May 2015, Joseph Jackson via Outages wrote: Anyone else have any outages on Zayo's network starting 08:34:55.656 UTC. I was seeing a impact in DFW and NYC. Closer to 8:45AM EDT for us...but looks like something went boom in the Ashburn area. I see a Level3 between PHL and LGA down at that time. Still down. ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages
participants (7)
-
Ca By -
James Milko -
Jon Lewis -
Joseph Jackson -
Phil Lee -
Robbie Trencheny -
Vinny_Abello＠Dell.com