Looks like all our clients on Level 3 are down right now. Casey Crowe IT Systems Engineer Palmetto Computer Consulting Office: (803) 832-2653 Cell: (803) 386-9302 www.palmettocc.net<http://www.palmettocc.net/> [PCC_logo]
We experienced connectivity loss from ~12:55 - 1:05 PM ET for multiple customers with Level 3 circuits in the Central Florida area. Service restored now. Thanks, Wyn From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Casey Crowe via Outages Sent: Thursday, March 02, 2017 1:04 PM To: outages@outages.org<mailto:outages@outages.org> Subject: [outages] Columbia SC Level 3 down Looks like all our clients on Level 3 are down right now. Casey Crowe IT Systems Engineer Palmetto Computer Consulting Office: (803) 832-2653 Cell: (803) 386-9302 www.palmettocc.net<http://www.palmettocc.net/> [PCC_logo]
Our Level3 circuit in our Houston, TX office just went down 20 minutes ago. Maybe it is related or not... not sure. David Warren, Systems Engineer Winstead PC | 500 Winstead Building | 2728 N. Harwood Street | Dallas, Texas 75201 214.745.5778 (Office) | 214.745.5390 (Fax) | dwarren@winstead.com<mailto:dwarren@winstead.com> | www.winstead.com<http://www.winstead.com/> From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Wyn Bryant via Outages Sent: Thursday, March 02, 2017 12:12 PM To: outages@outages.org Subject: Re: [outages] Columbia SC Level 3 down We experienced connectivity loss from ~12:55 - 1:05 PM ET for multiple customers with Level 3 circuits in the Central Florida area. Service restored now. Thanks, Wyn From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Casey Crowe via Outages Sent: Thursday, March 02, 2017 1:04 PM To: outages@outages.org<mailto:outages@outages.org> Subject: [outages] Columbia SC Level 3 down Looks like all our clients on Level 3 are down right now. Casey Crowe IT Systems Engineer Palmetto Computer Consulting Office: (803) 832-2653 Cell: (803) 386-9302 www.palmettocc.net<http://www.palmettocc.net/> [PCC_logo] ________________________________ Information contained in this transmission is attorney privileged and confidential. It is intended for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copy of this communication is strictly prohibited. If you have received this communication in error, please immediately notify us by telephone. --
We just had several customers lose Level 3 (Legacy TWT) circuits in Cincinnati OH On Thu, Mar 2, 2017 at 1:04 PM, Casey Crowe via Outages <outages@outages.org
wrote:
Looks like all our clients on Level 3 are down right now.
*Casey Crowe*
*IT Systems Engineer*
Palmetto Computer Consulting
Office: (803) 832-2653
Cell: (803) 386-9302
www.palmettocc.net
[image: PCC_logo]
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Apologies, service is not restored in all cases. Some that are back up are due to redundant connections. Have cases of Level 3 circuits down throughout south east US. From: Wyn Bryant Sent: Thursday, March 02, 2017 1:12 PM To: 'outages@outages.org' <outages@outages.org> Subject: RE: Columbia SC Level 3 down We experienced connectivity loss from ~12:55 - 1:05 PM ET for multiple customers with Level 3 circuits in the Central Florida area. Service restored now. Thanks, Wyn From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Casey Crowe via Outages Sent: Thursday, March 02, 2017 1:04 PM To: outages@outages.org<mailto:outages@outages.org> Subject: [outages] Columbia SC Level 3 down Looks like all our clients on Level 3 are down right now. Casey Crowe IT Systems Engineer Palmetto Computer Consulting Office: (803) 832-2653 Cell: (803) 386-9302 www.palmettocc.net<http://www.palmettocc.net/> [PCC_logo]
After a 10 minute hold - spoke w/ L3 support - they acknowledge a national IP event on legacy twt network most of our circuits have come back up (about 45 minutes down) On Thu, Mar 2, 2017 at 1:20 PM, Wyn Bryant via Outages <outages@outages.org> wrote:
Apologies, service is not restored in all cases. Some that are back up are due to redundant connections. Have cases of Level 3 circuits down throughout south east US.
*From:* Wyn Bryant *Sent:* Thursday, March 02, 2017 1:12 PM *To:* 'outages@outages.org' <outages@outages.org> *Subject:* RE: Columbia SC Level 3 down
We experienced connectivity loss from ~12:55 – 1:05 PM ET for multiple customers with Level 3 circuits in the Central Florida area. Service restored now.
Thanks,
Wyn
*From:* Outages [mailto:outages-bounces@outages.org <outages-bounces@outages.org>] *On Behalf Of *Casey Crowe via Outages *Sent:* Thursday, March 02, 2017 1:04 PM *To:* outages@outages.org *Subject:* [outages] Columbia SC Level 3 down
Looks like all our clients on Level 3 are down right now.
*Casey Crowe*
*IT Systems Engineer*
Palmetto Computer Consulting
Office: (803) 832-2653
Cell: (803) 386-9302
www.palmettocc.net
[image: PCC_logo]
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
FYI: We have an unrelated ticket open with Level3 and received a response that they are troubleshooting issues in several North American markets. Randy Raitz, CISSP Director, IT & Operations T 303-209-1616 | M 303-594-8817 ReadyTalk, a PGi Company | 1900 16th Street, Suite 600 | Denver, CO 80202 From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Wyn Bryant via Outages Sent: Thursday, March 2, 2017 11:20 AM To: outages@outages.org Subject: Re: [outages] Columbia SC Level 3 down Apologies, service is not restored in all cases. Some that are back up are due to redundant connections. Have cases of Level 3 circuits down throughout south east US. From: Wyn Bryant Sent: Thursday, March 02, 2017 1:12 PM To: 'outages@outages.org' <outages@outages.org<mailto:outages@outages.org>> Subject: RE: Columbia SC Level 3 down We experienced connectivity loss from ~12:55 - 1:05 PM ET for multiple customers with Level 3 circuits in the Central Florida area. Service restored now. Thanks, Wyn From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Casey Crowe via Outages Sent: Thursday, March 02, 2017 1:04 PM To: outages@outages.org<mailto:outages@outages.org> Subject: [outages] Columbia SC Level 3 down Looks like all our clients on Level 3 are down right now. Casey Crowe IT Systems Engineer Palmetto Computer Consulting Office: (803) 832-2653 Cell: (803) 386-9302 www.palmettocc.net<http://www.palmettocc.net/> [PCC_logo] Disclaimer The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.
In Charlotte NC - Our Level 3 (formerly TWT) neighbor was down for at least 10 minutes, it seems to be functional again. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Casey Crowe via Outages Sent: Thursday, March 2, 2017 1:04 PM To: outages@outages.org Subject: [outages] Columbia SC Level 3 down Looks like all our clients on Level 3 are down right now. Casey Crowe IT Systems Engineer Palmetto Computer Consulting Office: (803) 832-2653 Cell: (803) 386-9302 www.palmettocc.net<https://linkprotect.cudasvc.com/url?a=http://www.palmettocc.net/&c=E,1,zOIB4FhITo6LtIkEenFxWggevFGjZudosEHSzz_EShCzfZ9xyenkFD3SIZZANsrKfJFPjRO8ezoIvCU9X40vCoonF3Au2Z7YvdjrJVWPR1878iNfSJE,&typo=0> [PCC_logo]
Same also in Charlotte, NC between 1:15 - 1:30pm ET. No other reports yet. Jon On Mar 2, 2017, at 1:07 PM, Maurice Jeter via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: In Charlotte NC - Our Level 3 (formerly TWT) neighbor was down for at least 10 minutes, it seems to be functional again. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Casey Crowe via Outages Sent: Thursday, March 2, 2017 1:04 PM To: outages@outages.org<mailto:outages@outages.org> Subject: [outages] Columbia SC Level 3 down Looks like all our clients on Level 3 are down right now. Casey Crowe IT Systems Engineer Palmetto Computer Consulting Office: (803) 832-2653 Cell: (803) 386-9302 www.palmettocc.net<https://linkprotect.cudasvc.com/url?a=http://www.palmettocc.net/&c=E,1,zOIB4FhITo6LtIkEenFxWggevFGjZudosEHSzz_EShCzfZ9xyenkFD3SIZZANsrKfJFPjRO8ezoIvCU9X40vCoonF3Au2Z7YvdjrJVWPR1878iNfSJE,&typo=0> <image001.gif> _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages
FWIW legacy TWTC circuit in Greenville SC seems unaffected. Similar legacy TWTC circuit we have in Mobile AL seems fine as well. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Casey Crowe via Outages Sent: Thursday, March 02, 2017 12:04 PM To: outages@outages.org Subject: [outages] Columbia SC Level 3 down Looks like all our clients on Level 3 are down right now. Casey Crowe IT Systems Engineer Palmetto Computer Consulting Office: (803) 832-2653 Cell: (803) 386-9302 www.palmettocc.net<http://www.palmettocc.net/> [PCC_logo]
Obviously I spoke too soon. Greenville just dropped offline. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Neader, Brent via Outages Sent: Thursday, March 02, 2017 1:30 PM To: outages@outages.org Subject: Re: [outages] Columbia SC Level 3 down FWIW legacy TWTC circuit in Greenville SC seems unaffected. Similar legacy TWTC circuit we have in Mobile AL seems fine as well. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Casey Crowe via Outages Sent: Thursday, March 02, 2017 12:04 PM To: outages@outages.org<mailto:outages@outages.org> Subject: [outages] Columbia SC Level 3 down Looks like all our clients on Level 3 are down right now. Casey Crowe IT Systems Engineer Palmetto Computer Consulting Office: (803) 832-2653 Cell: (803) 386-9302 www.palmettocc.net<http://www.palmettocc.net/> [PCC_logo]
These are the latest notes I have from Level3. We are still down in Houston, TX. *** CASCADED EXTERNAL NOTES 02-Mar-2017 20:10:06 GMT From CASE: 12322193 - Event The IP NOC continues to work to identify the root cause of the issue, but have advised that the routers are beginning to stabilize at this time, as only two routers remain impacted. The IP NOC will continue to work to stabilize the remaining routers while the root cause investigations remain ongoing. David Warren, Systems Engineer Winstead PC | 500 Winstead Building | 2728 N. Harwood Street | Dallas, Texas 75201 214.745.5778 (Office) | 214.745.5390 (Fax) | dwarren@winstead.com<mailto:dwarren@winstead.com> | www.winstead.com<http://www.winstead.com/> From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Neader, Brent via Outages Sent: Thursday, March 02, 2017 2:15 PM To: outages@outages.org Subject: Re: [outages] Columbia SC Level 3 down Obviously I spoke too soon. Greenville just dropped offline. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Neader, Brent via Outages Sent: Thursday, March 02, 2017 1:30 PM To: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Columbia SC Level 3 down FWIW legacy TWTC circuit in Greenville SC seems unaffected. Similar legacy TWTC circuit we have in Mobile AL seems fine as well. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Casey Crowe via Outages Sent: Thursday, March 02, 2017 12:04 PM To: outages@outages.org<mailto:outages@outages.org> Subject: [outages] Columbia SC Level 3 down Looks like all our clients on Level 3 are down right now. Casey Crowe IT Systems Engineer Palmetto Computer Consulting Office: (803) 832-2653 Cell: (803) 386-9302 www.palmettocc.net<http://www.palmettocc.net/> [PCC_logo] ________________________________ Information contained in this transmission is attorney privileged and confidential. It is intended for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copy of this communication is strictly prohibited. If you have received this communication in error, please immediately notify us by telephone. --
I'm in Columbia - all of our legacy TWTC circuits are layer 2 and seem unaffected - our DIA with L3 is with AS3356, and also seems unaffected. Berry On 3/2/2017 3:15 PM, Neader, Brent via Outages wrote:
Obviously I spoke too soon. Greenville just dropped offline.
*From:*Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Neader, Brent via Outages *Sent:* Thursday, March 02, 2017 1:30 PM *To:* outages@outages.org *Subject:* Re: [outages] Columbia SC Level 3 down
FWIW legacy TWTC circuit in Greenville SC seems unaffected. Similar legacy TWTC circuit we have in Mobile AL seems fine as well.
*From:*Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Casey Crowe via Outages *Sent:* Thursday, March 02, 2017 12:04 PM *To:* outages@outages.org <mailto:outages@outages.org> *Subject:* [outages] Columbia SC Level 3 down
Looks like all our clients on Level 3 are down right now.
*Casey Crowe*
/IT Systems Engineer/
Palmetto Computer Consulting
Office: (803) 832-2653
Cell: (803) 386-9302
www.palmettocc.net <http://www.palmettocc.net/>
PCC_logo
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Greenville is back From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Berry Mobley via Outages Sent: Thursday, March 02, 2017 2:48 PM To: outages@outages.org Subject: Re: [outages] Columbia SC Level 3 down I'm in Columbia - all of our legacy TWTC circuits are layer 2 and seem unaffected - our DIA with L3 is with AS3356, and also seems unaffected. Berry On 3/2/2017 3:15 PM, Neader, Brent via Outages wrote: Obviously I spoke too soon. Greenville just dropped offline. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Neader, Brent via Outages Sent: Thursday, March 02, 2017 1:30 PM To: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Columbia SC Level 3 down FWIW legacy TWTC circuit in Greenville SC seems unaffected. Similar legacy TWTC circuit we have in Mobile AL seems fine as well. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Casey Crowe via Outages Sent: Thursday, March 02, 2017 12:04 PM To: outages@outages.org<mailto:outages@outages.org> Subject: [outages] Columbia SC Level 3 down Looks like all our clients on Level 3 are down right now. Casey Crowe IT Systems Engineer Palmetto Computer Consulting Office: (803) 832-2653 Cell: (803) 386-9302 www.palmettocc.net<http://www.palmettocc.net/> [PCC_logo] _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages
I Still see a circuit in Houston down. Patience is sometimes hard to come by. ________________________________ From: Outages <outages-bounces@outages.org> on behalf of Neader, Brent via Outages <outages@outages.org> Sent: Thursday, March 2, 2017 3:58 PM To: outages@outages.org Subject: Re: [outages] Columbia SC Level 3 down Greenville is back From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Berry Mobley via Outages Sent: Thursday, March 02, 2017 2:48 PM To: outages@outages.org Subject: Re: [outages] Columbia SC Level 3 down I'm in Columbia - all of our legacy TWTC circuits are layer 2 and seem unaffected - our DIA with L3 is with AS3356, and also seems unaffected. Berry On 3/2/2017 3:15 PM, Neader, Brent via Outages wrote: Obviously I spoke too soon. Greenville just dropped offline. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Neader, Brent via Outages Sent: Thursday, March 02, 2017 1:30 PM To: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Columbia SC Level 3 down FWIW legacy TWTC circuit in Greenville SC seems unaffected. Similar legacy TWTC circuit we have in Mobile AL seems fine as well. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Casey Crowe via Outages Sent: Thursday, March 02, 2017 12:04 PM To: outages@outages.org<mailto:outages@outages.org> Subject: [outages] Columbia SC Level 3 down Looks like all our clients on Level 3 are down right now. Casey Crowe IT Systems Engineer Palmetto Computer Consulting Office: (803) 832-2653 Cell: (803) 386-9302 www.palmettocc.net<http://www.palmettocc.net/> [PCC_logo] _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages
participants (10)
-
Andrew MacGregor -
Berry Mobley -
Casey Crowe -
Jeff Waddell -
Jon Hurley -
Maurice Jeter -
Neader, Brent -
Randall (Randy) Raitz -
Warren, David -
Wyn Bryant