Anyone have more details or an ETR? Nate On 2/14/2012 12:59 PM, Brandon Galbraith wrote:

Their Phonebooth product is up and running FWIW. Obviously not using Core2.

On Tue, Feb 14, 2012 at 12:57 PM, Geoffrey Mina <gmina@connectfirst.com <mailto:gmina@connectfirst.com>> wrote:

We are also seeing a full outage of the Core2 network. Opened a ticket, but no response from BW support.

Â

*Geoff MINA*

Â

*From:*outages-bounces@outages.org <mailto:outages-bounces@outages.org> [mailto:outages-bounces@outages.org <mailto:outages-bounces@outages.org>] *On Behalf Of *Michael R. Self *Sent:* Tuesday, February 14, 2012 11:49 AM *To:* outages@outages.org <mailto:outages@outages.org> *Subject:* [outages] Bandwidth.com outage

Â

I'm seeing a total outage at Bandwidth.com on their CORE 2 network for all incoming and outgoing SIP traffic.

Â

Anybody else? Â So much for redundant IP's and switches on CORE 2âEUR¦..

Â

Â

____________________________________________

Description: cid:1299ACAC-5785-4008-ACE6-5B36ABA9ADA4* **Michael R Self*

My PhoneFusion ONE number -Â *954.644.5000 <tel:954.644.5000>*Â (office/home/fax/mobile/SMS/VoIP) */_Disclaimer_/*/_:*Â If this email isn't for you,*Â _/*/_get rid of it._/*

_______________________________________________ Outages mailing list Outages@outages.org <mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages

-- Brandon Galbraith US Voice: 630.492.0464

_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

We are seeing signaling, but no audio. Looks like they are making progress… albeit slow. Geoff MINA From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Robin Rodriguez Sent: Tuesday, February 14, 2012 1:15 PM To: outages@outages.org Subject: Re: [outages] Bandwidth.com outage Bandwidth.com hasn't issued an all-clear but CORE2 Orig is functioning again for us out of LA and NYC On Tue, Feb 14, 2012 at 1:53 PM, Nate Burke <nate@blastcomm.com> wrote: Anyone have more details or an ETR? Nate On 2/14/2012 12:59 PM, Brandon Galbraith wrote: Their Phonebooth product is up and running FWIW. Obviously not using Core2. On Tue, Feb 14, 2012 at 12:57 PM, Geoffrey Mina <gmina@connectfirst.com> wrote: We are also seeing a full outage of the Core2 network. Opened a ticket, but no response from BW support.  Geoff MINA  From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Michael R. Self Sent: Tuesday, February 14, 2012 11:49 AM To: outages@outages.org Subject: [outages] Bandwidth.com outage  I'm seeing a total outage at Bandwidth.com on their CORE 2 network for all incoming and outgoing SIP traffic.  Anybody else?  So much for redundant IP's and switches on CORE 2…..   ____________________________________________ Michael R Self My PhoneFusion ONE number - 954.644.5000 (office/home/fax/mobile/SMS/VoIP) Disclaimer: If this email isn't for you, get rid of it. _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages -- Brandon Galbraith US Voice: 630.492.0464 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages -- Robin Rodriguez

Bandwidth informed us yesterday morning that the issue is closed. They issued a formal RFO that puts blame squarely on "another transport provider" that was broadcasting their routes and they worked with this unnamed provider. </whatiknow> ____________________________________________ Michael R Self My PhoneFusion ONE number - 954.644.5000 (office/home/fax/mobile/SMS/VoIP) Disclaimer: If this email isn't for you, get rid of it. On 2/16/12 10:31 AM, "Robert Kulagowski" <rkulagow@gmail.com> wrote:

On Tue, Feb 14, 2012 at 2:15 PM, Robin Rodriguez <rrodriguez@ifbyphone.com> wrote:

...

Bandwidth.com hasn't issued an all-clear but CORE2 Orig is functioning again for us out of LA and NYC

Was a final RFO ever posted? Was this something that bandwidth.com did "wrong", or was it because someone started announcing their prefixes? _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

I have two datacenters that are trying to handle calls from Core2. One is working properly, one is not. Good route: Host Loss% Last Avg Best Wrst StDev 1. ge-7-13.car1.Atlanta2.Level3.net 0.0% 0.2 10.2 0.2 136.1 32.1 2. vlan51.ebr1.Atlanta2.Level3.net 0.0% 0.6 2.2 0.2 10.5 3.2 3. ae-6-6.ebr1.Washington12.Level3.net 0.0% 12.1 14.4 12.0 22.5 3.5 4. ae-1-100.ebr2.Washington12.Level3.net 0.0% 12.9 14.4 12.0 24.0 3.7 5. 4.69.148.49 0.0% 17.3 17.5 17.2 19.5 0.5 6. ae-61-61.csw1.NewYork1.Level3.net 0.0% 19.7 22.3 17.4 30.6 4.3 7. ae-13-60.car3.NewYork1.Level3.net 0.0% 17.5 38.1 17.5 219.1 45.2 8. BANDWIDTH.C.car3.NewYork1.Level3.net 0.0% 18.1 20.0 17.6 28.3 3.3 9. 67.231.8.103 0.0% 18.5 18.0 17.7 18.9 0.2 Bad Route: Host Loss% Last Avg Best Wrst StDev 1. 67.220.101.162 1.3% 0.5 3.7 0.2 170.3 20.8 2. ten8-4.core2.atl1.qualitytech.com 0.0% 0.5 3.1 0.3 193.6 22.3 3. ten8-1.peer1.atl1.qualitytech.com 0.0% 0.3 1.3 0.2 66.9 7.7 4. 66-193-162-93.static.twtelecom.net 0.0% 0.7 1.4 0.5 34.9 4.4 5. orl1-ar3-xe-1-0-0-0.us.twtelecom.net 0.0% 9.6 9.5 9.3 13.2 0.6 6. so0-0-0.orld71-cbr2.bhn.net 0.0% 10.3 15.4 10.2 175.5 23.2 7. be3-60g.orld71-car1.bhn.net 0.0% 15.3 18.1 13.7 26.0 2.9 8. 72-31-208-54.net.bhntampa.com 0.0% 23.7 21.0 13.6 27.4 3.7 9. 72-31-208-99.net.bhntampa.com 0.0% 13.8 20.3 13.6 72.6 10.8 10. 72-31-220-9.net.bhntampa.com 0.0% 13.7 14.2 13.6 38.5 2.9 11. ??? Geoff MINA --------------------------------- Founder/CTO Connect First Inc. P: 720.335.5924 T: 888.410.3071 F: 678.265.1158 E: gmina@connectfirst.com --------------------------------- www.connectfirst.com -----Original Message----- From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Devon True Sent: Tuesday, February 14, 2012 2:10 PM To: outages@outages.org Subject: Re: [outages] Bandwidth.com outage -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 14-Feb-12 1:49 PM, Michael R. Self wrote:

I'm seeing a total outage at Bandwidth.com on their CORE 2 network for all incoming and outgoing SIP traffic.

Anybody else? So much for redundant IP's and switches on CORE 2?..

I show two originating AS numbers: #sh bgp ipv4 uni 67.231.4.195 BGP routing table entry for 67.231.4.0/24, version 116524879 Paths: (6 available, best #1, table default) Not advertised to any peer 4323 33363 [snip] 701 174 12130 [snip] 3356 2828 12130 12130 12130 Anyone else see the same thing? We have notified both 4323 and 33363. I checked both RIPE RIS and HE's bgp toolkit and both indicated that 12130 is the correct origin AS. - -- Devon -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk86zZwACgkQWP2WrBTHBS9LrgCgs5U9cdun19hDOF/V3guC34d4 +nIAn0pGz0F4XXBJ1nkDpRnGEtRdypDl =j5Jo -----END PGP SIGNATURE----- _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages