From our account team moments ago. Test and report folks:
"We have currently received the confirmation that we are at 100% completed with our database uploads to the SCP, We have received confirmations from the field at this time. Please have your customers retest at this time. Currently efforts are underway to ensure the service remains stable at this time." Sent from Yahoo Mail on Android From:"Frank Bulk via Outages" <outages@outages.org> Date:Thu, Sep 3, 2015 at 11:27 PM Subject:Re: [outages] Anybody else having trouble with Centurylink toll-free #s? They’ll be experts at restoring the database after this is over. =) From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Jim Witherell via Outages Sent: Thursday, September 03, 2015 10:20 PM To: outages@outages.org Subject: Re: [outages] Anybody else having trouble with Centurylink toll-free #s? From our account team: "All service went back down after we were at 90% database overall restoration and the replication error occurred for the second time. We do not have confirmation of the cause of the database reload failure, but suspect that the source of the issue is from the Highlands Ranch location. We have disabled all replication paths in each of the 5 signal control points which will isolate the Highlands Ranch location to prevent reoccurrence of the prior database reload issues. Overall database reload is at 40% overall at this time. As with previous database reloads customers are starting to restore and will continue to restore as the database reload continues. ETR is approximately is 9:45 pm MDT. The steps taken to disable all replication paths on the 5 signal control points that handle call traffic routing has been determined to be the best course of action to ensure that the database reload does not drop again." Sent from Yahoo Mail on Android From:"Jim Witherell via Outages" <outages@outages.org> Date:Thu, Sep 3, 2015 at 11:14 PM Subject:Re: [outages] Anybody else having trouble with Centurylink toll-free #s? Update: Many numbers seemed to come back for us around 10pm, but then started failing shortly after. So effectively no progress for our vintage point. For context, we have close to 200 tfn's from coast to coast. A mix of sip dedicated and switched. Sent from Yahoo Mail on Android From:"Chris Lowe via Outages" <outages@outages.org> Date:Thu, Sep 3, 2015 at 8:26 PM Subject:Re: [outages] Anybody else having trouble with Centurylink toll-free #s? You aren¹t a single point of failure as you are updating the toll-free db which controls what carrier your calls are directed to. You pay your long distance charges to the carrier that is handling the calls (so for example, today, up to the time Centurylink started having issues, you would have been paying themŠ when they started having problems and you chang your numbers to point to another carrier (Level 3 for example), you¹d now be accruing charges on Level 3, and not Centurylink). I hope that makes sense. There is nothing stopping you from becoming your own resporg. The cost is a few thousand and you would need to have at least 1 person in your company get certified, but it¹s not that hard to do. After that, you are one of roughly 500 total resporg¹s that can control toll-free numbers. Important note, you can only control the numbers in your resporg. You have no access to other resporg¹s inventory, etc. There are also monthly costs associated with the amount of numbers in your resporg. At the end of the day, you would need to decide if the total cost of ownership is worth it for your business. All the information is published on their website, http://www.sms800.com As for competitors in the independent resporg space, the only ones I know personally are custom toll-free and ATL. ATL has been around for about 20 years, and it¹s their entire business model. A quick google search shows oncall resporg services, and resporg solutions also. I do not know anyone at either of the last 2 companies, so I cannot attest to how they operate. I¹ve know people (and the owners) at ATL for over a decade now, and as I said, CTF is a subsidiary of my company. The pros of being your own resporg are that it allows you to do certain things with the origination of toll-free numbers if you want to use multiple carriers full time, and not just as a primary/backup (read: least cost origination). The biggest con is probably the price, if it¹s not a part of your normal business offering, most companies cannot justify roughly 5k to establish themselves plus the recurring costs, if they aren¹t going to use it every day, which, under the context of this thread, you wouldn¹t be using the system every day. You¹d be using it only when you needed to make a change (I assume isn¹t very often under normal circumstances). -Chris Message: 2 Date: Thu, 3 Sep 2015 16:18:12 -0700 From: Jim Witherell <jawitherell@yahoo.com> To: "outages@outages.org" <outages@outages.org> Subject: Re: [outages] Outages Digest, Vol 88, Issue 7 Message-ID: <1441322292.98726.YahooMailAndroidMobile@web163404.mail.gq1.yahoo.com> Content-Type: text/plain; charset="iso-8859-1" Shameless or not, it's on my mind. Some follow ups (which may be interesting to others, which is why I'm not PMing you...? -how do you not become a single point of failure yourself? (I assume your just updating the resporg database)? -who would I pay my long distance usage charges to? (I assume whatever carrier I'm pointing to)? -any obvious pros and cons?? -can I as a manufacturing company with, let's say, 200 or so toll free numbersnumbers be resporg? (I assume economics are relevant here)? -if you list 4 of your competitors to show you're a community minded guy, I'll PM you and ask for your sales rep to give me a pitch.? This is interesting, and for the others I promise not to drag this out on the group.? Sent from Yahoo Mail on Android From:"Chris Lowe via Outages" <outages@outages.org> Date:Thu, Sep 3, 2015 at 6:27 PM Subject:Re: [outages] Outages Digest, Vol 88, Issue 7 The only true way to mitigate is to have the ability to switch the traffic to an alternate carrier using the toll-free mgmt system (sms800).? You?d also have to keep your routing up to date on both carriers (which is usually a PIA) unless you are using the carrier as simple transport, and your company is doing all the routing/heavy lifting.? If you?re not already a RespOrg, switching the carrier on your number(s) is not an option for you.? There are companies that offer this service (managing toll-free numbers independently), Custom Toll-Free (shameless promotion as they are a subsidiary of my company), and ATL Communications are two such companies. -Chris Message: 1 Date: Thu, 3 Sep 2015 15:14:06 -0700 From: Jim Witherell <jawitherell@yahoo.com> To: "Outages@outages.org" <Outages@outages.org> Cc: outages <outages@outages.org> Subject: Re: [outages] Anybody else having trouble with??? Centurylink ??? toll-free #s? Message-ID: ??? <1441318446.40296.YahooMailAndroidMobile@web163405.mail.gq1.yahoo.com> Content-Type: text/plain; charset="utf-8" Sooooo... While we're all waiting.. Anyone have some research references on toll free outage mitigation or 'carrier redundancy"? My experience is the carrier is the single point of failure and you are absolutely 100% reliant on them. Ways to mitigate that? Whenever asked (and I just was), I say out hands are tied. I think that's true. Agree?? Sent from Yahoo Mail on Android _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages