We've just had 3 links drop simultaneously to (different) equipment in Telehouse North. Fibre link to Vodafone - port is down BGP peering to GTT is dropped Copper link to BT - port is down Anyone else seeing anything? We spoke to BT and they have confirmed a "major national problem". Phil Lavin Telecoms Systems Manager CloudCall www.cloudcall.com<http://www.cloudcall.com/>
Just had this from someone: We've lost power to a full rack on the 3rd floor of THN. We've been told there are currently power issues affecting areas of the 3rd floor THN Phil Lavin Telecoms Systems Manager CloudCall www.cloudcall.com<http://www.cloudcall.com/> From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Phil Lavin via Outages Sent: 21 July 2016 08:48 To: outages@outages.org Subject: [outages] Telehouse North - Major Problems We've just had 3 links drop simultaneously to (different) equipment in Telehouse North. Fibre link to Vodafone - port is down BGP peering to GTT is dropped Copper link to BT - port is down Anyone else seeing anything? We spoke to BT and they have confirmed a "major national problem". Phil Lavin Telecoms Systems Manager CloudCall www.cloudcall.com<http://www.cloudcall.com/>
We are on the 5th floor in Telehouse North. All our backbone, peering (including LINX) and upstream links are working fine. Mark. On 21/Jul/16 09:48, Phil Lavin via Outages wrote:
We’ve just had 3 links drop simultaneously to (different) equipment in Telehouse North.
Fibre link to Vodafone – port is down
BGP peering to GTT is dropped
Copper link to BT – port is down
Anyone else seeing anything? We spoke to BT and they have confirmed a “major national problem”.
Phil Lavin
Telecoms Systems Manager
*CloudCall* www.cloudcall.com <http://www.cloudcall.com/>**
LINX looks unaffected from here - all peerings are up. Phil Lavin Telecoms Systems Manager CloudCall www.cloudcall.com<http://www.cloudcall.com/> From: Mark Tinka [mailto:mark.tinka@seacom.mu] Sent: 21 July 2016 09:00 To: Phil Lavin <phil.lavin@cloudcall.com>; Karl Putland via Outages <outages@outages.org> Subject: Re: [outages] Telehouse North - Major Problems We are on the 5th floor in Telehouse North. All our backbone, peering (including LINX) and upstream links are working fine. Mark. On 21/Jul/16 09:48, Phil Lavin via Outages wrote: We've just had 3 links drop simultaneously to (different) equipment in Telehouse North. Fibre link to Vodafone - port is down BGP peering to GTT is dropped Copper link to BT - port is down Anyone else seeing anything? We spoke to BT and they have confirmed a "major national problem". Phil Lavin Telecoms Systems Manager CloudCall www.cloudcall.com<http://www.cloudcall.com/>
BT seems to be impacted again, seeing 180ms latency to London (traffic is being routed out via AMS and back), and around 90ms to Southbank. They’re pushing a lot of traffic out via cogentco which is now suffering packet loss; My traceroute [v0.86] Cals-MacBook-Pro.local (0.0.0.0) Thu Jul 21 09:42:10 2016 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. router 0.0% 17 0.9 1.1 0.9 1.7 0.0 2. ??? 3. 31.55.187.177 87.5% 17 7.6 7.6 7.6 7.6 0.0 4. 31.55.187.176 0.0% 17 8.4 7.8 7.3 8.4 0.0 5. core2-hu0-17-0-5.southbank.ukcore.bt.net 0.0% 17 8.8 8.8 8.3 9.5 0.0 6. core1-hu0-1-0-1-1.southbank.ukcore.bt.net 0.0% 17 8.9 9.0 8.4 9.7 0.0 7. 195.99.127.68 0.0% 17 8.6 9.2 8.6 10.6 0.2 8. 213.137.183.32 0.0% 16 7.5 7.7 7.2 9.2 0.4 9. t2c3-xe-2-1-1-0.nl-ams2.eu.bt.net 0.0% 16 14.6 14.4 14.0 15.4 0.3 10. t2c4-xe-3-0-1-1.nl-ams2.eu.bt.net 0.0% 16 15.1 15.5 14.9 18.4 0.6 11. be5400.ccr21.ams04.atlas.cogentco.com 0.0% 16 113.1 105.8 93.7 114.9 6.6 12. be2312.ccr42.ams03.atlas.cogentco.com 0.0% 16 113.1 104.8 95.2 113.6 6.3 13. be12488.ccr42.lon13.atlas.cogentco.com 93.3% 16 165.8 165.8 165.8 165.8 0.0 14. be2869.ccr22.lon01.atlas.cogentco.com 80.0% 16 148.0 151.8 148.0 155.5 3.7 15. te0-0-2-2.agr11.lon01.atlas.cogentco.com 73.3% 16 156.4 159.9 148.8 167.5 8.9 16. te0-0-2-0.nr11.b015774-1.lon01.atlas.cogentco.com 87.5% 16 168.3 161.6 154.8 168.3 9.5 17. 149.11.142.246 93.3% 16 166.7 166.7 166.7 166.7 0.0 18. wokir3-xe-2-1-1-0.sgns.net 80.0% 16 151.5 155.5 151.5 157.7 3.5 19. 213.212.113.165 85.7% 15 167.2 157.1 146.9 167.2 14.3 20. ??? On 21 July 2016 at 09:02:54, Phil Lavin via Outages (outages@outages.org) wrote: LINX looks unaffected from here – all peerings are up. Phil Lavin Telecoms Systems Manager CloudCall www.cloudcall.com From: Mark Tinka [mailto:mark.tinka@seacom.mu] Sent: 21 July 2016 09:00 To: Phil Lavin <phil.lavin@cloudcall.com>; Karl Putland via Outages <outages@outages.org> Subject: Re: [outages] Telehouse North - Major Problems We are on the 5th floor in Telehouse North. All our backbone, peering (including LINX) and upstream links are working fine. Mark. On 21/Jul/16 09:48, Phil Lavin via Outages wrote: We’ve just had 3 links drop simultaneously to (different) equipment in Telehouse North. Fibre link to Vodafone – port is down BGP peering to GTT is dropped Copper link to BT – port is down Anyone else seeing anything? We spoke to BT and they have confirmed a “major national problem”. Phil Lavin Telecoms Systems Manager CloudCall www.cloudcall.com _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
More here: http://arstechnica.co.uk/business/2016/07/bt-isps-telehouse-north-major-outa ge/ Frank From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Phil Lavin via Outages Sent: Thursday, July 21, 2016 2:48 AM To: outages@outages.org Subject: [outages] Telehouse North - Major Problems We've just had 3 links drop simultaneously to (different) equipment in Telehouse North. Fibre link to Vodafone - port is down BGP peering to GTT is dropped Copper link to BT - port is down Anyone else seeing anything? We spoke to BT and they have confirmed a "major national problem". Phil Lavin Telecoms Systems Manager CloudCall www.cloudcall.com <http://www.cloudcall.com/>
participants (5)
-
Cal Leeming -
frnkblk＠iname.com -
Gary Coates -
Mark Tinka -
Phil Lavin