Level3 to China Telecom Handoff issue
-- Anyone from Level3 -- First - I realize ICMP testing is not and end all be all, but seeing packet loss as well between two different carriers. Plan to potentially open tickets, but hoping for someone at Level3 to check a hop (below)? Anyone seeing or having issues with anything handing off from Level3 to china, specifically china telecom? Customer is dead in the water on a move, and at CHINA-TELEC.edge1.SanJose3.Level3.net [4.71.114.102] seeing some latency double or more from an ATT path that routes through Sprint. Customer trying to move ipsec tunnel over (and change ISP's) and cannot due to double+ the latency routing over from Level3 vs. Sprint. FYI - this issue has been the same for a week or more. Dont have any specific times, but customer calling me during the day EST to look at this and it's been the same every day... Two different traces below (I cut some out as the first hops are fine, it's the hand off).. First, the "good": 10 19 ms 19 ms 19 ms 144.232.1.73 11 24 ms 19 ms 20 ms 144.232.1.73 12 59 ms 60 ms 59 ms 144.232.11.17 13 60 ms 60 ms 61 ms sl-crs1-stk-0-0-0-3.sprintlink.net [144.232.8.167] 14 61 ms 60 ms 61 ms sl-crs1-ria-0-2-2-0.sprintlink.net [144.232.9.152] 15 61 ms 61 ms 61 ms 144.232.2.84 16 62 ms 62 ms 62 ms sl-st25-la-.sprintlink.net [144.232.9.20] 17 61 ms 61 ms 61 ms sl-china1-356229-0.sprintlink.net [144.228.79.118] Now the "bad": 10 22 ms 22 ms 22 ms ae-6-6.ebr1.Chicago2.Level3.net [4.69.140.190] 11 55 ms 49 ms 51 ms ae-3-3.ebr2.Denver1.Level3.net [4.69.132.61] 12 46 ms 46 ms 46 ms ae-1-100.ebr1.Denver1.Level3.net [4.69.151.181] 13 72 ms 72 ms 71 ms ae-3-3.ebr2.SanJose1.Level3.net [4.69.132.57] 14 82 ms 74 ms 74 ms ae-82-82.csw3.SanJose1.Level3.net [4.69.153.26] 15 72 ms 72 ms 72 ms ae-3-80.edge1.SanJose3.Level3.net [4.69.152.144] 16 172 ms 171 ms 171 ms CHINA-TELEC.edge1.SanJose3.Level3.net [4.71.114.102] Thanks to anyone that can check / verify this, or simply tell me - "Normal - good luck"...
I opened a ticket with Level 3 about 2 months ago on this same issue. I think I also replied to another similar question on this list. There is a very high level of utilization on this link. Level 3 is aware of the high utilization and has been for a while. I used BGP knobs to route specific prefixes that were in China to a Sprint circuit and performance was far better. Good luck. Tim From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Dave Samic Sent: Thursday, December 13, 2012 1:14 PM To: outages@outages.org Subject: [outages] Level3 to China Telecom Handoff issue -- Anyone from Level3 -- First - I realize ICMP testing is not and end all be all, but seeing packet loss as well between two different carriers. Plan to potentially open tickets, but hoping for someone at Level3 to check a hop (below)? Anyone seeing or having issues with anything handing off from Level3 to china, specifically china telecom? Customer is dead in the water on a move, and at CHINA-TELEC.edge1.SanJose3.Level3.net [4.71.114.102] seeing some latency double or more from an ATT path that routes through Sprint. Customer trying to move ipsec tunnel over (and change ISP's) and cannot due to double+ the latency routing over from Level3 vs. Sprint. FYI - this issue has been the same for a week or more. Dont have any specific times, but customer calling me during the day EST to look at this and it's been the same every day... Two different traces below (I cut some out as the first hops are fine, it's the hand off).. First, the "good": 10 19 ms 19 ms 19 ms 144.232.1.73 11 24 ms 19 ms 20 ms 144.232.1.73 12 59 ms 60 ms 59 ms 144.232.11.17 13 60 ms 60 ms 61 ms sl-crs1-stk-0-0-0-3.sprintlink.net [144.232.8.167] 14 61 ms 60 ms 61 ms sl-crs1-ria-0-2-2-0.sprintlink.net [144.232.9.152] 15 61 ms 61 ms 61 ms 144.232.2.84 16 62 ms 62 ms 62 ms sl-st25-la-.sprintlink.net [144.232.9.20] 17 61 ms 61 ms 61 ms sl-china1-356229-0.sprintlink.net [144.228.79.118] Now the "bad": 10 22 ms 22 ms 22 ms ae-6-6.ebr1.Chicago2.Level3.net [4.69.140.190] 11 55 ms 49 ms 51 ms ae-3-3.ebr2.Denver1.Level3.net [4.69.132.61] 12 46 ms 46 ms 46 ms ae-1-100.ebr1.Denver1.Level3.net [4.69.151.181] 13 72 ms 72 ms 71 ms ae-3-3.ebr2.SanJose1.Level3.net [4.69.132.57] 14 82 ms 74 ms 74 ms ae-82-82.csw3.SanJose1.Level3.net [4.69.153.26] 15 72 ms 72 ms 72 ms ae-3-80.edge1.SanJose3.Level3.net [4.69.152.144] 16 172 ms 171 ms 171 ms CHINA-TELEC.edge1.SanJose3.Level3.net [4.71.114.102] Thanks to anyone that can check / verify this, or simply tell me - "Normal - good luck"...
We've been through this with Level3 as well. They point to China Telcom as being the hold-up on increasing the capacity between the two. On Thu, Dec 13, 2012 at 12:13 PM, Dave Samic <dave.samic@paragrid.net>wrote:
-- Anyone from Level3 --****
** **
First - I realize ICMP testing is not and end all be all, but seeing packet loss as well between two different carriers. Plan to potentially open tickets, but hoping for someone at Level3 to check a hop (below)?****
** **
Anyone seeing or having issues with anything handing off from Level3 to china, specifically china telecom?****
** **
Customer is dead in the water on a move, and at CHINA-TELEC.edge1.SanJose3.Level3.net [4.71.114.102] seeing some latency double or more from an ATT path that routes through Sprint. Customer trying to move ipsec tunnel over (and change ISP's) and cannot due to double+ the latency routing over from Level3 vs. Sprint.****
** **
FYI - this issue has been the same for a week or more. Dont have any specific times, but customer calling me during the day EST to look at this and it's been the same every day...****
** **
Two different traces below (I cut some out as the first hops are fine, it's the hand off)..****
** **
First, the "good":****
** **
10 19 ms 19 ms 19 ms 144.232.1.73****
11 24 ms 19 ms 20 ms 144.232.1.73****
12 59 ms 60 ms 59 ms 144.232.11.17****
13 60 ms 60 ms 61 ms sl-crs1-stk-0-0-0-3.sprintlink.net ****
[144.232.8.167]****
14 61 ms 60 ms 61 ms sl-crs1-ria-0-2-2-0.sprintlink.net ****
[144.232.9.152]****
15 61 ms 61 ms 61 ms 144.232.2.84****
16 62 ms 62 ms 62 ms sl-st25-la-.sprintlink.net [144.232.9.20] ****
17 61 ms 61 ms 61 ms sl-china1-356229-0.sprintlink.net ****
[144.228.79.118]****
** **
Now the "bad":****
** **
10 22 ms 22 ms 22 ms ae-6-6.ebr1.Chicago2.Level3.net ****
[4.69.140.190]****
11 55 ms 49 ms 51 ms ae-3-3.ebr2.Denver1.Level3.net[4.69.132.61] ****
12 46 ms 46 ms 46 ms ae-1-100.ebr1.Denver1.Level3.net ****
[4.69.151.181]****
13 72 ms 72 ms 71 ms ae-3-3.ebr2.SanJose1.Level3.net[4.69.132.57] ****
14 82 ms 74 ms 74 ms ae-82-82.csw3.SanJose1.Level3.net ****
[4.69.153.26]****
15 72 ms 72 ms 72 ms ae-3-80.edge1.SanJose3.Level3.net ****
[4.69.152.144]****
16 172 ms 171 ms 171 ms CHINA-TELEC.edge1.SanJose3.Level3.net ****
[4.71.114.102]****
** **
Thanks to anyone that can check / verify this, or simply tell me - "Normal - good luck"...****
** **
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Peachy :) Thanks everyone for the quick responses... From: Jason Baugher [mailto:jason@thebaughers.com] Sent: Thursday, December 13, 2012 13:27 To: Dave Samic Cc: outages@outages.org Subject: Re: [outages] Level3 to China Telecom Handoff issue We've been through this with Level3 as well. They point to China Telcom as being the hold-up on increasing the capacity between the two. On Thu, Dec 13, 2012 at 12:13 PM, Dave Samic <dave.samic@paragrid.net<mailto:dave.samic@paragrid.net>> wrote: -- Anyone from Level3 -- First - I realize ICMP testing is not and end all be all, but seeing packet loss as well between two different carriers. Plan to potentially open tickets, but hoping for someone at Level3 to check a hop (below)? Anyone seeing or having issues with anything handing off from Level3 to china, specifically china telecom? Customer is dead in the water on a move, and at CHINA-TELEC.edge1.SanJose3.Level3.net<http://CHINA-TELEC.edge1.SanJose3.Level3.net> [4.71.114.102] seeing some latency double or more from an ATT path that routes through Sprint. Customer trying to move ipsec tunnel over (and change ISP's) and cannot due to double+ the latency routing over from Level3 vs. Sprint. FYI - this issue has been the same for a week or more. Dont have any specific times, but customer calling me during the day EST to look at this and it's been the same every day... Two different traces below (I cut some out as the first hops are fine, it's the hand off).. First, the "good": 10 19 ms 19 ms 19 ms 144.232.1.73 11 24 ms 19 ms 20 ms 144.232.1.73 12 59 ms 60 ms 59 ms 144.232.11.17 13 60 ms 60 ms 61 ms sl-crs1-stk-0-0-0-3.sprintlink.net<http://sl-crs1-stk-0-0-0-3.sprintlink.net> [144.232.8.167] 14 61 ms 60 ms 61 ms sl-crs1-ria-0-2-2-0.sprintlink.net<http://sl-crs1-ria-0-2-2-0.sprintlink.net> [144.232.9.152] 15 61 ms 61 ms 61 ms 144.232.2.84 16 62 ms 62 ms 62 ms sl-st25-la-.sprintlink.net<http://sprintlink.net> [144.232.9.20] 17 61 ms 61 ms 61 ms sl-china1-356229-0.sprintlink.net<http://sl-china1-356229-0.sprintlink.net> [144.228.79.118<tel:%5B144.228.79.118>] Now the "bad": 10 22 ms 22 ms 22 ms ae-6-6.ebr1.Chicago2.Level3.net<http://ae-6-6.ebr1.Chicago2.Level3.net> [4.69.140.190] 11 55 ms 49 ms 51 ms ae-3-3.ebr2.Denver1.Level3.net<http://ae-3-3.ebr2.Denver1.Level3.net> [4.69.132.61] 12 46 ms 46 ms 46 ms ae-1-100.ebr1.Denver1.Level3.net<http://ae-1-100.ebr1.Denver1.Level3.net> [4.69.151.181] 13 72 ms 72 ms 71 ms ae-3-3.ebr2.SanJose1.Level3.net<http://ae-3-3.ebr2.SanJose1.Level3.net> [4.69.132.57] 14 82 ms 74 ms 74 ms ae-82-82.csw3.SanJose1.Level3.net<http://ae-82-82.csw3.SanJose1.Level3.net> [4.69.153.26] 15 72 ms 72 ms 72 ms ae-3-80.edge1.SanJose3.Level3.net<http://ae-3-80.edge1.SanJose3.Level3.net> [4.69.152.144] 16 172 ms 171 ms 171 ms CHINA-TELEC.edge1.SanJose3.Level3.net<http://CHINA-TELEC.edge1.SanJose3.Level3.net> [4.71.114.102] Thanks to anyone that can check / verify this, or simply tell me - "Normal - good luck"... _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages
participants (3)
-
Dave Samic -
Jason Baugher -
Tim Glen