Cogent /ATT and Verizon Peering alternatives

We are seeing lots of problems with our audio streaming traffic dropping lots of packets where our transit provider Cogent hands off to ATT & Verizon. Right now, I'm seeing 43% packet loss Cogent is blaming ATT/Verizon and says there is nothing they can do about it. We're the collateral damage. Who else is having the same issue? I threaten to cancel my service, Cogent says, "sorry to lose you as a customer", and go on their merry way. Are other transit providers having issues peering / getting to ATT/Verizon? I need to know who to avoid. traceroute to 71.252.251.1 (71.252.251.1), 30 hops max, 60 byte packets 1 gi3-11.ccr01.sfo02.atlas.cogentco.com (38.104.130.89) 0.982 ms 0.985 ms 1.195 ms 2 te0-0-0-28.ccr22.sfo01.atlas.cogentco.com (154.54.3.129) 0.531 ms te0-0-0-28.ccr21.sfo01.atlas.cogentco.com (154.54.3.121) 0.604 ms 0.636 ms 3 be2165.ccr22.sjc01.atlas.cogentco.com (154.54.28.66) 2.277 ms be2164.ccr21.sjc01.atlas.cogentco.com (154.54.28.34) 1.806 ms 2.449 ms 4 be2000.ccr21.sjc03.atlas.cogentco.com (154.54.6.106) 3.824 ms 2.504 ms 2.596 ms 5 * * verizon.sjc03.atlas.cogentco.com (154.54.12.170) 94.373 ms 6 * * * 7 * * * 8 L100.DLLSTX-VFTTP-64.verizon-gni.net (71.252.251.1) 135.242 ms 86.272 ms 86.231 ms

I'm seeing anywhere from 7-33% packet loss on cognet n phoenix to your destination. It averages around 9% though, Most of the packet loss happens on the last run before it hits verizon. mtr 71.252.251.1 Host Loss% Snt Last Avg Best Wrst StDev 1. - 0.0% 42 0.4 0.7 0.3 7.2 1.1 2. gi1-3.ccr01 4.8% 42 0.9 3.1 0.7 32.6 6.5 3. te0-4-1-5.c 0.0% 42 1.4 1.6 1.2 5.4 0.8 4. be2333.mpd2 2.4% 41 14.3 13.4 12.8 18.3 1.0 5. be2178.ccr2 0.0% 41 13.9 14.7 13.8 19.9 1.3 6. uunet.lax05 7.3% 41 53.6 55.0 52.1 73.7 4.1 7. P0-0-0-0.DL 9.8% 41 84.9 84.4 81.6 91.6 1.9 8. ??? 9. L100.DLLSTX 9.8% 41 83.5 84.1 82.1 87.4 0.9 traceroute to 71.252.251.1 (71.252.251.1), 30 hops max, 60 byte packets 1 - 2.799 ms 2.583 ms 3.022 ms 2 gi1-3.ccr01.phx05.atlas.cogentco.com (38.122.88.65) 3.014 ms 3.006 ms 2.999 ms 3 te0-4-0-5.ccr21.phx02.atlas.cogentco.com (154.54.84.221) 3.488 ms 154.54.89.109 (154.54.89.109) 4.160 ms 154.54.89.105 (154.54.89.105) 3.468 ms 4 be2333.mpd22.lax01.atlas.cogentco.com (154.54.41.249) 15.542 ms be2330.ccr21.lax01.atlas.cogentco.com (154.54.30.189) 14.962 ms be2333.mpd22.lax01.atlas.cogentco.com (154.54.41.249) 15.032 ms 5 be2180.ccr23.lax05.atlas.cogentco.com (154.54.41.58) 16.159 ms be2181.ccr23.lax05.atlas.cogentco.com (154.54.41.114) 15.435 ms 14.948 ms 6 uunet.lax05.atlas.cogentco.com (154.54.13.86) 65.357 ms 62.954 ms 95.462 ms 7 P0-8-0-0.DLLSTX-LCR-22.verizon-gni.net (130.81.199.37) 83.373 ms P0-0-0-0.DLLSTX-LCR-21.verizon-gni.net (130.81.151.243) 82.127 ms 86.080 ms 8 * * * 9 L100.DLLSTX-VFTTP-64.verizon-gni.net (71.252.251.1) 84.306 ms 84.674 ms 85.160 ms On Mon, Feb 10, 2014 at 6:02 PM, Rusty Hodge <rusty@hodge.com> wrote:
We are seeing lots of problems with our audio streaming traffic dropping lots of packets where our transit provider Cogent hands off to ATT & Verizon.
Right now, I'm seeing 43% packet loss
Cogent is blaming ATT/Verizon and says there is nothing they can do about it. We're the collateral damage.
Who else is having the same issue?
I threaten to cancel my service, Cogent says, "sorry to lose you as a customer", and go on their merry way.
Are other transit providers having issues peering / getting to ATT/Verizon? I need to know who to avoid.
traceroute to 71.252.251.1 (71.252.251.1), 30 hops max, 60 byte packets 1 gi3-11.ccr01.sfo02.atlas.cogentco.com (38.104.130.89) 0.982 ms 0.985 ms 1.195 ms 2 te0-0-0-28.ccr22.sfo01.atlas.cogentco.com (154.54.3.129) 0.531 ms te0-0-0-28.ccr21.sfo01.atlas.cogentco.com (154.54.3.121) 0.604 ms 0.636 ms 3 be2165.ccr22.sjc01.atlas.cogentco.com (154.54.28.66) 2.277 ms be2164.ccr21.sjc01.atlas.cogentco.com (154.54.28.34) 1.806 ms 2.449 ms 4 be2000.ccr21.sjc03.atlas.cogentco.com (154.54.6.106) 3.824 ms 2.504 ms 2.596 ms 5 * * verizon.sjc03.atlas.cogentco.com (154.54.12.170) 94.373 ms 6 * * * 7 * * * 8 L100.DLLSTX-VFTTP-64.verizon-gni.net (71.252.251.1) 135.242 ms 86.272 ms 86.231 ms
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

On 2/10/14, Rusty Hodge <rusty@hodge.com> wrote:
We are seeing lots of problems with our audio streaming traffic dropping lots of packets where our transit provider Cogent hands off to ATT & Verizon.
Right now, I'm seeing 43% packet loss
Cogent is blaming ATT/Verizon and says there is nothing they can do about it. We're the collateral damage.
Who else is having the same issue?
I threaten to cancel my service, Cogent says, "sorry to lose you as a customer", and go on their merry way.
Are other transit providers having issues peering / getting to ATT/Verizon? I need to know who to avoid.
traceroute to 71.252.251.1 (71.252.251.1), 30 hops max, 60 byte packets 1 gi3-11.ccr01.sfo02.atlas.cogentco.com (38.104.130.89) 0.982 ms 0.985 ms 1.195 ms 2 te0-0-0-28.ccr22.sfo01.atlas.cogentco.com (154.54.3.129) 0.531 ms te0-0-0-28.ccr21.sfo01.atlas.cogentco.com (154.54.3.121) 0.604 ms 0.636 ms 3 be2165.ccr22.sjc01.atlas.cogentco.com (154.54.28.66) 2.277 ms be2164.ccr21.sjc01.atlas.cogentco.com (154.54.28.34) 1.806 ms 2.449 ms 4 be2000.ccr21.sjc03.atlas.cogentco.com (154.54.6.106) 3.824 ms 2.504 ms 2.596 ms 5 * * verizon.sjc03.atlas.cogentco.com (154.54.12.170) 94.373 ms 6 * * * 7 * * * 8 L100.DLLSTX-VFTTP-64.verizon-gni.net (71.252.251.1) 135.242 ms 86.272 ms 86.231 ms
If it helps any - traceroute from Verizon home FIOS to you & to your destination C:\>tracert 38.104.130.89 Tracing route to gi3-11.ccr01.sfo02.atlas.cogentco.com [38.104.130.89] over a maximum of 30 hops: [.. snip ..] 5 10 ms 7 ms 7 ms G1-6-0-6.WASHDC-LCR-21.verizon-gni.net [130.81.217.150] 6 9 ms 8 ms 7 ms so-12-1-0-0.RES-BB-RTR1.verizon-gni.net [130.81.151.230] 7 10 ms 9 ms 9 ms 0.xe-9-0-0.BR2.IAD8.ALTER.NET [152.63.38.133] 8 * 85 ms * te0-1-0-14.ccr41.iad02.atlas.cogentco.com [154.54.12.45] 9 83 ms 82 ms 82 ms be2113.mpd21.dca01.atlas.cogentco.com [154.54.6.170] 10 89 ms 89 ms 92 ms be2154.mpd21.ord01.atlas.cogentco.com [154.54.30.198] 11 90 ms 89 ms 90 ms be2158.mpd21.mci01.atlas.cogentco.com [154.54.7.129] 12 126 ms 120 ms 119 ms be2256.ccr21.sfo01.atlas.cogentco.com [154.54.6.89] 13 91 ms 87 ms 87 ms gi3-11.ccr01.sfo02.atlas.cogentco.com [38.104.130.89] Trace complete. C:\>tracert 71.252.251.1 Tracing route to L100.DLLSTX-VFTTP-64.verizon-gni.net [71.252.251.1] over a maximum of 30 hops: [.. snip ..] 5 9 ms 7 ms 10 ms G1-6-0-5.WASHDC-LCR-21.verizon-gni.net [130.81.217.148] 6 54 ms 7 ms 7 ms ae4-0.RES-BB-RTR1.verizon-gni.net [130.81.199.122] 7 47 ms 47 ms 47 ms xe-4-0-3-0.DFW9-BB-RTR1.verizon-gni.net [130.81.23.217] 8 53 ms 49 ms 47 ms P0-8-0-0.DLLSTX-LCR-21.verizon-gni.net [130.81.199.35] 9 * * * Request timed out. 10 51 ms 49 ms 51 ms L100.DLLSTX-VFTTP-64.verizon-gni.net [71.252.251.1] Trace complete. Regards, Lee

On Feb 10, 2014, at 8:02 PM, Rusty Hodge wrote:
We are seeing lots of problems with our audio streaming traffic dropping lots of packets where our transit provider Cogent hands off to ATT & Verizon.
Right now, I'm seeing 43% packet loss
Cogent is blaming ATT/Verizon and says there is nothing they can do about it. We're the collateral damage.
Isn't this the result of VZ/Comcast/ATT and others refusing peering with Netflix and Netflix purchasing transit via Cogent to reach them? That standoff has been going on for some time, and I would guess with the recent court ruling regarding "net neutrality", it's not going to get better until Netflix pays VZ and other large eyeball networks to carry their traffic (or pays twice, depending on your perspective). Charles timely news item: http://arstechnica.com/information-technology/2014/02/netflix-performance-on...
Who else is having the same issue?
I threaten to cancel my service, Cogent says, "sorry to lose you as a customer", and go on their merry way.
Are other transit providers having issues peering / getting to ATT/Verizon? I need to know who to avoid.
traceroute to 71.252.251.1 (71.252.251.1), 30 hops max, 60 byte packets 1 gi3-11.ccr01.sfo02.atlas.cogentco.com (38.104.130.89) 0.982 ms 0.985 ms 1.195 ms 2 te0-0-0-28.ccr22.sfo01.atlas.cogentco.com (154.54.3.129) 0.531 ms te0-0-0-28.ccr21.sfo01.atlas.cogentco.com (154.54.3.121) 0.604 ms 0.636 ms 3 be2165.ccr22.sjc01.atlas.cogentco.com (154.54.28.66) 2.277 ms be2164.ccr21.sjc01.atlas.cogentco.com (154.54.28.34) 1.806 ms 2.449 ms 4 be2000.ccr21.sjc03.atlas.cogentco.com (154.54.6.106) 3.824 ms 2.504 ms 2.596 ms 5 * * verizon.sjc03.atlas.cogentco.com (154.54.12.170) 94.373 ms 6 * * * 7 * * * 8 L100.DLLSTX-VFTTP-64.verizon-gni.net (71.252.251.1) 135.242 ms 86.272 ms 86.231 ms
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

On 11 February 2014 01:56, Charles Sprickman <spork@bway.net> wrote:
On Feb 10, 2014, at 8:02 PM, Rusty Hodge wrote:
We are seeing lots of problems with our audio streaming traffic dropping lots of packets where our transit provider Cogent hands off to ATT & Verizon.
Right now, I'm seeing 43% packet loss
Cogent is blaming ATT/Verizon and says there is nothing they can do about it. We're the collateral damage.
Isn't this the result of VZ/Comcast/ATT and others refusing peering with Netflix and Netflix purchasing transit via Cogent to reach them?
Its more complicated than that - Netflix pays CDNs, transits etc for delivery of much of its content. The original email as I understood was a user seeing packet loss between their business Internet service with Verizon and either a server hosted within Cogent or another Internet access service. ie an end user trying to connect with another end user/host. As such I don't think its about Netflix, but this end user who is paying two providers neither of which are delivering. I would complain to both. I'd also try moving - preferably at both sides. Steve
That standoff has been going on for some time, and I would guess with the recent court ruling regarding "net neutrality", it's not going to get better until Netflix pays VZ and other large eyeball networks to carry their traffic (or pays twice, depending on your perspective).
Charles
timely news item:
http://arstechnica.com/information-technology/2014/02/netflix-performance-on...
Who else is having the same issue?
I threaten to cancel my service, Cogent says, "sorry to lose you as a
customer", and go on their merry way.
Are other transit providers having issues peering / getting to
ATT/Verizon? I need to know who to avoid.
traceroute to 71.252.251.1 (71.252.251.1), 30 hops max, 60 byte packets 1 gi3-11.ccr01.sfo02.atlas.cogentco.com (38.104.130.89) 0.982 ms
0.985 ms 1.195 ms
2 te0-0-0-28.ccr22.sfo01.atlas.cogentco.com (154.54.3.129) 0.531 ms te0-0-0-28.ccr21.sfo01.atlas.cogentco.com (154.54.3.121) 0.604 ms 0.636 ms 3 be2165.ccr22.sjc01.atlas.cogentco.com (154.54.28.66) 2.277 ms be2164.ccr21.sjc01.atlas.cogentco.com (154.54.28.34) 1.806 ms 2.449 ms 4 be2000.ccr21.sjc03.atlas.cogentco.com (154.54.6.106) 3.824 ms 2.504 ms 2.596 ms 5 * * verizon.sjc03.atlas.cogentco.com (154.54.12.170) 94.373 ms 6 * * * 7 * * * 8 L100.DLLSTX-VFTTP-64.verizon-gni.net (71.252.251.1) 135.242 ms 86.272 ms 86.231 ms
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

On Feb 10, 2014, at 9:12 PM, Stephen Wilcox wrote:
On 11 February 2014 01:56, Charles Sprickman <spork@bway.net> wrote:
On Feb 10, 2014, at 8:02 PM, Rusty Hodge wrote:
We are seeing lots of problems with our audio streaming traffic dropping lots of packets where our transit provider Cogent hands off to ATT & Verizon.
Right now, I'm seeing 43% packet loss
Cogent is blaming ATT/Verizon and says there is nothing they can do about it. We're the collateral damage.
Isn't this the result of VZ/Comcast/ATT and others refusing peering with Netflix and Netflix purchasing transit via Cogent to reach them?
Its more complicated than that - Netflix pays CDNs, transits etc for delivery of much of its content.
And VZ wants some payment from Netflix as well - most likely by selling Netflix transit I suppose.
The original email as I understood was a user seeing packet loss between their business Internet service with Verizon and either a server hosted within Cogent or another Internet access service. ie an end user trying to connect with another end user/host. As such I don't think its about Netflix, but this end user who is paying two providers neither of which are delivering.
Netflix's overall traffic volume is huge. If Cogent is in fact the path of last resort to Verizon for Netflix, and VZ and Cogent are having a spat, this would obviously impact everyone looking to move traffic between VZ and Cogent, not just Netflix customers.
I would complain to both. I'd also try moving - preferably at both sides.
On the residential side, you have a duopoly if you're lucky, moving isn't an option for everyone. And if Comcast is also having issues, then you've exhausted both of your options (cable and FiOS). Total speculation, but seeing the complaints from FiOS users about Netflix, Youtube and Amazon makes me think VZ is really itching to test the new court ruling. All three of those "problem networks" have great peering terms, yet VZ, from what I can see does not peer with any of them. On our network, I see more than 50% of our traffic lands on Google or Amazon - as an ISP, it's pretty bold to decide to reach those networks via anything but a peering relationship. Charles
Steve
That standoff has been going on for some time, and I would guess with the recent court ruling regarding "net neutrality", it's not going to get better until Netflix pays VZ and other large eyeball networks to carry their traffic (or pays twice, depending on your perspective).
Charles
timely news item:
http://arstechnica.com/information-technology/2014/02/netflix-performance-on...
Who else is having the same issue?
I threaten to cancel my service, Cogent says, "sorry to lose you as a customer", and go on their merry way.
Are other transit providers having issues peering / getting to ATT/Verizon? I need to know who to avoid.
traceroute to 71.252.251.1 (71.252.251.1), 30 hops max, 60 byte packets 1 gi3-11.ccr01.sfo02.atlas.cogentco.com (38.104.130.89) 0.982 ms 0.985 ms 1.195 ms 2 te0-0-0-28.ccr22.sfo01.atlas.cogentco.com (154.54.3.129) 0.531 ms te0-0-0-28.ccr21.sfo01.atlas.cogentco.com (154.54.3.121) 0.604 ms 0.636 ms 3 be2165.ccr22.sjc01.atlas.cogentco.com (154.54.28.66) 2.277 ms be2164.ccr21.sjc01.atlas.cogentco.com (154.54.28.34) 1.806 ms 2.449 ms 4 be2000.ccr21.sjc03.atlas.cogentco.com (154.54.6.106) 3.824 ms 2.504 ms 2.596 ms 5 * * verizon.sjc03.atlas.cogentco.com (154.54.12.170) 94.373 ms 6 * * * 7 * * * 8 L100.DLLSTX-VFTTP-64.verizon-gni.net (71.252.251.1) 135.242 ms 86.272 ms 86.231 ms
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

On 11 February 2014 20:10, Charles Sprickman <spork@bway.net> wrote:
On Feb 10, 2014, at 9:12 PM, Stephen Wilcox wrote:
On 11 February 2014 01:56, Charles Sprickman <spork@bway.net> wrote:
On Feb 10, 2014, at 8:02 PM, Rusty Hodge wrote:
We are seeing lots of problems with our audio streaming traffic dropping lots of packets where our transit provider Cogent hands off to ATT & Verizon.
Right now, I'm seeing 43% packet loss
Cogent is blaming ATT/Verizon and says there is nothing they can do about it. We're the collateral damage.
Isn't this the result of VZ/Comcast/ATT and others refusing peering with Netflix and Netflix purchasing transit via Cogent to reach them?
Its more complicated than that - Netflix pays CDNs, transits etc for delivery of much of its content.
And VZ wants some payment from Netflix as well - most likely by selling Netflix transit I suppose.
The original email as I understood was a user seeing packet loss between their business Internet service with Verizon and either a server hosted within Cogent or another Internet access service. ie an end user trying to connect with another end user/host. As such I don't think its about Netflix, but this end user who is paying two providers neither of which are delivering.
Netflix's overall traffic volume is huge. If Cogent is in fact the path of last resort to Verizon for Netflix, and VZ and Cogent are having a spat, this would obviously impact everyone looking to move traffic between VZ and Cogent, not just Netflix customers.
I would complain to both. I'd also try moving - preferably at both sides.
On the residential side, you have a duopoly if you're lucky, moving isn't an option for everyone. And if Comcast is also having issues, then you've exhausted both of your options (cable and FiOS).
Total speculation, but seeing the complaints from FiOS users about Netflix, Youtube and Amazon makes me think VZ is really itching to test the new court ruling. All three of those "problem networks" have great peering terms, yet VZ, from what I can see does not peer with any of them. On our network, I see more than 50% of our traffic lands on Google or Amazon - as an ISP, it's pretty bold to decide to reach those networks via anything but a peering relationship.
Fully agree with the concerns against VZ (altho don't blame on maligned intent what can be explained through incompetence either). On the other hand, Netflix have a choice of upstreams and CDNs and Cogent doesn't have to accept business that it can't fulfill. It could also mitigate against the service problems that other customers see in the same way anyone deals with a DDoS affecting their peers ie throttle, reduce the problem traffic. Hence why, I would where possible, move to more responsible providers who don't still think its the 1990s and that "Tier1s" are the core of the Internet (as you point out out content is > 50% of the Internet and is the network core of the 2010s). Steve
Charles
Steve
That standoff has been going on for some time, and I would guess with the recent court ruling regarding "net neutrality", it's not going to get better until Netflix pays VZ and other large eyeball networks to carry their traffic (or pays twice, depending on your perspective).
Charles
timely news item:
http://arstechnica.com/information-technology/2014/02/netflix-performance-on...
Who else is having the same issue?
I threaten to cancel my service, Cogent says, "sorry to lose you as a
customer", and go on their merry way.
Are other transit providers having issues peering / getting to
ATT/Verizon? I need to know who to avoid.
traceroute to 71.252.251.1 (71.252.251.1), 30 hops max, 60 byte packets 1 gi3-11.ccr01.sfo02.atlas.cogentco.com (38.104.130.89) 0.982 ms
0.985 ms 1.195 ms
2 te0-0-0-28.ccr22.sfo01.atlas.cogentco.com (154.54.3.129) 0.531 ms te0-0-0-28.ccr21.sfo01.atlas.cogentco.com (154.54.3.121) 0.604 ms 0.636 ms 3 be2165.ccr22.sjc01.atlas.cogentco.com (154.54.28.66) 2.277 ms be2164.ccr21.sjc01.atlas.cogentco.com (154.54.28.34) 1.806 ms 2.449 ms 4 be2000.ccr21.sjc03.atlas.cogentco.com (154.54.6.106) 3.824 ms 2.504 ms 2.596 ms 5 * * verizon.sjc03.atlas.cogentco.com (154.54.12.170) 94.373 ms 6 * * * 7 * * * 8 L100.DLLSTX-VFTTP-64.verizon-gni.net (71.252.251.1) 135.242 ms 86.272 ms 86.231 ms
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

We have been collateral damage in this fight for a long time too, we adjusted our routing for a VZN traffic to transit our US Signal link to resolve this issue. --- Ben Mason email: su1droot@gmail.com web: http://www.benmasonphoto.com gchat: su1droot@gmail.com On Tue, Feb 11, 2014 at 3:44 PM, Stephen Wilcox <steve.wilcox@ixreach.com>wrote:
On 11 February 2014 20:10, Charles Sprickman <spork@bway.net> wrote:
On Feb 10, 2014, at 9:12 PM, Stephen Wilcox wrote:
On 11 February 2014 01:56, Charles Sprickman <spork@bway.net> wrote:
On Feb 10, 2014, at 8:02 PM, Rusty Hodge wrote:
We are seeing lots of problems with our audio streaming traffic dropping lots of packets where our transit provider Cogent hands off to ATT & Verizon.
Right now, I'm seeing 43% packet loss
Cogent is blaming ATT/Verizon and says there is nothing they can do about it. We're the collateral damage.
Isn't this the result of VZ/Comcast/ATT and others refusing peering with Netflix and Netflix purchasing transit via Cogent to reach them?
Its more complicated than that - Netflix pays CDNs, transits etc for delivery of much of its content.
And VZ wants some payment from Netflix as well - most likely by selling Netflix transit I suppose.
The original email as I understood was a user seeing packet loss between their business Internet service with Verizon and either a server hosted within Cogent or another Internet access service. ie an end user trying to connect with another end user/host. As such I don't think its about Netflix, but this end user who is paying two providers neither of which are delivering.
Netflix's overall traffic volume is huge. If Cogent is in fact the path of last resort to Verizon for Netflix, and VZ and Cogent are having a spat, this would obviously impact everyone looking to move traffic between VZ and Cogent, not just Netflix customers.
I would complain to both. I'd also try moving - preferably at both sides.
On the residential side, you have a duopoly if you're lucky, moving isn't an option for everyone. And if Comcast is also having issues, then you've exhausted both of your options (cable and FiOS).
Total speculation, but seeing the complaints from FiOS users about Netflix, Youtube and Amazon makes me think VZ is really itching to test the new court ruling. All three of those "problem networks" have great peering terms, yet VZ, from what I can see does not peer with any of them. On our network, I see more than 50% of our traffic lands on Google or Amazon - as an ISP, it's pretty bold to decide to reach those networks via anything but a peering relationship.
Fully agree with the concerns against VZ (altho don't blame on maligned intent what can be explained through incompetence either).
On the other hand, Netflix have a choice of upstreams and CDNs and Cogent doesn't have to accept business that it can't fulfill. It could also mitigate against the service problems that other customers see in the same way anyone deals with a DDoS affecting their peers ie throttle, reduce the problem traffic.
Hence why, I would where possible, move to more responsible providers who don't still think its the 1990s and that "Tier1s" are the core of the Internet (as you point out out content is > 50% of the Internet and is the network core of the 2010s).
Steve
Charles
Steve
That standoff has been going on for some time, and I would guess with the recent court ruling regarding "net neutrality", it's not going to get better until Netflix pays VZ and other large eyeball networks to carry their traffic (or pays twice, depending on your perspective).
Charles
timely news item:
http://arstechnica.com/information-technology/2014/02/netflix-performance-on...
Who else is having the same issue?
I threaten to cancel my service, Cogent says, "sorry to lose you as a
customer", and go on their merry way.
Are other transit providers having issues peering / getting to
ATT/Verizon? I need to know who to avoid.
traceroute to 71.252.251.1 (71.252.251.1), 30 hops max, 60 byte packets 1 gi3-11.ccr01.sfo02.atlas.cogentco.com (38.104.130.89) 0.982 ms
0.985 ms 1.195 ms
2 te0-0-0-28.ccr22.sfo01.atlas.cogentco.com (154.54.3.129) 0.531 ms te0-0-0-28.ccr21.sfo01.atlas.cogentco.com (154.54.3.121) 0.604 ms 0.636 ms 3 be2165.ccr22.sjc01.atlas.cogentco.com (154.54.28.66) 2.277 ms be2164.ccr21.sjc01.atlas.cogentco.com (154.54.28.34) 1.806 ms 2.449 ms 4 be2000.ccr21.sjc03.atlas.cogentco.com (154.54.6.106) 3.824 ms 2.504 ms 2.596 ms 5 * * verizon.sjc03.atlas.cogentco.com (154.54.12.170) 94.373 ms 6 * * * 7 * * * 8 L100.DLLSTX-VFTTP-64.verizon-gni.net (71.252.251.1) 135.242 ms 86.272 ms 86.231 ms
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

By total chance wearing my SomaFM T-shirt today... If it's of any help, here are some mtrs coming from two different locations/environments. This is only half the picture though -- the return path to the below sources is unknown. I've provided source IPs in case you want to run tests back (please use ICMP protocol). src IP: 76.102.14.35 (Comcast; Mountain View, CA) dst IP: 71.252.251.1 (Verizon; Dallas, TX) Packets Pings Host Loss% Snt Rcv Last Avg Best Wrst 1. gw.home.lan (192.168.1.1) 0.0% 91 91 0.2 0.3 0.2 0.4 2. 76.102.12.1 0.0% 91 91 8.0 9.1 7.9 25.2 3. te-0-2-0-5-ur06.santaclara.ca.sfba.comcast.net (68.86.249.253) 0.0% 91 90 16.5 8.9 8.1 16.5 4. te-0-5-0-2-ur05.santaclara.ca.sfba.comcast.net (69.139.198.41) 0.0% 90 90 8.6 9.0 8.3 16.6 5. te-1-1-0-0-ar01.sfsutro.ca.sfba.comcast.net (69.139.198.82) 0.0% 90 90 14.7 12.2 9.9 17.1 te-1-1-0-3-ar01.sfsutro.ca.sfba.comcast.net (68.85.155.62) te-1-1-0-9-ar01.sfsutro.ca.sfba.comcast.net (69.139.198.178) te-1-1-0-10-ar01.sfsutro.ca.sfba.comcast.net (68.87.226.118) te-1-1-0-4-ar01.sfsutro.ca.sfba.comcast.net (68.85.155.66) te-1-1-0-8-ar01.sfsutro.ca.sfba.comcast.net (68.85.155.146) te-1-1-0-13-ar01.sfsutro.ca.sfba.comcast.net (69.139.199.106) te-1-1-0-1-ar01.sfsutro.ca.sfba.comcast.net (69.139.198.90) 6. he-1-7-0-0-cr01.sanjose.ca.ibone.comcast.net (68.86.90.153) 0.0% 90 90 14.1 13.4 10.8 21.8 7. be-10-pe02.11greatoaks.ca.ibone.comcast.net (68.86.82.82) 0.0% 90 90 14.7 15.2 14.0 51.0 8. 23.30.206.94 11.1% 90 80 73.8 72.3 65.9 79.6 9. B100.DLLSTX-LCR-21.verizon-gni.net (130.81.199.59) 10.0% 90 81 111.8 109.5 106.7 116.5 10. ??? 11. L100.DLLSTX-VFTTP-64.verizon-gni.net (71.252.251.1) 6.7% 90 84 113.4 110.4 107.1 131.9 src IP: 208.79.90.130 (ARP Networks; Sylmar, CA) dst IP: 71.252.251.1 (Verizon; Dallas, TX) Packets Pings Host Loss% Snt Rcv Last Avg Best Wrst 1. 208.79.90.129 0.0% 91 91 1.0 4.9 0.9 69.8 2. s7.lax.arpnetworks.com (208.79.88.135) 0.0% 91 91 0.5 12.1 0.4 188.5 3. ge-0-7-0-24.r04.lsanca03.us.bb.gin.ntt.net (129.250.198.185) 0.0% 91 91 0.8 0.9 0.7 1.4 4. ae-3.r05.lsanca03.us.bb.gin.ntt.net (129.250.2.221) 0.0% 90 90 1.2 1.2 0.9 4.3 5. ae-0.verizonbusiness.lsanca03.us.bb.gin.ntt.net (129.250.8.86) 7.8% 90 83 29.4 32.0 28.9 85.7 6. P0-8-0-0.DLLSTX-LCR-21.verizon-gni.net (130.81.199.35) 2.2% 90 88 69.3 69.5 67.0 76.3 7. ??? 8. L100.DLLSTX-VFTTP-64.verizon-gni.net (71.252.251.1) 3.3% 90 87 68.7 71.1 64.5 200.9 It's very hard to discern something useful from these single sets of mtrs because ICMP deprioritisation has become so commonplace**. For example (going off gut feeling), in the 2nd mtr above, I'd say that the NTT/Verio hop (#5) is probably ICMP deprio, while subsequent loss is probably real. Also, 23.30.206.94 lives within Comcast IP space, but it's impossible to know what company (Verizon or Comcast) is responsible for it. ** -- Folks often forget that even when using classic UDP traceroute, ICMP is still involved (specifically time exceeded). -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB | On Mon, Feb 10, 2014 at 05:02:40PM -0800, Rusty Hodge wrote:
We are seeing lots of problems with our audio streaming traffic dropping lots of packets where our transit provider Cogent hands off to ATT & Verizon.
Right now, I'm seeing 43% packet loss
Cogent is blaming ATT/Verizon and says there is nothing they can do about it. We're the collateral damage.
Who else is having the same issue?
I threaten to cancel my service, Cogent says, "sorry to lose you as a customer", and go on their merry way.
Are other transit providers having issues peering / getting to ATT/Verizon? I need to know who to avoid.
traceroute to 71.252.251.1 (71.252.251.1), 30 hops max, 60 byte packets 1 gi3-11.ccr01.sfo02.atlas.cogentco.com (38.104.130.89) 0.982 ms 0.985 ms 1.195 ms 2 te0-0-0-28.ccr22.sfo01.atlas.cogentco.com (154.54.3.129) 0.531 ms te0-0-0-28.ccr21.sfo01.atlas.cogentco.com (154.54.3.121) 0.604 ms 0.636 ms 3 be2165.ccr22.sjc01.atlas.cogentco.com (154.54.28.66) 2.277 ms be2164.ccr21.sjc01.atlas.cogentco.com (154.54.28.34) 1.806 ms 2.449 ms 4 be2000.ccr21.sjc03.atlas.cogentco.com (154.54.6.106) 3.824 ms 2.504 ms 2.596 ms 5 * * verizon.sjc03.atlas.cogentco.com (154.54.12.170) 94.373 ms 6 * * * 7 * * * 8 L100.DLLSTX-VFTTP-64.verizon-gni.net (71.252.251.1) 135.242 ms 86.272 ms 86.231 ms
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

Hi, On Mon, Feb 10, 2014 at 05:02:40PM -0800, Rusty Hodge wrote:
I threaten to cancel my service, Cogent says, "sorry to lose you as a customer", and go on their merry way.
I think this is the only language Cogent understands. gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany gert@greenie.muc.de fax: +49-89-35655025 gert@net.informatik.tu-muenchen.de
participants (8)
-
Anthony Kosednar
-
Ben Mason
-
Charles Sprickman
-
Gert Doering
-
Jeremy Chadwick
-
Lee
-
Rusty Hodge
-
Stephen Wilcox