We received the following notice regarding scheduled maintenance from XO. It is apparently due to some hardware upgrades. While this notice only refers to AZ, they may have scheduled similar outages in other areas today as well. Dear Customer, This email notice is to inform you of scheduled maintenance occurring in the Phoenix, AZ area affecting your XO Data/Access service. On 7/2/2008, between the hours of 11:00 PM MST to 6:00 AM MST, you will experience a 120 minute interruption of service. XO Communications Customer Care and Network Teams will be monitoring this activity to ensure necessary downtime is kept to a minimum while the maintenance is completed. At your convenience, feel free to reply to this message or call us at the toll free number listed on your XO bill, as our Customer Care teams will be available to handle any questions, comments, or concerns. Please reference Maintenance Request 53135 when contacting Customer Care. Thank You, XO Communications On 7/2/08 5:56 PM, "Matt Palmer" <mpalmer@hezmatt.org> wrote:
Greetings Outageers,
One of our customers is seeing some problems between themselves and our DC, and both-way traceroutes are showing problems just corewards of our upstream, Internap:
Customer to us:
traceroute to 74.201.255.113 (74.201.255.113), 64 hops max, 40 byte packets 1 192.168.1.1 (192.168.1.1) 3.188 ms 7.275 ms 5.663 ms 2 ATM1-0-60.win17.Melbourne.telstra.net (165.228.176.49) 12.096 ms 9.410 ms 13.803 ms 3 TenGigE0-8-0-5.win-core1.Melbourne.telstra.net (203.50.80.129) 8.397 ms 8.633 ms 11.843 ms 4 Bundle-Pos1.ken-core4.Sydney.telstra.net (203.50.6.21) 22.303 ms 26.382 ms 29.742 ms 5 Port-Channel1.pad-gw2.Sydney.telstra.net (203.50.6.29) 21.379 ms 26.954 ms 31.464 ms 6 10GigabitEthernet1-0.sydp-core02.Sydney.reach.com (203.50.13.46) 115.516 ms 24.861 ms 204.107 ms 7 i-0-0.wil-core02.net.reach.com (202.84.144.101) 167.032 ms 170.378 ms 175.605 ms 8 p9_0-ashc1.net.reach.com (202.84.140.90) 182.026 ms 184.921 ms 181.750 ms 9 g4_14-pax06.net.reach.com (202.84.251.106) 181.076 ms 183.144 ms 181.113 ms 10 206.111.12.173.ptr.us.xo.net (206.111.12.173) 192.088 ms 324.819 ms 194.243 ms 11 * * te-11-0-0.rar3.sanjose-ca.us.xo.net (207.88.12.69) 196.075 ms 12 207.88.14.133.ptr.us.xo.net (207.88.14.133) 261.502 ms 259.664 ms 266.501 ms 13 207.88.14.134.ptr.us.xo.net (207.88.14.134) 264.905 ms 257.116 ms 258.146 ms (oh so many stars)
Us back to the customer:
traceroute to 165.228.176.49 (165.228.176.49), 30 hops max, 40 byte packets 1 33.254.201.74.static.ey03.engineyard.com (74.201.254.33) 0.907 ms 0.908 ms 0.928 ms 2 76.254.201.74.static.ey03.engineyard.com (74.201.254.76) 0.210 ms 0.241 ms 0.273 ms 3 core1.te6-1-bbnet1.nym007.pnap.net (216.52.95.24) 1.559 ms core1.te6-2-bbnet2.nym007.pnap.net (216.52.95.88) 1.338 ms core1.te6-1-bbnet1.nym007.pnap.net (216.52.95.24) 1.369 ms 4 208.173.129.61 (208.173.129.61) 1.441 ms 1.525 ms 1.658 ms 5 * * * 6 cr2-loopback.sfo.savvis.net (206.24.210.71) 98.457 ms 94.617 ms 101.634 ms 7 pr1-so-0-0-0.PaloAltoPaix.savvis.net (204.70.200.193) 89.389 ms 89.397 ms 89.478 ms (constellations from here on)
I'm fine from my connection on the same side of the pond, since it looks like my ISP has a direct peering with Internap:
traceroute to 74.201.255.113 (74.201.255.113), 30 hops max, 40 byte packets 1 172.14.17.1 (172.14.17.1) 34.982 ms 35.987 ms 48.326 ms 2 lns10.syd6.internode.on.net (150.101.197.88) 50.219 ms 51.812 ms 53.293 ms 3 vl14.cor2.syd6.internode.on.net (150.101.197.83) 91.320 ms 92.807 ms 94.274 ms 4 po3.bdr1.syd6.internode.on.net (150.101.199.232) 226.619 ms 231.256 ms 234.897 ms 5 pos2-1.bdr1.lax1.internode.on.net (203.16.213.158) 228.912 ms 233.132 ms 236.411 ms 6 cr1-eqix-peer.lax009.internap.net (206.223.123.29) 245.254 ms 208.194 ms * 7 cr1.wdc005.inappnet-65.cr1.lax009.internap.net (66.79.146.141) 279.374 ms 283.716 ms 289.151 ms 8 cr1.nym009.inappnet-84.cr1.wdc005.internap.net (66.79.146.209) 309.854 ms 316.382 ms 318.303 ms 9 core3.nym.inappnet-82.cr1.nym007.internap.net (66.79.152.130) 298.775 ms 300.648 ms 307.357 ms 10 border2.pc2-bbnet2.nyj001.pnap.net (216.52.95.74) 323.780 ms 325.643 ms 331.222 ms 11 74.201.254.73 (74.201.254.73) 348.343 ms 350.727 ms * 12 113.255.201.74.static.ey03.engineyard.com (74.201.255.113) 305.041 ms 303.767 ms 305.289 ms
Wondering if anyone else is seeing similar problems, or if anyone knows of any widespread problems with Internap/XO/Savvis that I can pass on to my customer to let them know what's going on.
Thanks, - Matt _______________________________________________ outages mailing list outages@isotf.org https://puck.nether.net/mailman/listinfo/outages