January 2010 - Outages - Outages Lists

Bank of America
by Robert D. Scott 29 Jan '10

29 Jan '10

Any word on who doinked the Bank of America web page/ Robert D. Scott Robert(a)ufl.edu Senior Network Engineer 352-273-0113 Phone CNS - Network Services 352-392-2061 CNS Phone Tree University of Florida 352-392-9440 FAX Florida Lambda Rail 352-294-3571 FLR NOC Gainesville, FL 32611 321-663-0421 Cell

3 2

Fiber cut in NJ?
by David Israel 14 Jan '10

14 Jan '10

I am hearing rumors of a fiber cut in New Jersey affecting multiple providers; anybody have any details? -- David Israel Senior IT Network Engineer University System of Maryland Email: davei(a)usmd.edu Phone: 301.445.2768

4 3

Re: [outages] Fiber Cut in NJ (Cogent)
by Aaron Baker 14 Jan '10

14 Jan '10

http://status.cogentco.com/ indicates the following: Cogent Network Status Report Last Updated Thu Jan 14 12:48:38 2010 *Cogent Network Status/DNS Server Status Description: *Welcome to Cogent Communications’ Network Status Message. Customers with traffic going through or on the east coast may be experiencing packet loss and higher than normal latency due to a fiber cut in New Jersey. Splice crews are at the site. There is no ETR at this time. The Cogent ticket to reference for this issue is HD2082680. A separate issue has been identified for customers in Toronto and Chicago. The routing issue has been resolved. The Cogent ticket to reference for this issue is HD2082851

1 0

Fiber cut 2010-01-14
by Kevin Loch 14 Jan '10

14 Jan '10

Does anyone have details? I saw large drops in traffic to several peers around 9:47 EST. - Kevin

2 1

FW: Cogent Outage?
by Eric Tykwinski 14 Jan '10

14 Jan '10

We currently aren't seeing many issues, but it's being reported on NANOG. Sincerely, Eric Tykwinski TrueNet, Inc. P: 610-429-8300 F: 610-429-3222 -----Original Message----- From: Raphael Maunier [mailto:rmaunier@neotelecoms.com] Sent: Thursday, January 14, 2010 12:40 PM To: Robert Glover Cc: nanog(a)nanog.org Subject: Re: Cogent Outage? There is a fiber cut and it's located near Brunswick, NJ -- Raphaël Maunier NEO TELECOMS On Jan 14, 2010, at 6:36 PM, Robert Glover wrote: > Hello, > > Our 100Mb Metro-E link is up and running just fine... We are fed out of their San Francisco, CA facility. > > However, this is posted on the Cogent Status page: > > > **************************************************************************** ** > ** Cogent Network Status Report Last Updated Thu Jan 14 10:38:30 2010 ** > **************************************************************************** ** > **************************************************************************** ** > *Network Status:* Warning > *DNS Servers Status:* Normal > *Dialup/IPASS Status: *Normal > *Mail Servers Status:* Normal > *Webservers Status: *Normal > > > *Cogent Network Status/DNS Server Status Description: * Welcome to Cogent Communications' Network Status Message. Customers with traffic going through or on the east coast may be experiencing packet loss and higher than normal latency due to a fiber cut in New Jersey. There is no ETR at this time. The Cogent ticket to reference for this issue is HD2082680. > > > Sincerely, > Bobby Glover > Director of Information Services > South Valley Internet > > On 1/14/2010 9:31 AM, Joe Johnson wrote: >> We just lost Cogent across the country, along with several sister companies. Can't get through to a support person. Any idea what's going on? >> >> Joe Johnson >> Chief Information Officer >> Riverside Consulting Group, Ltd. >> Phone: 708.442.6033 x3456 >> Fax: 708.442.9722 >> joe(a)riversidecg.com >> www.riversidecg.com >> >> >> >> >> >

1 0

Outage to www.google.com through Cogent in Denver
by Chris Stone 14 Jan '10

14 Jan '10

Anyone else seeing this? [root@watcher root]# mtr --report --report-cycles=25 74.125.95.105 HOST: watcher.axint.net Loss% Snt Last Avg Best Wrst StDev 1. ai-core.axint.net 0.0% 25 0.6 1.0 0.4 9.5 1.8 2. fa0-2.na01.b009854-0.den01.a 0.0% 25 0.9 1.0 0.7 1.8 0.2 3. gi1-1.3509.mpd01.den01.atlas 0.0% 25 1.1 1.0 0.7 1.6 0.2 4. te0-4-0-2.mpd21.mci01.atlas. 0.0% 25 12.5 12.9 12.5 14.0 0.4 5. ??? 100.0 25 0.0 0.0 0.0 0.0 0.0 [root@csmdv ~]# mtr --report --report-cycles=25 74.125.95.105 HOST: csmdv.highmtnsoftware.com Loss% Snt Last Avg Best Wrst StDev 1. 10.10.100.254 0.0% 25 1.0 0.9 0.9 1.4 0.1 2. 192.168.0.1 4.0% 25 1.5 1.7 1.5 4.3 0.6 3. dsl-denver-gateway-01.axint. 4.0% 25 54.4 56.8 54.0 85.1 7.2 4. fa0-2.na01.b009854-0.den01.a 4.0% 25 54.6 56.4 54.2 78.4 5.4 5. gi1-1.3509.mpd01.den01.atlas 4.0% 25 55.0 57.7 54.1 111.7 11.7 6. te0-4-0-2.mpd21.mci01.atlas. 4.0% 25 66.6 68.1 66.3 81.3 3.8 7. ??? 100.0 25 0.0 0.0 0.0 0.0 0.0 8. te3-2.mpd01.yyz02.atlas.coge 92.0% 25 95.5 94.6 93.7 95.5 1.3 9. ??? 100.0 25 0.0 0.0 0.0 0.0 0.0 Chris

2 2

cable provider problems yesterday around 1pm EST?
by Rich Casto 14 Jan '10

14 Jan '10

Yesterday (Wed, Jan 13) we experienced connectivity loss from both our Level 3 and AT&T connections to our telecommuter population who primarily use the following cable providers: Time-Warner (RoadRunner), Cox, and Comcast. Our AT&T circuits go into NYC and our Level 3 goes into Newark, NJ. Did anyone else notice any similar issues? Thanks! -- Rich

2 1

Major fiber cut - Cox Communication in Fairfax VA
by Russell Goings 13 Jan '10

13 Jan '10

Cox Communications has a major multiple strand fiber cut in Fairfax County, Virginia. Outage began at 6:02 PM and is on-going. It is affecting data and voice. Call volume is sending calls to their business support number to an overflow call center They are working to repair but as of yet have no ETR. They have promised another update by 9:30 PM. Just another day in paradise. Russell G. Sysadmin Feld Entertainment, Inc.

2 1

[Fwd: AT&T Router down in Newark, NJ]
by virendra rode 08 Jan '10

08 Jan '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Not sure how wide of an impact this outage has. When responding please do not cross-post. regards, /virendra - -------- Original Message -------- Subject: AT&T Router down in Newark, NJ Date: Fri, 8 Jan 2010 17:36:09 -0500 From: Matt Simmons <standalone.sysadmin(a)gmail.com> To: nanog(a)nanog.org In case this affects any of you: Dear AT&T IP Services Customer: This e-mail is to notify you we are currently experiencing an impairment with Newark Gigabit Access Router 1. We expect to have additional information as soon as possible, and we deeply apologize for any inconvenience this impairment may cause you. Our trouble ticket number is 119512734. Below are the IP addresses of your affected circuit(s) for which we have you listed as a contact. If you feel that this list is in error please reply with your change requests. x.x.x.x For more information as it becomes available, please visit our website: https://mis-att.bus.att.com<http://redir.aspx?C=a34e3c8e8d1949b5a5227a8e887664c2&URL=https%3a%2f%2fmis-…> If you require further information please feel free to email AT&T at: rm-awmis(a)ems.att.com Thank you for using AT&T. Sincerely The AT&T Customer Care Team - -- LITTLE GIRL: But which cookie will you eat FIRST? COOKIE MONSTER: Me think you have misconception of cookie-eating process. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFLR8YZpbZvCIJx1bcRAjb4AJsEbrFOv0+YU7QV2hQWWnbh0c/ShgCbBNr7 96AfNMNG7MlO8l1VPvGZwrQ= =85GA -----END PGP SIGNATURE-----

1 0

AT&T T1 Service issue in San Jose / Saratoga CA?
by Andrew Davis 04 Jan '10

04 Jan '10

Is anyone else experiencing an issue with AT&T T1 service in San Jose / Saratoga CA? Got into work this morning to find our T1 line completely dead. Our vendor reports it as an underlying AT&T issue. I just wanted to see if we're the only ones out or if there's others with T1 issues today. Thanks! Andrew Davis WestGate Church Information Technology & Support (w): 408.252.3700 x218 When Life Gives You Questions, Google has Answers!

1 0