I had been seeing nearly 50% packet loss to na4.salesforce.com starting at around 11:20 EST on 2010-03-08 for roughly 20-30 minutes. I only have a route to 14340 through 2914. Packets appeared to be dropped between 128.241.219.22 and 204.14.234.134. The path appears stable now though. Does anyone have any additional information on 14340's issue? It appears that this is happening more and more. Growing pains? Calls to SF Level 1 support are less than productive and requests to escalate to someone with clue are fruitless. I know this is a single site outage and not a provider outage, but I'm hoping somone clueful might have more information. Thanks in advance for any additional information. Mike
I experienced this and saw socket timeouts with AppExchange. Seems stable by the time I got to checking my network path. Not sure if we have anyone from Saleforce on Outages. Zaid On 3/8/10 9:51 AM, "Michael Moeller" <mwmoeller@gmail.com> wrote:
I had been seeing nearly 50% packet loss to na4.salesforce.com starting at around 11:20 EST on 2010-03-08 for roughly 20-30 minutes. I only have a route to 14340 through 2914. Packets appeared to be dropped between 128.241.219.22 and 204.14.234.134. The path appears stable now though.
Does anyone have any additional information on 14340's issue? It appears that this is happening more and more. Growing pains? Calls to SF Level 1 support are less than productive and requests to escalate to someone with clue are fruitless.
I know this is a single site outage and not a provider outage, but I'm hoping somone clueful might have more information.
Thanks in advance for any additional information.
Mike _______________________________________________ outages mailing list outages@outages.org https://puck.nether.net/mailman/listinfo/outages
I saw issues while generating a ticket with a vendor who uses salesforce, as well as customers complaining of salesforce "weirdness" this morning. Seems to have cleared up for me. -----Original Message----- From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Zaid Ali Sent: Monday, March 08, 2010 11:46 AM To: Michael Moeller; outages@outages.org Subject: Re: [outages] Salesforce Outage I experienced this and saw socket timeouts with AppExchange. Seems stable by the time I got to checking my network path. Not sure if we have anyone from Saleforce on Outages. Zaid On 3/8/10 9:51 AM, "Michael Moeller" <mwmoeller@gmail.com> wrote:
I had been seeing nearly 50% packet loss to na4.salesforce.com starting at around 11:20 EST on 2010-03-08 for roughly 20-30 minutes. I only have a route to 14340 through 2914. Packets appeared to be dropped between 128.241.219.22 and 204.14.234.134. The path appears stable now though.
Does anyone have any additional information on 14340's issue? It appears that this is happening more and more. Growing pains? Calls to SF Level 1 support are less than productive and requests to escalate to someone with clue are fruitless.
I know this is a single site outage and not a provider outage, but I'm hoping somone clueful might have more information.
Thanks in advance for any additional information.
Mike _______________________________________________ outages mailing list outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ outages mailing list outages@outages.org https://puck.nether.net/mailman/listinfo/outages
On Mar 8, 2010 3:25pm, Blake Pfankuch <bpfankuch@cpgreeley.com> wrote:
I saw issues while generating a ticket with a vendor who uses salesforce, as well as customers complaining of salesforce "weirdness" this morning. Seems to have cleared up for me.
It looks like it cleared up around 17:14 UTC, I haven't heard any more complaints from my users. I'm just curious as to what is going on. It appears this is happening much more often lately. Mike
On Mon, Mar 8, 2010 at 9:51 AM, Michael Moeller <mwmoeller@gmail.com> wrote:
I had been seeing nearly 50% packet loss to na4.salesforce.com starting at around 11:20 EST on 2010-03-08 for roughly 20-30 minutes. I only have a route to 14340 through 2914. Packets appeared to be dropped between 128.241.219.22 and 204.14.234.134. The path appears stable now though.
Does anyone have any additional information on 14340's issue? It appears that this is happening more and more. Growing pains? Calls to SF Level 1 support are less than productive and requests to escalate to someone with clue are fruitless.
I know this is a single site outage and not a provider outage, but I'm hoping somone clueful might have more information.
Thanks in advance for any additional information.
Mike
http://trust.salesforce.com/trust/status/ At the top of the page they have some additional information. Dave
On Mar 8, 2010 3:53pm, Big Wave Dave <bigwavedave@gmail.com> wrote:
At the top of the page they have some additional information.
Dave
Dave, Thanks. I'd fogotten to mention in my first post that I checked Trust earlier - but was hoping that perhaps someone had more specifics. Thank you, though. Mike
----- "Big Wave Dave" <bigwavedave@gmail.com> wrote:
http://trust.salesforce.com/trust/status/
At the top of the page they have some additional information.
Added to the Outages.org Dashboard page of status page links; thanks. Oh, and I'm sure Ken Levine would be pleased to know he has a fan. :-) Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com '87 e24 St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274 Start a man a fire, and he'll be warm all night. Set a man on fire, and he'll be warm for the rest of his life.
On Thu, Mar 11, 2010 at 9:56 AM, Jay R. Ashworth <jra@baylink.com> wrote:
----- "Big Wave Dave" <bigwavedave@gmail.com> wrote:
http://trust.salesforce.com/trust/status/
At the top of the page they have some additional information.
Added to the Outages.org Dashboard page of status page links; thanks.
Oh, and I'm sure Ken Levine would be pleased to know he has a fan. :-)
Cheers, -- jra
Jay, Thanks for adding that to the dashboard page. Unfortunately, today trust.salesforce.com was down - as were the rest of their services. Did anyone else using Salesforce just experience a 35 minute outage? Support is stating that the issue is now resolved. However, it seems like their infrastructure is suffering some major brain damage. Best, Mike
I am experiencing problems, Our sync queue is backed up ! Zaid On 3/11/10 8:01 AM, "Michael Moeller" <mwmoeller@gmail.com> wrote:
On Thu, Mar 11, 2010 at 9:56 AM, Jay R. Ashworth <jra@baylink.com> wrote:
----- "Big Wave Dave" <bigwavedave@gmail.com> wrote:
http://trust.salesforce.com/trust/status/
At the top of the page they have some additional information.
Added to the Outages.org Dashboard page of status page links; thanks.
Oh, and I'm sure Ken Levine would be pleased to know he has a fan. :-)
Cheers, -- jra
Jay, Thanks for adding that to the dashboard page. Unfortunately, today trust.salesforce.com was down - as were the rest of their services.
Did anyone else using Salesforce just experience a 35 minute outage? Support is stating that the issue is now resolved. However, it seems like their infrastructure is suffering some major brain damage.
Best, Mike
_______________________________________________ outages mailing list outages@outages.org https://puck.nether.net/mailman/listinfo/outages
We experienced timeouts, network path seemed fine to me so wondering if its something else inside the network. I see things picking up now and events being uploaded. Zaid On 3/11/10 8:08 AM, "Zaid Ali" <zaid@genius.com> wrote:
I am experiencing problems, Our sync queue is backed up !
Zaid
On 3/11/10 8:01 AM, "Michael Moeller" <mwmoeller@gmail.com> wrote:
On Thu, Mar 11, 2010 at 9:56 AM, Jay R. Ashworth <jra@baylink.com> wrote:
----- "Big Wave Dave" <bigwavedave@gmail.com> wrote:
http://trust.salesforce.com/trust/status/
At the top of the page they have some additional information.
Added to the Outages.org Dashboard page of status page links; thanks.
Oh, and I'm sure Ken Levine would be pleased to know he has a fan. :-)
Cheers, -- jra
Jay, Thanks for adding that to the dashboard page. Unfortunately, today trust.salesforce.com was down - as were the rest of their services.
Did anyone else using Salesforce just experience a 35 minute outage? Support is stating that the issue is now resolved. However, it seems like their infrastructure is suffering some major brain damage.
Best, Mike
_______________________________________________ outages mailing list outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ outages mailing list outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Did anyone else using Salesforce just experience a 35 minute outage?
Aye. Packet loss around 90%, latency on surviving packets in the 1.8 second range. Traceroute seemed to indicate fine (during normal operation it doesn't go past ae-44-99.car4.sanjose1.level3.net for me). Cleared up now. We're pre-production over here, just integration people working with SFDC. I have the feeling the crazed "WHATS WRONG WITH THE INTERNET!" I got was a taste of the future. I'm imagining every phone in engineering/helpdesk starts ringing simultaneously... What are you folks doing to gather stats/inform helpdesk when the cloud fails? Any IPSLA monitors help? Thanks, -porkchop
On 3/11/10 8:56 AM, "Kaegler, Mike" <KaeglerM@tessco.com> wrote:
Did anyone else using Salesforce just experience a 35 minute outage?
Aye. Packet loss around 90%, latency on surviving packets in the 1.8 second range. Traceroute seemed to indicate fine (during normal operation it doesn't go past ae-44-99.car4.sanjose1.level3.net for me).
Cleared up now. We're pre-production over here, just integration people working with SFDC. I have the feeling the crazed "WHATS WRONG WITH THE INTERNET!" I got was a taste of the future. I'm imagining every phone in engineering/helpdesk starts ringing simultaneously...
What are you folks doing to gather stats/inform helpdesk when the cloud fails? Any IPSLA monitors help?
Looks like they have their hands full with some upper level management explaining to do. http://www.networkworld.com/news/2010/031110-salesforce-outage.html Zaid
participants (7)
-
Big Wave Dave -
Blake Pfankuch -
Jay R. Ashworth -
Kaegler, Mike -
Michael Moeller -
mwmoeller＠gmail.com
-
Zaid Ali