AWS recovering a bit after losing AZ in me-central-1
https://health.aws.amazon.com/health/status Service Multiple services Severity Degraded RSS Increased Error Rates Mar 01 7:09 AM PST We wanted to provide some additional information on the isolated power issue. At this time, most AWS Services have weighted away from the affected Availability Zone (mec1-az2) and are seeing recovery for their affected operations and workflows. For EC2 Instances, EBS Volumes, and other resources that are impacted in the affected Zone, we will have a longer tail of recovery. At this time, power has not yet been restored to the affected AZ. For now, we recommend continuing to retry any failed API requests. If immediate recovery is required, we recommend customers restore from EBS Snapshots and/or replace affected resources by launching replacement resources in one of the unaffected zones, or an alternate region. As of this time, recovery is still several hours away. We will provide an update by 8:30 AM PST, or sooner if we have additional information to share. Mar 01 6:09 AM PST We can confirm that a localized power issue has affected a single Availability Zone in the ME-CENTRAL-1 Region (mec1-az2). EC2 Instances, DB Instances, EBS Volumes, and others resources are currently unavailable and will experience connectivity issues at this time. Other AWS Services are also experiencing error rates and latencies for some workflows. We have weighed away traffic for most services at this time. We recommend customers utilize one of the other Availability Zones in the ME-CENTRAL-1 Region at this time, as existing instances in other AZ's remain unaffected by this issue. We are actively working to restore power and connectivity, at which time we will begin to work to recover affected resources. As of this time, we expect recovery is multiple hours away. We will provide an update by 7:15 AM PST, or sooner if we have additional information to share. Mar 01 5:19 AM PST We are investigating connectivity and power issues affecting APIs and instances in a single Availability Zone (mec1-az2) in the ME-CENTRAL-1 Region due to a localized power issue. Existing instances in this zone will also be affected. Other AWS Services may also be experiencing increased errors and latencies for their workflows, and we are working to route requests away from this affected Availability Zone. We recommend customers make use of other Availability Zones at this time. Targeting new launches using RunInstances in the remaining AZs should succeed. Existing instances in the other AZs are not affected. Mar 01 4:51 AM PST We are investigating issues with AWS services in the ME-CENTRAL-1 Region.
this is affecting managed services like OpenSearch, as well as basic services like EC2. we are finding that AWS CLI will tell us a resource exists, such as EC2, but we cannot access the resources, e.g. ssh to the instance.
I can’t even sign into my AWS Console. It says “an AWS account with that sign in information does not exist” when I’ve only put in the email address. Not only is this a fundamental security violation, because you should never disclose that an account doesn’t exist when only the account ID is entered, but it’s plain scary that I can’t get to any of my AWS infrastructure to see what’s going on. -mel via cell
On Mar 1, 2026, at 9:21 AM, Paul M - outages via Outages <outages@outages.org> wrote:
this is affecting managed services like OpenSearch, as well as basic services like EC2.
we are finding that AWS CLI will tell us a resource exists, such as EC2, but we cannot access the resources, e.g. ssh to the instance. ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
And I also note that my instances are offline. A customer had called their ISP thinking it was their problem, and their ISP told him that since 4:30 AM PST AWS had a major outage due to an object that struck a data center creating sparks and fire. As a result, the power was shut off to the facility, including backup power. They had no ETA on the outage. I’m in the AZ2 zone. -mel via cell
On Mar 1, 2026, at 9:52 AM, Mel Beckman <mel@beckman.org> wrote:
I can’t even sign into my AWS Console. It says “an AWS account with that sign in information does not exist” when I’ve only put in the email address. Not only is this a fundamental security violation, because you should never disclose that an account doesn’t exist when only the account ID is entered, but it’s plain scary that I can’t get to any of my AWS infrastructure to see what’s going on.
-mel via cell
On Mar 1, 2026, at 9:21 AM, Paul M - outages via Outages <outages@outages.org> wrote:
this is affecting managed services like OpenSearch, as well as basic services like EC2.
we are finding that AWS CLI will tell us a resource exists, such as EC2, but we cannot access the resources, e.g. ssh to the instance. ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
The other ISP was kind enough to forward to me the AWS event history. I don’t know if this is published publicly anywhere online, but I haven’t been able to locate it. I would like to know if anyone else finds it so I can follow for updates. Affected AWS services The following AWS services have been affected by this issue. Disrupted (1 service) Amazon Elastic Compute Cloud History [09:41 AM PST] We want to provide some additional information on the power issue in a single Availability Zone in the ME-CENTRAL-1 Region. At around 4:30 AM PST, one of our Availability Zones (mec1-az2) was impacted by objects that struck the data center, creating sparks and fire. The fire department shut off power to the facility and generators as they worked to put out the fire. We are still awaiting permission to turn the power back on, and once we have, we will ensure we restore power and connectivity safely. It will take several hours to restore connectivity to the impacted AZ. The other AZs in the region are functioning normally. Customers who were running their applications redundantly across the AZs are not impacted by this event. EC2 Instance launches will continue to be impaired in the impacted AZ. We recommend that customers continue to retry any failed API requests. If immediate recovery of an affected resource (EC2 Instance, EBS Volume, RDS DB Instance, etc.) is required, we recommend restoring from your most recent backup, by launching replacement resources in one of the unaffected zones, or an alternate AWS Region. We will provide an update by 12:30 PM PST, or sooner if we have additional information to share. [08:59 AM PST] We continue to work toward restoring power in the affected Availability Zone in the ME-CENTRAL-1 Region (mec1-az2). In parallel, we are actively working on improving error rates and latencies that some customers are observing for EC2 Networking and EC2 Describe APIs. Due to increased demand in the unaffected Availability Zones, customers may experience longer than usual provisioning times or may need to retry requests for certain instance types, or pick an alternative instance type. We will provide an update by 10:30 AM PST, or sooner if we have additional information to share. [07:09 AM PST] We wanted to provide some additional information on the isolated power issue. At this time, most AWS Services have weighted away from the affected Availability Zone (mec1-az2) and are seeing recovery for their affected operations and workflows. For EC2 Instances, EBS Volumes, and other resources that are impacted in the affected Zone, we will have a longer tail of recovery. At this time, power has not yet been restored to the affected AZ. For now, we recommend continuing to retry any failed API requests. If immediate recovery is required, we recommend customers restore from EBS Snapshots and/or replace affected resources by launching replacement resources in one of the unaffected zones, or an alternate region. As of this time, recovery is still several hours away. We will provide an update by 8:30 AM PST, or sooner if we have additional information to share. [06:09 AM PST] We can confirm that a localized power issue has affected a single Availability Zone in the ME-CENTRAL-1 Region (mec1-az2). EC2 Instances, DB Instances, EBS Volumes, and others resources are currently unavailable and will experience connectivity issues at this time. Other AWS Services are also experiencing error rates and latencies for some workflows. We have weighed away traffic for most services at this time. We recommend customers utilize one of the other Availability Zones in the ME-CENTRAL-1 Region at this time, as existing instances in other AZ's remain unaffected by this issue. We are actively working to restore power and connectivity, at which time we will begin to work to recover affected resources. As of this time, we expect recovery is multiple hours away. We will provide an update by 7:15 AM PST, or sooner if we have additional information to share. [05:19 AM PST] We are investigating connectivity and power issues affecting APIs and instances in a single Availability Zone (mec1-az2) in the ME-CENTRAL-1 Region due to a localized power issue. Existing instances in this zone will also be affected. Other AWS Services may also be experiencing increased errors and latencies for their workflows, and we are working to route requests away from this affected Availability Zone. We recommend customers make use of other Availability Zones at this time. Targeting new launches using RunInstances in the remaining AZs should succeed. Existing instances in the other AZs are not affected. [04:51 AM PST] We are investigating issues with AWS services in the ME-CENTRAL-1 Region. -mel beckman On Mar 1, 2026, at 9:59 AM, Mel Beckman <mel@beckman.org> wrote: And I also note that my instances are offline. A customer had called their ISP thinking it was their problem, and their ISP told him that since 4:30 AM PST AWS had a major outage due to an object that struck a data center creating sparks and fire. As a result, the power was shut off to the facility, including backup power. They had no ETA on the outage. I’m in the AZ2 zone. -mel via cell On Mar 1, 2026, at 9:52 AM, Mel Beckman <mel@beckman.org> wrote: I can’t even sign into my AWS Console. It says “an AWS account with that sign in information does not exist” when I’ve only put in the email address. Not only is this a fundamental security violation, because you should never disclose that an account doesn’t exist when only the account ID is entered, but it’s plain scary that I can’t get to any of my AWS infrastructure to see what’s going on. -mel via cell On Mar 1, 2026, at 9:21 AM, Paul M - outages via Outages <outages@outages.org> wrote: this is affecting managed services like OpenSearch, as well as basic services like EC2. we are finding that AWS CLI will tell us a resource exists, such as EC2, but we cannot access the resources, e.g. ssh to the instance. ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/ Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/ Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
As always, AWS status is available here: https://health.aws.amazon.com/health/status Jason On Sun, Mar 1, 2026 at 14:34 Mel Beckman via Outages <outages@outages.org> wrote:
The other ISP was kind enough to forward to me the AWS event history. I don’t know if this is published publicly anywhere online, but I haven’t been able to locate it. I would like to know if anyone else finds it so I can follow for updates.
Affected AWS services
The following AWS services have been affected by this issue.
Disrupted (1 service)
Amazon Elastic Compute Cloud
History
[09:41 AM PST] We want to provide some additional information on the power issue in a single Availability Zone in the ME-CENTRAL-1 Region. At around 4:30 AM PST, one of our Availability Zones (mec1-az2) was impacted by objects that struck the data center, creating sparks and fire. The fire department shut off power to the facility and generators as they worked to put out the fire. We are still awaiting permission to turn the power back on, and once we have, we will ensure we restore power and connectivity safely. It will take several hours to restore connectivity to the impacted AZ. The other AZs in the region are functioning normally. Customers who were running their applications redundantly across the AZs are not impacted by this event. EC2 Instance launches will continue to be impaired in the impacted AZ. We recommend that customers continue to retry any failed API requests. If immediate recovery of an affected resource (EC2 Instance, EBS Volume, RDS DB Instance, etc.) is required, we recommend restoring from your most recent backup, by launching replacement resources in one of the unaffected zones, or an alternate AWS Region. We will provide an update by 12:30 PM PST, or sooner if we have additional information to share.
[08:59 AM PST] We continue to work toward restoring power in the affected Availability Zone in the ME-CENTRAL-1 Region (mec1-az2). In parallel, we are actively working on improving error rates and latencies that some customers are observing for EC2 Networking and EC2 Describe APIs. Due to increased demand in the unaffected Availability Zones, customers may experience longer than usual provisioning times or may need to retry requests for certain instance types, or pick an alternative instance type. We will provide an update by 10:30 AM PST, or sooner if we have additional information to share.
[07:09 AM PST] We wanted to provide some additional information on the isolated power issue. At this time, most AWS Services have weighted away from the affected Availability Zone (mec1-az2) and are seeing recovery for their affected operations and workflows. For EC2 Instances, EBS Volumes, and other resources that are impacted in the affected Zone, we will have a longer tail of recovery. At this time, power has not yet been restored to the affected AZ. For now, we recommend continuing to retry any failed API requests. If immediate recovery is required, we recommend customers restore from EBS Snapshots and/or replace affected resources by launching replacement resources in one of the unaffected zones, or an alternate region. As of this time, recovery is still several hours away. We will provide an update by 8:30 AM PST, or sooner if we have additional information to share.
[06:09 AM PST] We can confirm that a localized power issue has affected a single Availability Zone in the ME-CENTRAL-1 Region (mec1-az2). EC2 Instances, DB Instances, EBS Volumes, and others resources are currently unavailable and will experience connectivity issues at this time. Other AWS Services are also experiencing error rates and latencies for some workflows. We have weighed away traffic for most services at this time. We recommend customers utilize one of the other Availability Zones in the ME-CENTRAL-1 Region at this time, as existing instances in other AZ's remain unaffected by this issue. We are actively working to restore power and connectivity, at which time we will begin to work to recover affected resources. As of this time, we expect recovery is multiple hours away. We will provide an update by 7:15 AM PST, or sooner if we have additional information to share.
[05:19 AM PST] We are investigating connectivity and power issues affecting APIs and instances in a single Availability Zone (mec1-az2) in the ME-CENTRAL-1 Region due to a localized power issue. Existing instances in this zone will also be affected. Other AWS Services may also be experiencing increased errors and latencies for their workflows, and we are working to route requests away from this affected Availability Zone. We recommend customers make use of other Availability Zones at this time. Targeting new launches using RunInstances in the remaining AZs should succeed. Existing instances in the other AZs are not affected. [04:51 AM PST] We are investigating issues with AWS services in the ME-CENTRAL-1 Region.
-mel beckman
On Mar 1, 2026, at 9:59 AM, Mel Beckman <mel@beckman.org> wrote:
And I also note that my instances are offline. A customer had called their ISP thinking it was their problem, and their ISP told him that since 4:30 AM PST AWS had a major outage due to an object that struck a data center creating sparks and fire. As a result, the power was shut off to the facility, including backup power. They had no ETA on the outage. I’m in the AZ2 zone.
-mel via cell
On Mar 1, 2026, at 9:52 AM, Mel Beckman <mel@beckman.org> wrote:
I can’t even sign into my AWS Console. It says “an AWS account with that sign in information does not exist” when I’ve only put in the email address. Not only is this a fundamental security violation, because you should never disclose that an account doesn’t exist when only the account ID is entered, but it’s plain scary that I can’t get to any of my AWS infrastructure to see what’s going on.
-mel via cell
On Mar 1, 2026, at 9:21 AM, Paul M - outages via Outages < outages@outages.org> wrote:
this is affecting managed services like OpenSearch, as well as basic services like EC2.
we are finding that AWS CLI will tell us a resource exists, such as EC2, but we cannot access the resources, e.g. ssh to the instance. ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
It turns out the incident status is public, at: <https://health.aws.amazon.com/health/status> aws.amazon.com<https://health.aws.amazon.com/health/status> [favicon.ico]<https://health.aws.amazon.com/health/status> -mel beckman On Mar 1, 2026, at 9:59 AM, Mel Beckman <mel@beckman.org> wrote: And I also note that my instances are offline. A customer had called their ISP thinking it was their problem, and their ISP told him that since 4:30 AM PST AWS had a major outage due to an object that struck a data center creating sparks and fire. As a result, the power was shut off to the facility, including backup power. They had no ETA on the outage. I’m in the AZ2 zone. -mel via cell On Mar 1, 2026, at 9:52 AM, Mel Beckman <mel@beckman.org> wrote: I can’t even sign into my AWS Console. It says “an AWS account with that sign in information does not exist” when I’ve only put in the email address. Not only is this a fundamental security violation, because you should never disclose that an account doesn’t exist when only the account ID is entered, but it’s plain scary that I can’t get to any of my AWS infrastructure to see what’s going on. -mel via cell On Mar 1, 2026, at 9:21 AM, Paul M - outages via Outages <outages@outages.org> wrote: this is affecting managed services like OpenSearch, as well as basic services like EC2. we are finding that AWS CLI will tell us a resource exists, such as EC2, but we cannot access the resources, e.g. ssh to the instance. ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/ Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/ Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
other AZs seem to be running ok, we typically have services running in all three, and we've been able to relocate resources. but, it would be unwise to assume there's no possibility of another AZ going down, or even the entire region going offline if there's more disruption to the power grid as diesel generators will only run for so long.
Oh, and of course, I wish everybody working at AWS there the very best as they look to contain this crisis.
On Sun, 1 Mar 2026 at 17:48, Mel Beckman <mel@beckman.org> wrote:
I can’t even sign into my AWS Console. It says “an AWS account with that sign in information does not exist” when I’ve only put in the email address. Not only is this a fundamental security violation, because you should never disclose that an account doesn’t exist when only the account ID is entered, but it’s plain scary that I can’t get to any of my AWS infrastructure to see what’s going on.
we use Okta to sign in, it was working fine just now (0945 UTC, 2026-03-02)
On Mon, 2 Mar 2026 at 09:54, Paul M - outages <paul+outages@mansfield.co.uk> wrote:
we use Okta to sign in, it was working fine just now (0945 UTC, 2026-03-02)
however, the API service is not working $ aws ec2 describe-instances An error occurred (Unavailable) when calling the DescribeInstances operation (reached max retries: 2): The service is unavailable. Please try again shortly. their status page was just updated.
AWS stopped broadcasting on their status page, so this is copied from a direct message from them MEC1 Update from AWS TAM The availability of services in has improved up to a point where we believe you will now be able to get your data out - but the situation is volatile and this may be temporary. It’s important to act now to copy your data to alternate regions. If you run regulated workloads, you should contact regulators to discuss options immediately. We urge you to copy your data and retry operations that were failing previously to other regions right now For your regulated data, if you haven't engaged the relevant regulators to discuss options, please do so immediately We have seen continued availability improvements on s3, efs, ebs and rds and believe you will be able to access most of their data now Conditions remain volatile and there is a risk availability will temporarily or permanently degrade The copy of EBS and RDS snapshots will now succeed if they are healthy In case of failure, we recommend to export the partial EBS snapshot and run data recovery on it We will publish a guide for the recovery of incomplete EBS snapshots in the coming hours
at least one of the Dubai airports was attached; the Dubai World Central Al Maktoum International airport is not far from the AWS datacentre, so power lines and other infrastructure could be collateral damage
participants (3)
-
Jason L. Sparks -
Mel Beckman -
Paul M - outages