It clearly *is* from 2010: https://engineering.fb.com/2010/09/23/uncategorized/more-details-on-today-s-... Damian On Tue, Mar 5, 2024 at 11:13 AM Jay R. Ashworth via Outages < outages@outages.org> wrote:

Well, it seems equivocal whether that's about *this* outage rather than an earlier one -- though it's clearly not from 2010.

More to follow.

Cheers, -- jra

----- Original Message -----

...

From: "Jay R. Ashworth via Outages" <outages@outages.org> To: outages@outages.org Sent: Tuesday, March 5, 2024 1:57:24 PM Subject: [outages] After-Action Report: Facebook/Meta outage

...

[ I can't set the header from Zimbra, but please, Reply-To: -discuss ]

Here's a link to FB's AAR about this morning's 70-minuteish outage.

It's been pointed out to me that this affected other services as well, if they are an OAuth client, and you chose to use FB's OAuth service to create your account there.

https://m.facebook.com/nt/screen/?params=%7B%22note_id%22%3A1015879143614220...

...

[ Hat-tip: Jorge Amodio ]

Cheers, -- jra

-- Jay R. Ashworth Baylink

jra@baylink.com

...

Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 <(727)%20647-1274> _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

-- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 <(727)%20647-1274> _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

Indeed - this happens every time facebook has a major outage, because they stopped putting out posts explaining what happened and because people tend not to look too closely before spreading links that look plausible. On Tue, Mar 5, 2024 at 2:56 PM Jay R. Ashworth via Outages < outages@outages.org> wrote:

Well, when I pulled up the link from Ross's mobile version, it had comments under it from 10/22; if those people didn't know where to go either... :-}

----- Original Message -----

...

From: "Damian Menscher" <damian@google.com> To: "jra" <jra@baylink.com> Cc: outages@outages.org Sent: Tuesday, March 5, 2024 2:16:33 PM Subject: Re: [outages] After-Action Report: Facebook/Meta outage

...

It clearly *is* from 2010:

https://engineering.fb.com/2010/09/23/uncategorized/more-details-on-today-s-...

...

Damian

On Tue, Mar 5, 2024 at 11:13 AM Jay R. Ashworth via Outages < outages@outages.org> wrote:

...

Well, it seems equivocal whether that's about *this* outage rather than an earlier one -- though it's clearly not from 2010.

More to follow.

Cheers, -- jra

----- Original Message -----

...

From: "Jay R. Ashworth via Outages" <outages@outages.org> To: outages@outages.org Sent: Tuesday, March 5, 2024 1:57:24 PM Subject: [outages] After-Action Report: Facebook/Meta outage

...

[ I can't set the header from Zimbra, but please, Reply-To: -discuss ]

Here's a link to FB's AAR about this morning's 70-minuteish outage.

It's been pointed out to me that this affected other services as

well, if

...

...

...

they are an OAuth client, and you chose to use FB's OAuth service to create your account there.

https://m.facebook.com/nt/screen/?params=%7B%22note_id%22%3A1015879143614220...

...

...

[ Hat-tip: Jorge Amodio ]

Cheers, -- jra

-- Jay R. Ashworth Baylink

jra@baylink.com

...

Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 <(727)%20647-1274> _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

-- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 <(727)%20647-1274> _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

-- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

We are not renewing Splunk. We have migrated to Azure Sentinel for logging and monitoring. Aaron ________________________________ From: Outages <outages-bounces@outages.org> on behalf of Jay Ashworth via Outages <outages@outages.org> Sent: Tuesday, March 5, 2024 2:59 PM To: Ross Tajvar <ross@tajvar.io> Cc: outages@outages.org <outages@outages.org> Subject: Re: [outages] After-Action Report: Facebook/Meta outage Yeah, but I shouldn't have gotten bitten. I'm supposed to be holding myself to a higher standard, as a moderator of this particular group. On March 5, 2024 2:58:00 PM EST, Ross Tajvar <ross@tajvar.io> wrote: Indeed - this happens every time facebook has a major outage, because they stopped putting out posts explaining what happened and because people tend not to look too closely before spreading links that look plausible. On Tue, Mar 5, 2024 at 2:56 PM Jay R. Ashworth via Outages <outages@outages.org<mailto:outages@outages.org>> wrote: Well, when I pulled up the link from Ross's mobile version, it had comments under it from 10/22; if those people didn't know where to go either... :-} ----- Original Message -----

From: "Damian Menscher" <damian@google.com<mailto:damian@google.com>> To: "jra" <jra@baylink.com<mailto:jra@baylink.com>> Cc: outages@outages.org<mailto:outages@outages.org> Sent: Tuesday, March 5, 2024 2:16:33 PM Subject: Re: [outages] After-Action Report: Facebook/Meta outage

It clearly *is* from 2010: https://engineering.fb.com/2010/09/23/uncategorized/more-details-on-today-s-...

Damian

On Tue, Mar 5, 2024 at 11:13 AM Jay R. Ashworth via Outages < outages@outages.org<mailto:outages@outages.org>> wrote:

...

Well, it seems equivocal whether that's about *this* outage rather than an earlier one -- though it's clearly not from 2010.

More to follow.

Cheers, -- jra

----- Original Message -----

...

From: "Jay R. Ashworth via Outages" <outages@outages.org<mailto:outages@outages.org>> To: outages@outages.org<mailto:outages@outages.org> Sent: Tuesday, March 5, 2024 1:57:24 PM Subject: [outages] After-Action Report: Facebook/Meta outage

...

[ I can't set the header from Zimbra, but please, Reply-To: -discuss ]

Here's a link to FB's AAR about this morning's 70-minuteish outage.

It's been pointed out to me that this affected other services as well, if they are an OAuth client, and you chose to use FB's OAuth service to create your account there.

https://m.facebook.com/nt/screen/?params=%7B%22note_id%22%3A1015879143614220...

...

[ Hat-tip: Jorge Amodio ]

Cheers, -- jra

-- Jay R. Ashworth Baylink

jra@baylink.com<mailto:jra@baylink.com>

...

Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 <(727)%20647-1274> _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages

-- Jay R. Ashworth Baylink jra@baylink.com<mailto:jra@baylink.com> Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 <(727)%20647-1274> _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages

-- Jay R. Ashworth Baylink jra@baylink.com<mailto:jra@baylink.com> Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages -- Sent from my Android device with K-9 Mail. Please excuse my brevity. WARNING CONCERNING CONFIDENTIALITY This email message, including any attachments, is for the sole use by its intended recipient or recipients and may contain confidential and privileged information. Any unauthorized review, use, disclosure, or distribution is prohibited. If you are not the intended recipient of this message, please immediately notify the sender and delete the original message without making a copy. _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

There was definitely a reset of Auth. Seen here in the middle east. Whether that was intentional eg security patch similar to Google I'm not sure but it recovered after a few minutes and looks okay since On Wed, 6 Mar 2024, 00:17 Aaron Hope via Outages, <outages@outages.org> wrote:

GAH! Too many emails about Facebook today! Now I got bit!

Sorry all :)

Aaron ------------------------------ *From:* Outages <outages-bounces@outages.org> on behalf of Aaron Hope via Outages <outages@outages.org> *Sent:* Tuesday, March 5, 2024 3:14 PM *To:* Ross Tajvar <ross@tajvar.io>; Jay Ashworth <jra@baylink.com> *Cc:* outages@outages.org <outages@outages.org> *Subject:* Re: [outages] After-Action Report: Facebook/Meta outage

We are not renewing Splunk. We have migrated to Azure Sentinel for logging and monitoring.

Aaron ------------------------------ *From:* Outages <outages-bounces@outages.org> on behalf of Jay Ashworth via Outages <outages@outages.org> *Sent:* Tuesday, March 5, 2024 2:59 PM *To:* Ross Tajvar <ross@tajvar.io> *Cc:* outages@outages.org <outages@outages.org> *Subject:* Re: [outages] After-Action Report: Facebook/Meta outage

Yeah, but I shouldn't have gotten bitten. I'm supposed to be holding myself to a higher standard, as a moderator of this particular group.

On March 5, 2024 2:58:00 PM EST, Ross Tajvar <ross@tajvar.io> wrote:

Indeed - this happens every time facebook has a major outage, because they stopped putting out posts explaining what happened and because people tend not to look too closely before spreading links that look plausible.

On Tue, Mar 5, 2024 at 2:56 PM Jay R. Ashworth via Outages < outages@outages.org> wrote:

Well, when I pulled up the link from Ross's mobile version, it had comments under it from 10/22; if those people didn't know where to go either... :-}

----- Original Message -----

...

From: "Damian Menscher" <damian@google.com> To: "jra" <jra@baylink.com> Cc: outages@outages.org Sent: Tuesday, March 5, 2024 2:16:33 PM Subject: Re: [outages] After-Action Report: Facebook/Meta outage

...

It clearly *is* from 2010:

https://engineering.fb.com/2010/09/23/uncategorized/more-details-on-today-s-...

...

Damian

On Tue, Mar 5, 2024 at 11:13 AM Jay R. Ashworth via Outages < outages@outages.org> wrote:

...

Well, it seems equivocal whether that's about *this* outage rather than an earlier one -- though it's clearly not from 2010.

More to follow.

Cheers, -- jra

----- Original Message -----

...

From: "Jay R. Ashworth via Outages" <outages@outages.org> To: outages@outages.org Sent: Tuesday, March 5, 2024 1:57:24 PM Subject: [outages] After-Action Report: Facebook/Meta outage

...

[ I can't set the header from Zimbra, but please, Reply-To: -discuss ]

Here's a link to FB's AAR about this morning's 70-minuteish outage.

It's been pointed out to me that this affected other services as

well, if

...

...

...

they are an OAuth client, and you chose to use FB's OAuth service to create your account there.

https://m.facebook.com/nt/screen/?params=%7B%22note_id%22%3A1015879143614220...

...

...

[ Hat-tip: Jorge Amodio ]

Cheers, -- jra

-- Jay R. Ashworth Baylink

jra@baylink.com

...

Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 <(727)%20647-1274> _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

-- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 <(727)%20647-1274> _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

-- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

-- Sent from my Android device with K-9 Mail. Please excuse my brevity. WARNING CONCERNING CONFIDENTIALITY This email message, including any attachments, is for the sole use by its intended recipient or recipients and may contain confidential and privileged information. Any unauthorized review, use, disclosure, or distribution is prohibited. If you are not the intended recipient of this message, please immediately notify the sender and delete the original message without making a copy. WARNING CONCERNING CONFIDENTIALITY This email message, including any attachments, is for the sole use by its intended recipient or recipients and may contain confidential and privileged information. Any unauthorized review, use, disclosure, or distribution is prohibited. If you are not the intended recipient of this message, please immediately notify the sender and delete the original message without making a copy. _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

-- *CONFIDENTIALITY AND DISCLAIMER NOTICE: * This email is intended only for the person to whom it is addressed and/or otherwise authorized personnel. The information contained herein and attached is confidential. If you are not the intended recipient, please be advised that viewing this message and any attachments, as well as copying, forwarding, printing, and disseminating any information related to this email is prohibited, and that you should not take any action based on the content of this email and/or its attachments. If you received this message in error, please contact the sender and destroy all copies of this email and any attachment. Please note that the views and opinions expressed herein are solely those of the author and do not necessarily reflect those of the company. While antivirus protection tools have been employed, you should check this email and attachments for the presence of viruses. No warranties or assurances are made in relation to the safety and content of this email and attachments. The Company accepts no liability for any damage caused by any virus transmitted by or contained in this email and attachments. No liability is accepted for any consequences arising from this email. *AVIS DE CONFIDENTIALITÉ ET DE NON RESPONSABILITE* :  Ce courriel, ainsi que toute pièce jointe, est confidentiel et peut être protégé par le secret professionnel. Si vous n’en êtes pas le destinataire visé, veuillez en aviser l’expéditeur immédiatement et le supprimer. Vous ne devez pas le copier, ni l’utiliser à quelque fin que ce soit, ni divulguer son contenu à qui que ce soit. BSO se réserve le droit de contrôler toute transmission qui passe par son réseau. Veuillez noter que les opinions exprimées dans cet e-mail sont uniquement celles de l'auteur et ne reflètent pas nécessairement celles de la société. Bien que des outils de protection antivirus aient été utilisés, vous devez vérifier cet e-mail et les pièces jointes pour toute présence de virus. Aucune garantie ou assurance n'est donnée concernant la sécurité et le contenu de cet e-mail et de ses pièces jointes. La Société décline toute responsabilité pour tout dommage causé par tout virus transmis par ou contenu dans cet e-mail et ses pièces jointes. Aucune responsabilité n'est acceptée pour les conséquences découlant de cet e-mail. _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages