Issue with AT&T customers reaching at least one of our IPs
Hello List, I need help getting in touch with someone helpful at AT&T (NetEng and/or trust & safety/abuse/compliance/security or preferably all the above). Starting on November 6th, customers behind AT&T have been unable to reach our IP address 149.137.136.16 which is the API VIP we use to serve downloads from Backblaze B2 buckets. If you go one up (.17) or one down (.15) no issues. I've tried calling the NOC contact per PeeringDB and they were ultimately unhelpful. Because we do not peer directly with AT&T they will not talk to us. I can confirm the reachability issue via AT&T's own public route server (route-server.ip.att.net) where I can ping 149.137.136.15 and .17 successfully, but cannot ping 149.137.136.16. MTRs from customers seem to show the path dies well before us. Here are a couple of examples that show 149.137.136.15 good and 149.137.136.16 bad: This one is from Michigan: $ mtr -w 149.137.136.15 Password: Start: 2025-11-12T13:39:10-0500 HOST: ip-192-168-0-231.ec2.internal Loss% Snt Last Avg Best Wrst StDev 1.|-- ip-192-168-0-1.ec2.internal 0.0% 10 4.0 2.9 2.5 4.0 0.4 2.|-- ip-192-168-1-254.ec2.internal 0.0% 10 3.3 3.3 2.6 3.5 0.2 3.|-- 108-202-236-1.lightspeed.livnmi.sbcglobal.net 0.0% 10 5.4 5.0 4.2 6.5 0.8 4.|-- 71.151.199.86 0.0% 10 6.5 5.8 4.8 6.9 0.6 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 7.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 9.|-- ae3.cr12-chi1.ip4.gtt.net 0.0% 10 14.7 14.8 13.7 15.9 0.6 10.|-- ae4.cr3-was1.ip4.gtt.net 0.0% 10 34.7 34.9 34.0 35.4 0.4 11.|-- ip4.gtt.net 0.0% 10 34.7 34.6 33.7 35.7 0.5 12.|-- api005.backblazeb2.com 0.0% 10 34.8 35.1 34.1 37.5 0.9 $ mtr -w 149.137.136.16 Start: 2025-11-12T13:39:54-0500 HOST: ip-192-168-0-231.ec2.internal Loss% Snt Last Avg Best Wrst StDev 1.|-- ip-192-168-0-1.ec2.internal 0.0% 10 2.5 2.8 2.5 3.0 0.1 2.|-- ip-192-168-1-254.ec2.internal 0.0% 10 3.4 3.5 3.0 3.8 0.2 3.|-- 108-202-236-1.lightspeed.livnmi.sbcglobal.net 0.0% 10 4.0 5.3 3.9 8.9 1.6 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 This one is from East Bay, California: $ mtr -w 149.137.136.15 Start: 2025-11-11T14:54:50-0800 HOST: maureen Loss% Snt Last Avg Best Wrst StDev 1.|-- router.lan 0.0% 10 0.6 0.6 0.4 0.8 0.1 2.|-- 192.168.1.254 0.0% 10 1.6 1.5 1.4 1.7 0.1 3.|-- 99-189-116-1.lightspeed.snmtca.sbcglobal.net 0.0% 10 19.7 19.9 19.6 20.6 0.3 4.|-- 71.148.180.125 0.0% 10 20.7 20.2 20.0 20.7 0.3 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6.|-- 32.130.91.83 40.0% 10 23.1 34.2 22.3 91.9 28.3 7.|-- 192.205.32.98 0.0% 10 29.4 24.9 23.1 30.2 2.6 8.|-- ae4.cr3-was1.ip4.gtt.net 0.0% 10 81.9 81.9 81.6 82.5 0.3 9.|-- ip4.gtt.net 0.0% 10 82.0 82.0 81.7 82.5 0.2 10.|-- api005.backblazeb2.com 0.0% 10 82.4 82.1 81.3 82.9 0.4 $ mtr -w 149.137.136.16 Start: 2025-11-11T13:49:14-0800 HOST: maureen Loss% Snt Last Avg Best Wrst StDev 1.|-- router.lan 0.0% 10 0.7 0.7 0.5 0.7 0.1 2.|-- 192.168.1.254 0.0% 10 1.5 1.5 1.4 1.5 0.1 3.|-- 99-189-116-1.lightspeed.snmtca.sbcglobal.net 0.0% 10 20.3 20.1 19.6 20.8 0.3 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 If there is anybody on this list that can throw me a bone, it would be very much appreciated. -- Anthony Hoppe Sr. Network Engineer ahoppe@backblaze.com Backblaze.com -- This email, including its contents and any attachment(s), may contain confidential and/or proprietary information and is solely for the review and use of the intended recipient(s). If you have received this email in error, please notify the sender and permanently delete this email, its content, and any attachment(s). Any disclosure, copying, or taking of any action in reliance on an email received in error is strictly prohibited.
Anthony, I see that your IP address is on this blacklist: https://matrix.spfbl.net/ Below is the message reason given: A domain is considered non-compliant when the WHOIS search result for that domain does not contain the email address of the domain owner. Update the registration data and remove privacy protection for this domain in WHOIS and wait one hour for the cached result of this WHOIS query to expire. This IP was flagged due to misconfiguration of the e-mail service or the suspicion that there is no MTA at it. The link below should take you to the message and the instructions to have the blacklist removed. I hope this helps! https://matrix.spfbl.net/en/149.137.136.16 Allan Paschall From: Anthony Hoppe via Outages <outages@outages.org> Date: Wednesday, November 12, 2025 at 12:21 PM To: outages@outages.org <outages@outages.org> Cc: Anthony Hoppe <ahoppe@backblaze.com> Subject: [Outages] Issue with AT&T customers reaching at least one of our IPs [This message originated from outside Bank of the Southwest] Hello List, I need help getting in touch with someone helpful at AT&T (NetEng and/or trust & safety/abuse/compliance/security or preferably all the above). Starting on November 6th, customers behind AT&T have been unable to reach our IP address 149.137.136.16 which is the API VIP we use to serve downloads from Backblaze B2 buckets. If you go one up (.17) or one down (.15) no issues. I've tried calling the NOC contact per PeeringDB and they were ultimately unhelpful. Because we do not peer directly with AT&T they will not talk to us. I can confirm the reachability issue via AT&T's own public route server (https://urldefense.proofpoint.com/v2/url?u=http-3A__route-2Dserver.ip.att.ne...) where I can ping 149.137.136.15 and .17 successfully, but cannot ping 149.137.136.16. MTRs from customers seem to show the path dies well before us. Here are a couple of examples that show 149.137.136.15 good and 149.137.136.16 bad: This one is from Michigan: $ mtr -w 149.137.136.15 Password: Start: 2025-11-12T13:39:10-0500 HOST: ip-192-168-0-231.ec2.internal Loss% Snt Last Avg Best Wrst StDev 1.|-- ip-192-168-0-1.ec2.internal 0.0% 10 4.0 2.9 2.5 4.0 0.4 2.|-- ip-192-168-1-254.ec2.internal 0.0% 10 3.3 3.3 2.6 3.5 0.2 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__108-2D202-2D236-2D1.ligh... 0.0% 10 5.4 5.0 4.2 6.5 0.8 4.|-- 71.151.199.86 0.0% 10 6.5 5.8 4.8 6.9 0.6 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 7.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 9.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae3.cr12-2Dchi1.ip4.gtt.... 0.0% 10 14.7 14.8 13.7 15.9 0.6 10.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae4.cr3-2Dwas1.ip4.gtt.n... 0.0% 10 34.7 34.9 34.0 35.4 0.4 11.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ip4.gtt.net&d=DwICAg&c=e... 0.0% 10 34.7 34.6 33.7 35.7 0.5 12.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__api005.backblazeb2.com&d... 0.0% 10 34.8 35.1 34.1 37.5 0.9 $ mtr -w 149.137.136.16 Start: 2025-11-12T13:39:54-0500 HOST: ip-192-168-0-231.ec2.internal Loss% Snt Last Avg Best Wrst StDev 1.|-- ip-192-168-0-1.ec2.internal 0.0% 10 2.5 2.8 2.5 3.0 0.1 2.|-- ip-192-168-1-254.ec2.internal 0.0% 10 3.4 3.5 3.0 3.8 0.2 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__108-2D202-2D236-2D1.ligh... 0.0% 10 4.0 5.3 3.9 8.9 1.6 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 This one is from East Bay, California: $ mtr -w 149.137.136.15 Start: 2025-11-11T14:54:50-0800 HOST: maureen Loss% Snt Last Avg Best Wrst StDev 1.|-- router.lan 0.0% 10 0.6 0.6 0.4 0.8 0.1 2.|-- 192.168.1.254 0.0% 10 1.6 1.5 1.4 1.7 0.1 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__99-2D189-2D116-2D1.light... 0.0% 10 19.7 19.9 19.6 20.6 0.3 4.|-- 71.148.180.125 0.0% 10 20.7 20.2 20.0 20.7 0.3 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6.|-- 32.130.91.83 40.0% 10 23.1 34.2 22.3 91.9 28.3 7.|-- 192.205.32.98 0.0% 10 29.4 24.9 23.1 30.2 2.6 8.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae4.cr3-2Dwas1.ip4.gtt.n... 0.0% 10 81.9 81.9 81.6 82.5 0.3 9.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ip4.gtt.net&d=DwICAg&c=e... 0.0% 10 82.0 82.0 81.7 82.5 0.2 10.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__api005.backblazeb2.com&d... 0.0% 10 82.4 82.1 81.3 82.9 0.4 $ mtr -w 149.137.136.16 Start: 2025-11-11T13:49:14-0800 HOST: maureen Loss% Snt Last Avg Best Wrst StDev 1.|-- router.lan 0.0% 10 0.7 0.7 0.5 0.7 0.1 2.|-- 192.168.1.254 0.0% 10 1.5 1.5 1.4 1.5 0.1 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__99-2D189-2D116-2D1.light... 0.0% 10 20.3 20.1 19.6 20.8 0.3 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 If there is anybody on this list that can throw me a bone, it would be very much appreciated. -- Anthony Hoppe Sr. Network Engineer ahoppe@backblaze.com https://urldefense.proofpoint.com/v2/url?u=http-3A__Backblaze.com&d=DwICAg&c... -- This email, including its contents and any attachment(s), may contain confidential and/or proprietary information and is solely for the review and use of the intended recipient(s). If you have received this email in error, please notify the sender and permanently delete this email, its content, and any attachment(s). Any disclosure, copying, or taking of any action in reliance on an email received in error is strictly prohibited. ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.outages.org_accou... To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.outages.org_archi... Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org NOTICE: The information contained in this message is confidential and for the sole use of the intended recipient/s. If you are not the intended recipient, please (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
Anthony has mentioned that .15 doesn’t have issues, which returns the same blacklist info. https://matrix.spfbl.net/en/149.137.136.15 -Chris 
On Nov 12, 2025, at 11:36, Allan Paschall via Outages <outages@outages.org> wrote:
Anthony,
I see that your IP address is on this blacklist: https://matrix.spfbl.net/
Below is the message reason given:
A domain is considered non-compliant when the WHOIS search result for that domain does not contain the email address of the domain owner. Update the registration data and remove privacy protection for this domain in WHOIS and wait one hour for the cached result of this WHOIS query to expire.
This IP was flagged due to misconfiguration of the e-mail service or the suspicion that there is no MTA at it.
The link below should take you to the message and the instructions to have the blacklist removed. I hope this helps!
https://matrix.spfbl.net/en/149.137.136.16
Allan Paschall
From: Anthony Hoppe via Outages <outages@outages.org <mailto:outages@outages.org>> Date: Wednesday, November 12, 2025 at 12:21 PM To: outages@outages.org <mailto:outages@outages.org> <outages@outages.org <mailto:outages@outages.org>> Cc: Anthony Hoppe <ahoppe@backblaze.com <mailto:ahoppe@backblaze.com>> Subject: [Outages] Issue with AT&T customers reaching at least one of our IPs
[This message originated from outside Bank of the Southwest]
Hello List,
I need help getting in touch with someone helpful at AT&T (NetEng and/or trust & safety/abuse/compliance/security or preferably all the above). Starting on November 6th, customers behind AT&T have been unable to reach our IP address 149.137.136.16 which is the API VIP we use to serve downloads from Backblaze B2 buckets.
If you go one up (.17) or one down (.15) no issues.
I've tried calling the NOC contact per PeeringDB and they were ultimately unhelpful. Because we do not peer directly with AT&T they will not talk to us.
I can confirm the reachability issue via AT&T's own public route server (https://urldefense.proofpoint.com/v2/url?u=http-3A__route-2Dserver.ip.att.ne...) where I can ping 149.137.136.15 and .17 successfully, but cannot ping 149.137.136.16. MTRs from customers seem to show the path dies well before us. Here are a couple of examples that show 149.137.136.15 good and 149.137.136.16 bad:
This one is from Michigan:
$ mtr -w 149.137.136.15 Password: Start: 2025-11-12T13:39:10-0500 HOST: ip-192-168-0-231.ec2.internal Loss% Snt Last Avg Best Wrst StDev 1.|-- ip-192-168-0-1.ec2.internal 0.0% 10 4.0 2.9 2.5 4.0 0.4 2.|-- ip-192-168-1-254.ec2.internal 0.0% 10 3.3 3.3 2.6 3.5 0.2 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__108-2D202-2D236-2D1.ligh... 0.0% 10 5.4 5.0 4.2 6.5 0.8 4.|-- 71.151.199.86 0.0% 10 6.5 5.8 4.8 6.9 0.6 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 7.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 9.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae3.cr12-2Dchi1.ip4.gtt.... 0.0% 10 14.7 14.8 13.7 15.9 0.6 10.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae4.cr3-2Dwas1.ip4.gtt.n... 0.0% 10 34.7 34.9 34.0 35.4 0.4 11.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ip4.gtt.net&d=DwICAg&c=e... 0.0% 10 34.7 34.6 33.7 35.7 0.5 12.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__api005.backblazeb2.com&d... 0.0% 10 34.8 35.1 34.1 37.5 0.9
$ mtr -w 149.137.136.16 Start: 2025-11-12T13:39:54-0500 HOST: ip-192-168-0-231.ec2.internal Loss% Snt Last Avg Best Wrst StDev 1.|-- ip-192-168-0-1.ec2.internal 0.0% 10 2.5 2.8 2.5 3.0 0.1 2.|-- ip-192-168-1-254.ec2.internal 0.0% 10 3.4 3.5 3.0 3.8 0.2 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__108-2D202-2D236-2D1.ligh... 0.0% 10 4.0 5.3 3.9 8.9 1.6 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
This one is from East Bay, California:
$ mtr -w 149.137.136.15 Start: 2025-11-11T14:54:50-0800 HOST: maureen Loss% Snt Last Avg Best Wrst StDev 1.|-- router.lan 0.0% 10 0.6 0.6 0.4 0.8 0.1 2.|-- 192.168.1.254 0.0% 10 1.6 1.5 1.4 1.7 0.1 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__99-2D189-2D116-2D1.light... 0.0% 10 19.7 19.9 19.6 20.6 0.3 4.|-- 71.148.180.125 0.0% 10 20.7 20.2 20.0 20.7 0.3 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6.|-- 32.130.91.83 40.0% 10 23.1 34.2 22.3 91.9 28.3 7.|-- 192.205.32.98 0.0% 10 29.4 24.9 23.1 30.2 2.6 8.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae4.cr3-2Dwas1.ip4.gtt.n... 0.0% 10 81.9 81.9 81.6 82.5 0.3 9.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ip4.gtt.net&d=DwICAg&c=e... 0.0% 10 82.0 82.0 81.7 82.5 0.2 10.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__api005.backblazeb2.com&d... 0.0% 10 82.4 82.1 81.3 82.9 0.4
$ mtr -w 149.137.136.16 Start: 2025-11-11T13:49:14-0800 HOST: maureen Loss% Snt Last Avg Best Wrst StDev 1.|-- router.lan 0.0% 10 0.7 0.7 0.5 0.7 0.1 2.|-- 192.168.1.254 0.0% 10 1.5 1.5 1.4 1.5 0.1 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__99-2D189-2D116-2D1.light... 0.0% 10 20.3 20.1 19.6 20.8 0.3 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
If there is anybody on this list that can throw me a bone, it would be very much appreciated.
--
Anthony Hoppe Sr. Network Engineer ahoppe@backblaze.com <mailto:ahoppe@backblaze.com> https://urldefense.proofpoint.com/v2/url?u=http-3A__Backblaze.com&d=DwICAg&c...
-- This email, including its contents and any attachment(s), may contain confidential and/or proprietary information and is solely for the review and use of the intended recipient(s). If you have received this email in error, please notify the sender and permanently delete this email, its content, and any attachment(s). Any disclosure, copying, or taking of any action in reliance on an email received in error is strictly prohibited. ______________________________________________ Outages mailing list outages@outages.org <mailto:outages@outages.org> Sign up for an account https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.outages.org_accou... To subscribe send an email to outages-join@outages.org <mailto:outages-join@outages.org> To unsubscribe send an email to outages-leave@outages.org <mailto:outages-leave@outages.org> To contact the list owners outages-owner@outages.org <mailto:outages-owner@outages.org> Archives https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.outages.org_archi...
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org <mailto:outages-discussion@outages.org> NOTICE: The information contained in this message is confidential and for the sole use of the intended recipient/s. If you are not the intended recipient, please (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
Information on this topic has been conveyed via a third party. -- Nimrod On Wed, Nov 12, 2025, 16:34 Chris Woodfield via Outages <outages@outages.org> wrote:
Anthony has mentioned that .15 doesn’t have issues, which returns the same blacklist info.
https://matrix.spfbl.net/en/149.137.136.15
-Chris 
On Nov 12, 2025, at 11:36, Allan Paschall via Outages < outages@outages.org> wrote:
Anthony,
I see that your IP address is on this blacklist: https://matrix.spfbl.net/
Below is the message reason given:
A domain is considered non-compliant when the WHOIS search result for that domain does not contain the email address of the domain owner. Update the registration data and remove privacy protection for this domain in WHOIS and wait one hour for the cached result of this WHOIS query to expire.
This IP was flagged due to misconfiguration of the e-mail service or the suspicion that there is no MTA at it.
The link below should take you to the message and the instructions to have the blacklist removed. I hope this helps!
https://matrix.spfbl.net/en/149.137.136.16
Allan Paschall
From: Anthony Hoppe via Outages <outages@outages.org <mailto: outages@outages.org>> Date: Wednesday, November 12, 2025 at 12:21 PM To: outages@outages.org <mailto:outages@outages.org> < outages@outages.org <mailto:outages@outages.org>> Cc: Anthony Hoppe <ahoppe@backblaze.com <mailto:ahoppe@backblaze.com>> Subject: [Outages] Issue with AT&T customers reaching at least one of our IPs
[This message originated from outside Bank of the Southwest]
Hello List,
I need help getting in touch with someone helpful at AT&T (NetEng and/or trust & safety/abuse/compliance/security or preferably all the above). Starting on November 6th, customers behind AT&T have been unable to reach our IP address 149.137.136.16 which is the API VIP we use to serve downloads from Backblaze B2 buckets.
If you go one up (.17) or one down (.15) no issues.
I've tried calling the NOC contact per PeeringDB and they were ultimately unhelpful. Because we do not peer directly with AT&T they will not talk to us.
I can confirm the reachability issue via AT&T's own public route server ( https://urldefense.proofpoint.com/v2/url?u=http-3A__route-2Dserver.ip.att.ne...) where I can ping 149.137.136.15 and .17 successfully, but cannot ping 149.137.136.16. MTRs from customers seem to show the path dies well before us. Here are a couple of examples that show 149.137.136.15 good and 149.137.136.16 bad:
This one is from Michigan:
$ mtr -w 149.137.136.15 Password: Start: 2025-11-12T13:39:10-0500 HOST: ip-192-168-0-231.ec2.internal Loss% Snt Last Avg Best Wrst StDev 1.|-- ip-192-168-0-1.ec2.internal 0.0% 10 4.0 2.9 2.5 4.0 0.4 2.|-- ip-192-168-1-254.ec2.internal 0.0% 10 3.3 3.3 2.6 3.5 0.2 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__108-2D202-2D236-2D1.ligh... 0.0% 10 5.4 5.0 4.2 6.5 0.8 4.|-- 71.151.199.86 0.0% 10 6.5 5.8 4.8 6.9 0.6 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 7.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 9.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae3.cr12-2Dchi1.ip4.gtt.... 0.0% 10 14.7 14.8 13.7 15.9 0.6 10.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae4.cr3-2Dwas1.ip4.gtt.n... 0.0% 10 34.7 34.9 34.0 35.4 0.4 11.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ip4.gtt.net&d=DwICAg&c=e... 0.0% 10 34.7 34.6 33.7 35.7 0.5 12.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__api005.backblazeb2.com&d... 0.0% 10 34.8 35.1 34.1 37.5 0.9
$ mtr -w 149.137.136.16 Start: 2025-11-12T13:39:54-0500 HOST: ip-192-168-0-231.ec2.internal Loss% Snt Last Avg Best Wrst StDev 1.|-- ip-192-168-0-1.ec2.internal 0.0% 10 2.5 2.8 2.5 3.0 0.1 2.|-- ip-192-168-1-254.ec2.internal 0.0% 10 3.4 3.5 3.0 3.8 0.2 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__108-2D202-2D236-2D1.ligh... 0.0% 10 4.0 5.3 3.9 8.9 1.6 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
This one is from East Bay, California:
$ mtr -w 149.137.136.15 Start: 2025-11-11T14:54:50-0800 HOST: maureen Loss% Snt Last Avg Best Wrst StDev 1.|-- router.lan 0.0% 10 0.6 0.6 0.4 0.8 0.1 2.|-- 192.168.1.254 0.0% 10 1.6 1.5 1.4 1.7 0.1 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__99-2D189-2D116-2D1.light... 0.0% 10 19.7 19.9 19.6 20.6 0.3 4.|-- 71.148.180.125 0.0% 10 20.7 20.2 20.0 20.7 0.3 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6.|-- 32.130.91.83 40.0% 10 23.1 34.2 22.3 91.9 28.3 7.|-- 192.205.32.98 0.0% 10 29.4 24.9 23.1 30.2 2.6 8.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae4.cr3-2Dwas1.ip4.gtt.n... 0.0% 10 81.9 81.9 81.6 82.5 0.3 9.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ip4.gtt.net&d=DwICAg&c=e... 0.0% 10 82.0 82.0 81.7 82.5 0.2 10.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__api005.backblazeb2.com&d... 0.0% 10 82.4 82.1 81.3 82.9 0.4
$ mtr -w 149.137.136.16 Start: 2025-11-11T13:49:14-0800 HOST: maureen Loss% Snt Last Avg Best Wrst StDev 1.|-- router.lan 0.0% 10 0.7 0.7 0.5 0.7 0.1 2.|-- 192.168.1.254 0.0% 10 1.5 1.5 1.4 1.5 0.1 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__99-2D189-2D116-2D1.light... 0.0% 10 20.3 20.1 19.6 20.8 0.3 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
If there is anybody on this list that can throw me a bone, it would be very much appreciated.
--
Anthony Hoppe Sr. Network Engineer ahoppe@backblaze.com <mailto:ahoppe@backblaze.com>
https://urldefense.proofpoint.com/v2/url?u=http-3A__Backblaze.com&d=DwICAg&c...
-- This email, including its contents and any attachment(s), may contain confidential and/or proprietary information and is solely for the review and use of the intended recipient(s). If you have received this email in error, please notify the sender and permanently delete this email, its content, and any attachment(s). Any disclosure, copying, or taking of
any
action in reliance on an email received in error is strictly prohibited. ______________________________________________ Outages mailing list outages@outages.org <mailto:outages@outages.org> Sign up for an account https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.outages.org_accou... To subscribe send an email to outages-join@outages.org <mailto: outages-join@outages.org> To unsubscribe send an email to outages-leave@outages.org <mailto: outages-leave@outages.org> To contact the list owners outages-owner@outages.org <mailto: outages-owner@outages.org> Archives https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.outages.org_archi...
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org <mailto: outages-discussion@outages.org> NOTICE: The information contained in this message is confidential and for the sole use of the intended recipient/s. If you are not the intended recipient, please (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
for what it's worth, I had this same type of issue where you could reach one up one down on a /24 if you were an ATT customer and it turned out to be an aggregate that I left on a router but didn't announce, cogent was seeing it on the wrong bgp session and it would never leave their router but most other ips worked just fine. On Wednesday, November 12th, 2025 at 2:52 PM, Nimrod Levy via Outages <outages@outages.org> wrote:
Information on this topic has been conveyed via a third party.
-- Nimrod
On Wed, Nov 12, 2025, 16:34 Chris Woodfield via Outages outages@outages.org
wrote:
Anthony has mentioned that .15 doesn’t have issues, which returns the same blacklist info.
https://matrix.spfbl.net/en/149.137.136.15
-Chris 
On Nov 12, 2025, at 11:36, Allan Paschall via Outages < outages@outages.org> wrote:
Anthony,
I see that your IP address is on this blacklist: https://matrix.spfbl.net/
Below is the message reason given:
A domain is considered non-compliant when the WHOIS search result for that domain does not contain the email address of the domain owner. Update the registration data and remove privacy protection for this domain in WHOIS and wait one hour for the cached result of this WHOIS query to expire.
This IP was flagged due to misconfiguration of the e-mail service or the suspicion that there is no MTA at it.
The link below should take you to the message and the instructions to have the blacklist removed. I hope this helps!
https://matrix.spfbl.net/en/149.137.136.16
Allan Paschall
From: Anthony Hoppe via Outages <outages@outages.org <mailto: outages@outages.org>> Date: Wednesday, November 12, 2025 at 12:21 PM To: outages@outages.org mailto:outages@outages.org < outages@outages.org mailto:outages@outages.org> Cc: Anthony Hoppe <ahoppe@backblaze.com mailto:ahoppe@backblaze.com> Subject: [Outages] Issue with AT&T customers reaching at least one of our IPs
[This message originated from outside Bank of the Southwest]
Hello List,
I need help getting in touch with someone helpful at AT&T (NetEng and/or trust & safety/abuse/compliance/security or preferably all the above). Starting on November 6th, customers behind AT&T have been unable to reach our IP address 149.137.136.16 which is the API VIP we use to serve downloads from Backblaze B2 buckets.
If you go one up (.17) or one down (.15) no issues.
I've tried calling the NOC contact per PeeringDB and they were ultimately unhelpful. Because we do not peer directly with AT&T they will not talk to us.
I can confirm the reachability issue via AT&T's own public route server ( https://urldefense.proofpoint.com/v2/url?u=http-3A__route-2Dserver.ip.att.ne...) where I can ping 149.137.136.15 and .17 successfully, but cannot ping 149.137.136.16. MTRs from customers seem to show the path dies well before us. Here are a couple of examples that show 149.137.136.15 good and 149.137.136.16 bad:
This one is from Michigan:
$ mtr -w 149.137.136.15 Password: Start: 2025-11-12T13:39:10-0500 HOST: ip-192-168-0-231.ec2.internal Loss% Snt Last Avg Best Wrst StDev 1.|-- ip-192-168-0-1.ec2.internal 0.0% 10 4.0 2.9 2.5 4.0 0.4 2.|-- ip-192-168-1-254.ec2.internal 0.0% 10 3.3 3.3 2.6 3.5 0.2 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__108-2D202-2D236-2D1.ligh... 0.0% 10 5.4 5.0 4.2 6.5 0.8 4.|-- 71.151.199.86 0.0% 10 6.5 5.8 4.8 6.9 0.6 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 7.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 9.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae3.cr12-2Dchi1.ip4.gtt.... 0.0% 10 14.7 14.8 13.7 15.9 0.6 10.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae4.cr3-2Dwas1.ip4.gtt.n... 0.0% 10 34.7 34.9 34.0 35.4 0.4 11.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ip4.gtt.net&d=DwICAg&c=e... 0.0% 10 34.7 34.6 33.7 35.7 0.5 12.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__api005.backblazeb2.com&d... 0.0% 10 34.8 35.1 34.1 37.5 0.9
$ mtr -w 149.137.136.16 Start: 2025-11-12T13:39:54-0500 HOST: ip-192-168-0-231.ec2.internal Loss% Snt Last Avg Best Wrst StDev 1.|-- ip-192-168-0-1.ec2.internal 0.0% 10 2.5 2.8 2.5 3.0 0.1 2.|-- ip-192-168-1-254.ec2.internal 0.0% 10 3.4 3.5 3.0 3.8 0.2 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__108-2D202-2D236-2D1.ligh... 0.0% 10 4.0 5.3 3.9 8.9 1.6 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
This one is from East Bay, California:
$ mtr -w 149.137.136.15 Start: 2025-11-11T14:54:50-0800 HOST: maureen Loss% Snt Last Avg Best Wrst StDev 1.|-- router.lan 0.0% 10 0.6 0.6 0.4 0.8 0.1 2.|-- 192.168.1.254 0.0% 10 1.6 1.5 1.4 1.7 0.1 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__99-2D189-2D116-2D1.light... 0.0% 10 19.7 19.9 19.6 20.6 0.3 4.|-- 71.148.180.125 0.0% 10 20.7 20.2 20.0 20.7 0.3 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6.|-- 32.130.91.83 40.0% 10 23.1 34.2 22.3 91.9 28.3 7.|-- 192.205.32.98 0.0% 10 29.4 24.9 23.1 30.2 2.6 8.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae4.cr3-2Dwas1.ip4.gtt.n... 0.0% 10 81.9 81.9 81.6 82.5 0.3 9.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ip4.gtt.net&d=DwICAg&c=e... 0.0% 10 82.0 82.0 81.7 82.5 0.2 10.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__api005.backblazeb2.com&d... 0.0% 10 82.4 82.1 81.3 82.9 0.4
$ mtr -w 149.137.136.16 Start: 2025-11-11T13:49:14-0800 HOST: maureen Loss% Snt Last Avg Best Wrst StDev 1.|-- router.lan 0.0% 10 0.7 0.7 0.5 0.7 0.1 2.|-- 192.168.1.254 0.0% 10 1.5 1.5 1.4 1.5 0.1 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__99-2D189-2D116-2D1.light... 0.0% 10 20.3 20.1 19.6 20.8 0.3 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
If there is anybody on this list that can throw me a bone, it would be very much appreciated.
--
Anthony Hoppe Sr. Network Engineer ahoppe@backblaze.com mailto:ahoppe@backblaze.com
https://urldefense.proofpoint.com/v2/url?u=http-3A__Backblaze.com&d=DwICAg&c...
-- This email, including its contents and any attachment(s), may contain confidential and/or proprietary information and is solely for the review and use of the intended recipient(s). If you have received this email in error, please notify the sender and permanently delete this email, its content, and any attachment(s). Any disclosure, copying, or taking of any action in reliance on an email received in error is strictly prohibited. ______________________________________________ Outages mailing list outages@outages.org mailto:outages@outages.org Sign up for an account https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.outages.org_accou... To subscribe send an email to outages-join@outages.org <mailto: outages-join@outages.org> To unsubscribe send an email to outages-leave@outages.org <mailto: outages-leave@outages.org> To contact the list owners outages-owner@outages.org <mailto: outages-owner@outages.org> Archives https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.outages.org_archi...
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org <mailto: outages-discussion@outages.org> NOTICE: The information contained in this message is confidential and for the sole use of the intended recipient/s. If you are not the intended recipient, please (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
Hello, Thank you for contacting the Nitel NOC. In order to properly open a case, we need to identify the affected service in our system. Please provide us with the following information to locate the service: * NIT/HC Circuit ID: * Location Address: Additionally, in order to start the troubleshooting process, please confirm the following information: * Local Contact Name/Phone Number: * Site Access Hours: Our team is committed to resolving your issue as efficiently as possible. If you require immediate assistance, don't hesitate to contact us directly at (866) 892-0915. Thank you, Jafeth Quesada NOC Analyst ll 866.892.0915 Comcast Business Philadelphia, PA [A blue text on a black background Description automatically generated] Nitel is now part of Comcast Business. ________________________________ From: Allan Paschall via Outages <outages@outages.org> Sent: Wednesday, November 12, 2025 1:36 PM To: Outages (Planned & Unplanned) Reporting <outages@outages.org> Cc: Anthony Hoppe <ahoppe@backblaze.com>; Allan Paschall <APaschall@bankofsw.com> Subject: [Outages] Re: Issue with AT&T customers reaching at least one of our IPs CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Anthony, I see that your IP address is on this blacklist: https://matrix.spfbl.net/ Below is the message reason given: A domain is considered non-compliant when the WHOIS search result for that domain does not contain the email address of the domain owner. Update the registration data and remove privacy protection for this domain in WHOIS and wait one hour for the cached result of this WHOIS query to expire. This IP was flagged due to misconfiguration of the e-mail service or the suspicion that there is no MTA at it. The link below should take you to the message and the instructions to have the blacklist removed. I hope this helps! https://matrix.spfbl.net/en/149.137.136.16 Allan Paschall From: Anthony Hoppe via Outages <outages@outages.org> Date: Wednesday, November 12, 2025 at 12:21 PM To: outages@outages.org <outages@outages.org> Cc: Anthony Hoppe <ahoppe@backblaze.com> Subject: [Outages] Issue with AT&T customers reaching at least one of our IPs [This message originated from outside Bank of the Southwest] Hello List, I need help getting in touch with someone helpful at AT&T (NetEng and/or trust & safety/abuse/compliance/security or preferably all the above). Starting on November 6th, customers behind AT&T have been unable to reach our IP address 149.137.136.16 which is the API VIP we use to serve downloads from Backblaze B2 buckets. If you go one up (.17) or one down (.15) no issues. I've tried calling the NOC contact per PeeringDB and they were ultimately unhelpful. Because we do not peer directly with AT&T they will not talk to us. I can confirm the reachability issue via AT&T's own public route server (https://urldefense.proofpoint.com/v2/url?u=http-3A__route-2Dserver.ip.att.ne...) where I can ping 149.137.136.15 and .17 successfully, but cannot ping 149.137.136.16. MTRs from customers seem to show the path dies well before us. Here are a couple of examples that show 149.137.136.15 good and 149.137.136.16 bad: This one is from Michigan: $ mtr -w 149.137.136.15 Password: Start: 2025-11-12T13:39:10-0500 HOST: ip-192-168-0-231.ec2.internal Loss% Snt Last Avg Best Wrst StDev 1.|-- ip-192-168-0-1.ec2.internal 0.0% 10 4.0 2.9 2.5 4.0 0.4 2.|-- ip-192-168-1-254.ec2.internal 0.0% 10 3.3 3.3 2.6 3.5 0.2 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__108-2D202-2D236-2D1.ligh... 0.0% 10 5.4 5.0 4.2 6.5 0.8 4.|-- 71.151.199.86 0.0% 10 6.5 5.8 4.8 6.9 0.6 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 7.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 9.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae3.cr12-2Dchi1.ip4.gtt.... 0.0% 10 14.7 14.8 13.7 15.9 0.6 10.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae4.cr3-2Dwas1.ip4.gtt.n... 0.0% 10 34.7 34.9 34.0 35.4 0.4 11.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ip4.gtt.net&d=DwICAg&c=e... 0.0% 10 34.7 34.6 33.7 35.7 0.5 12.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__api005.backblazeb2.com&d... 0.0% 10 34.8 35.1 34.1 37.5 0.9 $ mtr -w 149.137.136.16 Start: 2025-11-12T13:39:54-0500 HOST: ip-192-168-0-231.ec2.internal Loss% Snt Last Avg Best Wrst StDev 1.|-- ip-192-168-0-1.ec2.internal 0.0% 10 2.5 2.8 2.5 3.0 0.1 2.|-- ip-192-168-1-254.ec2.internal 0.0% 10 3.4 3.5 3.0 3.8 0.2 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__108-2D202-2D236-2D1.ligh... 0.0% 10 4.0 5.3 3.9 8.9 1.6 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 This one is from East Bay, California: $ mtr -w 149.137.136.15 Start: 2025-11-11T14:54:50-0800 HOST: maureen Loss% Snt Last Avg Best Wrst StDev 1.|-- router.lan 0.0% 10 0.6 0.6 0.4 0.8 0.1 2.|-- 192.168.1.254 0.0% 10 1.6 1.5 1.4 1.7 0.1 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__99-2D189-2D116-2D1.light... 0.0% 10 19.7 19.9 19.6 20.6 0.3 4.|-- 71.148.180.125 0.0% 10 20.7 20.2 20.0 20.7 0.3 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6.|-- 32.130.91.83 40.0% 10 23.1 34.2 22.3 91.9 28.3 7.|-- 192.205.32.98 0.0% 10 29.4 24.9 23.1 30.2 2.6 8.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae4.cr3-2Dwas1.ip4.gtt.n... 0.0% 10 81.9 81.9 81.6 82.5 0.3 9.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ip4.gtt.net&d=DwICAg&c=e... 0.0% 10 82.0 82.0 81.7 82.5 0.2 10.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__api005.backblazeb2.com&d... 0.0% 10 82.4 82.1 81.3 82.9 0.4 $ mtr -w 149.137.136.16 Start: 2025-11-11T13:49:14-0800 HOST: maureen Loss% Snt Last Avg Best Wrst StDev 1.|-- router.lan 0.0% 10 0.7 0.7 0.5 0.7 0.1 2.|-- 192.168.1.254 0.0% 10 1.5 1.5 1.4 1.5 0.1 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__99-2D189-2D116-2D1.light... 0.0% 10 20.3 20.1 19.6 20.8 0.3 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 If there is anybody on this list that can throw me a bone, it would be very much appreciated. -- Anthony Hoppe Sr. Network Engineer ahoppe@backblaze.com https://urldefense.proofpoint.com/v2/url?u=http-3A__Backblaze.com&d=DwICAg&c... -- This email, including its contents and any attachment(s), may contain confidential and/or proprietary information and is solely for the review and use of the intended recipient(s). If you have received this email in error, please notify the sender and permanently delete this email, its content, and any attachment(s). Any disclosure, copying, or taking of any action in reliance on an email received in error is strictly prohibited. ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.outages.org_accou... To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.outages.org_archi... Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org NOTICE: The information contained in this message is confidential and for the sole use of the intended recipient/s. If you are not the intended recipient, please (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/ Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org ________________________________ CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You must destroy the original transmission and its attachments without reading or saving in any manner. Thank you.
This isn't an outage! Take it to -discussion -- J. Hellenthal The fact that there's a highway to Hell but only a stairway to Heaven says a lot about anticipated traffic volume.
On Nov 12, 2025, at 17:52, noc via Outages <outages@outages.org> wrote:
Hello,
Thank you for contacting the Nitel NOC. In order to properly open a case, we need to identify the affected service in our system. Please provide us with the following information to locate the service:
* NIT/HC Circuit ID: * Location Address:
Additionally, in order to start the troubleshooting process, please confirm the following information:
* Local Contact Name/Phone Number: * Site Access Hours:
Our team is committed to resolving your issue as efficiently as possible.
If you require immediate assistance, don't hesitate to contact us directly at (866) 892-0915.
Thank you,
Jafeth Quesada
NOC Analyst ll
866.892.0915
Comcast Business
Philadelphia, PA
[A blue text on a black background Description automatically generated]
Nitel is now part of Comcast Business.
________________________________ From: Allan Paschall via Outages <outages@outages.org> Sent: Wednesday, November 12, 2025 1:36 PM To: Outages (Planned & Unplanned) Reporting <outages@outages.org> Cc: Anthony Hoppe <ahoppe@backblaze.com>; Allan Paschall <APaschall@bankofsw.com> Subject: [Outages] Re: Issue with AT&T customers reaching at least one of our IPs
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
Anthony,
I see that your IP address is on this blacklist: https://matrix.spfbl.net/
Below is the message reason given:
A domain is considered non-compliant when the WHOIS search result for that domain does not contain the email address of the domain owner. Update the registration data and remove privacy protection for this domain in WHOIS and wait one hour for the cached result of this WHOIS query to expire.
This IP was flagged due to misconfiguration of the e-mail service or the suspicion that there is no MTA at it.
The link below should take you to the message and the instructions to have the blacklist removed. I hope this helps!
https://matrix.spfbl.net/en/149.137.136.16
Allan Paschall
From: Anthony Hoppe via Outages <outages@outages.org> Date: Wednesday, November 12, 2025 at 12:21 PM To: outages@outages.org <outages@outages.org> Cc: Anthony Hoppe <ahoppe@backblaze.com> Subject: [Outages] Issue with AT&T customers reaching at least one of our IPs
[This message originated from outside Bank of the Southwest]
Hello List,
I need help getting in touch with someone helpful at AT&T (NetEng and/or trust & safety/abuse/compliance/security or preferably all the above). Starting on November 6th, customers behind AT&T have been unable to reach our IP address 149.137.136.16 which is the API VIP we use to serve downloads from Backblaze B2 buckets.
If you go one up (.17) or one down (.15) no issues.
I've tried calling the NOC contact per PeeringDB and they were ultimately unhelpful. Because we do not peer directly with AT&T they will not talk to us.
I can confirm the reachability issue via AT&T's own public route server (https://urldefense.proofpoint.com/v2/url?u=http-3A__route-2Dserver.ip.att.ne...) where I can ping 149.137.136.15 and .17 successfully, but cannot ping 149.137.136.16. MTRs from customers seem to show the path dies well before us. Here are a couple of examples that show 149.137.136.15 good and 149.137.136.16 bad:
This one is from Michigan:
$ mtr -w 149.137.136.15 Password: Start: 2025-11-12T13:39:10-0500 HOST: ip-192-168-0-231.ec2.internal Loss% Snt Last Avg Best Wrst StDev 1.|-- ip-192-168-0-1.ec2.internal 0.0% 10 4.0 2.9 2.5 4.0 0.4 2.|-- ip-192-168-1-254.ec2.internal 0.0% 10 3.3 3.3 2.6 3.5 0.2 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__108-2D202-2D236-2D1.ligh... 0.0% 10 5.4 5.0 4.2 6.5 0.8 4.|-- 71.151.199.86 0.0% 10 6.5 5.8 4.8 6.9 0.6 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 7.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 9.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae3.cr12-2Dchi1.ip4.gtt.... 0.0% 10 14.7 14.8 13.7 15.9 0.6 10.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae4.cr3-2Dwas1.ip4.gtt.n... 0.0% 10 34.7 34.9 34.0 35.4 0.4 11.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ip4.gtt.net&d=DwICAg&c=e... 0.0% 10 34.7 34.6 33.7 35.7 0.5 12.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__api005.backblazeb2.com&d... 0.0% 10 34.8 35.1 34.1 37.5 0.9
$ mtr -w 149.137.136.16 Start: 2025-11-12T13:39:54-0500 HOST: ip-192-168-0-231.ec2.internal Loss% Snt Last Avg Best Wrst StDev 1.|-- ip-192-168-0-1.ec2.internal 0.0% 10 2.5 2.8 2.5 3.0 0.1 2.|-- ip-192-168-1-254.ec2.internal 0.0% 10 3.4 3.5 3.0 3.8 0.2 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__108-2D202-2D236-2D1.ligh... 0.0% 10 4.0 5.3 3.9 8.9 1.6 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
This one is from East Bay, California:
$ mtr -w 149.137.136.15 Start: 2025-11-11T14:54:50-0800 HOST: maureen Loss% Snt Last Avg Best Wrst StDev 1.|-- router.lan 0.0% 10 0.6 0.6 0.4 0.8 0.1 2.|-- 192.168.1.254 0.0% 10 1.6 1.5 1.4 1.7 0.1 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__99-2D189-2D116-2D1.light... 0.0% 10 19.7 19.9 19.6 20.6 0.3 4.|-- 71.148.180.125 0.0% 10 20.7 20.2 20.0 20.7 0.3 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 6.|-- 32.130.91.83 40.0% 10 23.1 34.2 22.3 91.9 28.3 7.|-- 192.205.32.98 0.0% 10 29.4 24.9 23.1 30.2 2.6 8.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ae4.cr3-2Dwas1.ip4.gtt.n... 0.0% 10 81.9 81.9 81.6 82.5 0.3 9.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__ip4.gtt.net&d=DwICAg&c=e... 0.0% 10 82.0 82.0 81.7 82.5 0.2 10.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__api005.backblazeb2.com&d... 0.0% 10 82.4 82.1 81.3 82.9 0.4
$ mtr -w 149.137.136.16 Start: 2025-11-11T13:49:14-0800 HOST: maureen Loss% Snt Last Avg Best Wrst StDev 1.|-- router.lan 0.0% 10 0.7 0.7 0.5 0.7 0.1 2.|-- 192.168.1.254 0.0% 10 1.5 1.5 1.4 1.5 0.1 3.|-- https://urldefense.proofpoint.com/v2/url?u=http-3A__99-2D189-2D116-2D1.light... 0.0% 10 20.3 20.1 19.6 20.8 0.3 4.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
If there is anybody on this list that can throw me a bone, it would be very much appreciated.
--
Anthony Hoppe Sr. Network Engineer ahoppe@backblaze.com https://urldefense.proofpoint.com/v2/url?u=http-3A__Backblaze.com&d=DwICAg&c...
-- This email, including its contents and any attachment(s), may contain confidential and/or proprietary information and is solely for the review and use of the intended recipient(s). If you have received this email in error, please notify the sender and permanently delete this email, its content, and any attachment(s). Any disclosure, copying, or taking of any action in reliance on an email received in error is strictly prohibited. ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.outages.org_accou... To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.outages.org_archi...
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org NOTICE: The information contained in this message is confidential and for the sole use of the intended recipient/s. If you are not the intended recipient, please (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
________________________________
CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You must destroy the original transmission and its attachments without reading or saving in any manner. Thank you. ______________________________________________ Outages mailing list outages@outages.org Sign up for an account https://lists.outages.org/accounts/signup/ To subscribe send an email to outages-join@outages.org To unsubscribe send an email to outages-leave@outages.org To contact the list owners outages-owner@outages.org Archives https://lists.outages.org/archives/list/outages@outages.org/
Thank you for using Outages Lists! For post outage discussion, please redirect to outages-discussion@outages.org
participants (7)
-
Allan Paschall -
Anthony Hoppe -
Chris Woodfield -
J. Hellenthal -
jd＠boldnetworks.net -
Nimrod Levy -
noc