Increased error rate of sending SMS alerts

Incident Report for Checkly

Postmortem

After a recent change in our SMS alerting service, we’ve introduced a regression which caused an increased error rate in SMS alert delivery to certain countries (list of affected countries with country codes below). After analysing our logs we determined that alerts were not correctly delivered between 2024-02-06 11:20 UTC and 2024-02-08 09:00 UTC. We’ve implemented a fix and improved the sensitivity of our alarms in the SMS service to prevent similar issues from happening in the future.

We apologize for any problems this may have caused. If you have any questions or concerns feel free to contact us via support@checklyhq.com

List affected countries, country codes

Belgium: +32
US: +1
Brazil: +55
Australia: +61
Turkey: +90

Posted Feb 08, 2024 - 10:23 UTC

Resolved

This incident has been resolved.

Posted Feb 08, 2024 - 09:39 UTC

Monitoring

We've identified the issue and updated our config to resolve it. We are monitoring the effectiveness of our fix.

Posted Feb 08, 2024 - 09:02 UTC

Investigating

We've identified that our SMS alerts in certain countries are not delivered successfully. We are investigating the root cause

Posted Feb 08, 2024 - 08:32 UTC

This incident affected: Alerting.