Check scheduling delay

Incident Report for Checkly

Postmortem

Duration: 9:37 - 9:40 UTC

Incident Summary:

On 10th June 2024, a deployment caused the Check Scheduler service to scale down to 0 instances for a brief period of 3 minutes.

Root Cause:

An incorrect feature flag configuration led to the unintended scaling down of the Check Scheduler service during a deployment.

Impact:

  • High Frequency Checks (<5 minutes): Skipped executions during the incident.
  • Other Checks: Experienced a brief scheduling delay of about 3 minutes.

Resolution:

The feature flag setting was corrected, and the Check Scheduler service was scaled back to normal.

We apologize for any problems this may have caused. If you have any questions or concerns feel free to contact us via support@checklyhq.com

Posted Jun 10, 2024 - 10:16 UTC

Resolved

This incident has been resolved.
Posted Jun 10, 2024 - 10:15 UTC

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Jun 10, 2024 - 09:47 UTC

Update

We are continuing to investigate this issue.
Posted Jun 10, 2024 - 09:47 UTC

Investigating

We are currently investigating this issue.
Posted Jun 10, 2024 - 09:38 UTC
This incident affected: Check Scheduler.