Degraded API performance

MAJORaffectedWeb Application,APIlasted7h 51min 40s

This incident has been resolved.

resolved
Nov 18 19:35:57 (UTC)

There are no responses with a 500 status from Cloudflare anymore. We continue to monitor the situation closely.

monitoring
Nov 18 17:35:07 (UTC)

We still see a tiny fraction of responses with status code 500 for Status pages from Cloudflare. Beside that, there should be no customer impact. Check runs, API and Web App are fully functional. We keep monitoring the situation until Cloudflare resolves their incident.

monitoring
Nov 18 16:13:24 (UTC)

Status pages are now operational again. We’ll keep monitoring closely.

monitoring
Nov 18 14:49:44 (UTC)

We implemented a workaround to allow the public API and web UI to work normally again. Status pages are still down as they still rely on Cloudflare.

identified
Nov 18 14:31:42 (UTC)

Most requests are now succeeding again, although some (around 25%) are still failing.

identified
Nov 18 12:40:24 (UTC)

The web application and the API are still in a degraded state. Checks are still running as expected.

identified
Nov 18 12:13:08 (UTC)

We're seeing an increase in failure rate of our API caused by a 3rd party provider https://www.cloudflarestatus.com/

investigating
Nov 18 11:44:17 (UTC)