At 16:26 UTC our monitoring system indicated elevated load on our API and database infrastructure. At 16:28 UTC this issue escalated further and caused an outage in our API services. As the alarm went out, our 24/7 operations team started the recovery and all systems were operational by 16:39 UTC. We had two brief performance degradations during the recovery process (16:48-16:50 and 17:06-17:08) that affected a small number of our customers.
The issue was caused by a misconfiguration in o...