At approximately 9AM PST, the Avochato cloud platform began receiving double the average volume of requests for that time period. This appeared to be due to high platform utilization at the top of the hour. Our system scaled to handle this load and no requests to our servers were dropped, however the response was delayed as our system queued the requests and handled them in order. This began to slowly impact the end-user experience, with latency peaking at roughly 9:30am PST, and subsequently began to decrease.
Messages that needed to be delivered to customers were queued and delivered during this time. However, accessing conversations suffered from 5-7 second delays. The root cause appeared to be a bottleneck in our cloud infrastructure preventing our servers from handling the sheer load, causing requests to be queued.
Latency persisted for some users until approximately 10:05am PST and is now has regressed to the mean.
While our systems automatically adjusted to handle the load, the time to do so left users with a suboptimal experience for navigating Avochato inboxes. The Avochato operations team investigated the source of the volume and took action to limit the volume of requests that were being caused by outsized usage.
Tonight we will be performing maintenance to address this bottleneck and continue increasing the threshold of concurrent volume that can impact the user experience.
We know how important it is to quickly respond to your customers and appreciate the trust you place in our team to deliver the best possible experience for those important to you.
Thanks again for choosing Avochato,
Christopher