On Friday 25th of August at 11:00 CEST, we had a partial outage that affected the conversation offers, the agent presence and the performance of Dixa with reports of messages not loading, offers not being accepted and general slowness of the platform. The incident was caused due to an exhaustion of capacity on one of our systems tied to the above functions.
The incident was resolved by restarting the services and rolling back a previous update of the agent interface which brought Dixa back to normal.
We thank you for your patience and apologize for the inconvenience this caused you. We have taken action to prevent this from happening again.
August 25, 2023 10:52 AM: A number of services connected to the agent interface ran out of memory
August 25, 2023 11:00 AM: We noticed a sudden drop in incoming agent connections followed by a rapid recovery.
August 25, 2023 11:18 AM: Reports are coming in to Dixa support team about messages not loading, offers not being accepted and general slowness of the platform.
August 25, 2023 11:26 AM: Incident process is triggered and status page is updated.
August 25, 2023 11:30 AM: Issue identified as one of our services running out of memory without automatically restarting. The services are restarted manually.
August 25, 2023 11:35 AM: Rolled back a product update that relied on that process to an older version.
August 25, 2023 12:00 PM: Service recovered.