Inconsistencies in agent interface
Incident Report for Dixa
Postmortem

Between August 29th and August 31st, we experienced performance issues within the search area of the Dixa platform. Conversation lists within the queues were showing inconsistencies in the status, which meant that conversations were appearing as open even though they were already closed, as well as not appearing on the list at all.

The inconsistencies in the search were caused by a version upgrade of the infrastructural components of search which caused delays in data ingestion.  

We would like to emphasise that the other areas like: Queue dashboard, Agents presence, and conversation offers were not affected by this and the conversations were being offered to available and assigned agents based on what was waiting the longest and according to queue priorities.  

Timeline (CEST)

August 29th, 2023 3:48 PM CEST - We observe a number of inconsistencies across some Dixa platforms.

August 29th, 4:35 PM CEST - The fix was deployed which stopped new occurrences of this issue from happening 

August 29th, 6:13 PM CEST - We moved the incident status to Identified and we were continuing working on resolving the inconsistencies in conversations that were created after the Search version upgrade was completed until the deployment was implemented earlier on the 29th of August at 4:35 PM CEST 

August 30th, 5:48 PM CEST - Majority of the inconsistencies were resolved by then, but we continue working on the few remaining cases 

August 31st, 11:10 AM CEST - The work to resolve the inconsistencies has been finalized and we moved the status to Monitoring

August 31st, 7:47 PM CEST - After an extended period of Monitoring the incident was Resolved.

Posted Sep 07, 2023 - 10:21 CEST

Resolved
This incident has been resolved. We apologise for the inconvenience it has caused.

Post mortem will be posted within 5 business days.
Posted Aug 31, 2023 - 19:47 CEST
Monitoring
A solution has been implemented and we are monitoring the results.
Posted Aug 31, 2023 - 11:10 CEST
Update
The solution is being implemented and majority of cases affected by this incident are resolved by now. We are working on the remaining cases but the impact is very minor at this point.
We will keep updating this page when further progress is made.
Posted Aug 30, 2023 - 17:48 CEST
Update
We continue working on the solution as we see in some cases the conversations list shows conversations as open whereas it was already closed.
The conversations are being offered as intended and the number of the conversations in the backlog on the dashboard is correct, that means conversations will be offered to agents who are set to working in Dixa and assigned to the queue.
Posted Aug 30, 2023 - 09:22 CEST
Update
We are continuing to work on a fix to this issue and will provide the next status update by 9:30 am CEST on Wednesday.
Posted Aug 29, 2023 - 22:29 CEST
Identified
After deploying an initial fix we can see significant improvement in the search results but in some cases we still observe an inconsistency between the number of the conversations on the dashboard and what is shown in the search. Therefore we continue working on the solution for this.
Posted Aug 29, 2023 - 18:13 CEST
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Aug 29, 2023 - 16:35 CEST
Investigating
Today, customers reported discrepancy issues between the dashboard, conversation overview and search.
Unfortunately, search and conversation overviews have been affected. The number of queued conversations showing in the dashboard is correct.

Our developers are working on a solution for the issue at hand - please accept our apologies for any inconvenience caused.

We will update this page with any new information regarding any progress.
Posted Aug 29, 2023 - 15:48 CEST
This incident affected: Agent Interface (Agent Interface).