Degraded performance on telephony

Incident Report for Dixa

Postmortem

Affected services: Connectivity Issues towards Danish phone numbers.

Date: 28/09-2021

The following information has been provided to us by our telephony partners.

Sequence of events:
28/09 06:48: Major Incident impacting all telephony services to Danish phone numbers.
28/09 06:48: Investigation initiated.
28/09 10:35: Issue identified.
28/09 11:00: Defective switch disconnected.
28/09 11:19: Major Incident closed, most telephony services normalized.
29/06 14:27: Operations fully normalized.

Root cause:
A defective hardware switch at a local telephony partner causing Customer-Premise Equipment (CPE) and network overload, impacting both primary and secondary CPEs, including failover between datacentres.

Corrective actions:
Long-term: Replacement of defective switch in next scheduled Service Window. No customer impact is expected, as traffic will be run over switch in alternative datacentre whilst the replacement is ongoing.

We deeply regret the inconvenience this incident has caused you and your customers.

Posted Oct 01, 2021 - 17:06 CEST

Resolved

We are happy to report that the incident with inbound calls to Danish phone numbers has been resolved and traffic delivery is stable.

We apologize once more for the inconvenience this has caused you and your customers.

Thank you for your patience. Postmortem to follow.

Posted Sep 28, 2021 - 12:20 CEST

Update

Our telephony partners report that service to Danish numbers has been restored, and our testing has also confirmed this.

We continue to monitor the situation closely.

Posted Sep 28, 2021 - 11:44 CEST

Monitoring

We see some improvements and some Danish calls are starting to come through.

We are closely monitoring the situation and will post a new update in 30 minutes.

Posted Sep 28, 2021 - 11:19 CEST

Update

Our phone carrier acknowledges ongoing connectivity issues with Danish numbers which are being worked on in order to resolve as soon as possible.

We apologize sincerely for the inconvenience this causes.

An update will be posted in 30 minutes.

Posted Sep 28, 2021 - 10:44 CEST

Update

We are still in contact with our Danish carriers and the issue is being treated with high priority.

An update will be posted in 30 minutes.

Posted Sep 28, 2021 - 10:12 CEST

Update

We are continuing to work on a fix for this issue.

Posted Sep 28, 2021 - 08:38 CEST

Identified

We are getting reports of phone calls being dropped.
This is caused by an outage at one of our danish telephony carriers.

We are in close contact with them, and will update this page as soon as we know anything.

We are sorry for the inconvenience this is causing.

Posted Sep 28, 2021 - 08:35 CEST

This incident affected: Telephony & SMS (Inbound, Outbound).