Increased number of failed calls

January 9, at 10:35 CET CLASSOUND Service faced critical call routing issues because of a wrong configuration inserted by human error that caused routing errors in the production clusters.

Recovery of 100 % of incoming routes and 95 % of outgoing routes was achieved by 13:20 CET. 5% of the remaining outgoing routes were restored by 16:30 CET.

The configuration issue was not detected on the stage clusters due to a failure of auto-tests. All necessary improvements are being applied to auto-tests and release procedures to prevent a similar situation from happening in the future.

Service has been restored. Recovery of 100 % of incoming routes and 95 % of outgoing routes was achieved by 11:20 GMT, 5 % of remaining outgoing routes was restored by 14:30 GMT. In case you are still experiencing any issues, please contact our technical support.

We have observed service recovery and finalizing repairs. This incident will be resolved when we confirm all repairs are complete

We involved additional teams in resolving this issue and we also took measures to mitigate the impact of this incident as much as possible.

We are observing an increased number of failed calls and currently working on this issue.