Data Center network outage
Incident Report for SuperOffice
Postmortem

SuperOffice CRM Online experienced some instability last week. The service has been unavailable for short periods of time, ranging from 6-18 minutes four times.

The root cause of the error could not be linked to a single component, but to a number of components of the network infrastructure, all the way from ISP to server clusters.

Errors in the infrastructure caused connections to be dropped with a wide variety of error messages to appear and resulting in service unavailability for some users/customers.

To mitigate the problems we activated our fail-over / fallback mechanisms, changed physical and virtual infrastructure components and changed network point of entry from one Oslo Data Center to our second redundant Center. We have also done changes in the application to handle instability better and deployed this to all customers.

We strongly apologize for the inconvenience and believe we have taken steps to prevent similar future problems.

Posted 15 days ago. Sep 09, 2019 - 17:53 CEST

Resolved
We have received a status from our ISP that the issue has been resolved. All systems are now back to normal.
Posted 18 days ago. Sep 06, 2019 - 11:34 CEST
Update
We are still experiencing communication issues and services can be unstable, we are working on finding a permanent solution.
Posted 18 days ago. Sep 06, 2019 - 11:11 CEST
Monitoring
A fix has been implemented and we are monitoring the results.
Posted 18 days ago. Sep 06, 2019 - 10:33 CEST
Identified
The issue has been identified and a fix is being implemented.
Posted 18 days ago. Sep 06, 2019 - 10:30 CEST
Investigating
Our Service Provider has an outage hence communication between some services is lost. We are working on finding a solution for this.
Posted 18 days ago. Sep 06, 2019 - 10:19 CEST
This incident affected: SuperOffice CRM Online (Login services, Sales & Marketing Client, Service Client).