Summary
The SuperOffice CRM Online service has experienced instability first two weeks of September and several users were not able to access the full service for periods of time.

There were two unrelated issues that were the main causes for the down time – disruptions in network traffic components and a 15 minute internet outage in our data center on September 6.

SuperOffice CRM Online uses 2 different clusters (Online and Online2). Customers on Online1 (65%) have been more affected by the incident than customers on Online2.

Instabilities occurred on several occasions in the period September 3-16. On September 11 we temporarily suspended some of the functionality related to document- and mail handling that affected all customers (namely “WebTools”). The suspension was resolved on September 12 for Online2 and on September 13 for Online1. A new version of WebTools were release on September 13 and all users were prompted and required to update to the new release (v 11.5). On the September 19 all previous versions of WebTools stopped working to safeguard the stability of the service.

Current status.
Problems are resolved and all systems are operating as normal as of Sept 18th. Updating WebTools to v.11.5 will enable full functionality for document- and mail-handling.

What happened?
The incident was initially caused by disruptions in one of the network traffic components. Although the CRM Online service is designed to be fault tolerant, the component providing document management and mail integration (WebTools) was not functioning properly during the network instability. The result was increased load on our servers and network.

What was done to resolve the problems?
On September 5, we moved all operations to our second physical datacenter and restored stability. When problems re-occurred on September 11, Webtools functionality was suspended to reduce load on the system. A manual workaround was published and available to all users. This issue was resolved with the release of a new version (v.11.5) of the WebTools component on September 13.

If you have further questions regarding the incident, please contact your SuperOffice representative.

We sincerely regret any inconvenience this disruption has caused you or your organization.