Summary
The incident caused problems logging in to SuperOffice Online for about 5% of our customers
Current status.
The problem has been resolved and all system are working normally
What happened?
Investigations has shown that the reason for some customers sites being “down” is that during the nightly recycle they lost their configuration and was left in an unavailable state.
Every night all customer instances are being recycled, Part of the process for each instance when starting up after a recycle is to read its configuration using a configservice. This night the configservice was not responding as expected due to a lock, resulting in several sites not being able to fetch their configuration,
What was done to resolve the problems?
The affected customers had to be manually recycled to receive a working configuration. We are continuing to investigate the reason for this failing configservice and will implement mechanisms for avoiding this in the future
We sincerely regret any inconvenience this disruption has caused.