Network disruption

Incident Report for SuperOffice

Postmortem

Incident Summary
On Monday, July 7, 2025, a planned network change by one of our infrastructure providers caused a complete loss of connectivity starting at 12:42 CEST. This affected all major services, including login, access to applications, APIs, and customer support channels (except phone).

We worked closely with the provider and took immediate action. Connectivity was restored by 13:43 CEST, and all systems were fully back online by 14:52 CEST. The incident was officially closed at 15:12 CEST.

Impact Overview

  • Affected Service: online.superoffice.com
  • Customer Impacted: All customers
  • Downtime (SLA): 1 hour and 6 minutes
  • Resolution Time: 50 minutes

Root Cause (Simplified)
The outage occurred during a planned technical preparation for migrating the SuperOffice environment to a new infrastructure platform. As part of this process, network settings were updated to shift from an older system to a newer one. However, due to an unexpected interaction between two systems—ACI and VMware—the network assigned a different identifier than expected, which disrupted access to the service.

Although SuperOffice operates in a dedicated environment designed for better reliability, this incident was caused by a misconfiguration during the migration setup. Once the network was restored, additional steps were needed to restart web services due to errors encountered during testing.

Response Actions

  • Incident response team was assembled within 5 minutes of detection.
  • Communication was restored by manually aligning network identifiers with security system settings.
  • Web services were restarted to resolve access errors.

Preventive Measures & Follow-Up Tasks

  • Migration preparation steps will now be included in the main maintenance window to avoid preemptive disruptions.
  • Integration of internal and external status pages is planned to ensure synchronized incident updates.
Posted Jul 23, 2025 - 15:35 CEST

Resolved

This incident has been resolved.
Posted Jul 07, 2025 - 15:37 CEST

Monitoring

A fix has been implemented and we are currently monitoring the results. Please note that due to high demand as many users try to access the system at once, you may experience slower performance. We appreciate your patience and will continue to keep a close eye on system stability.
Posted Jul 07, 2025 - 14:21 CEST

Identified

The issue has been identified and a fix is being implemented.
Posted Jul 07, 2025 - 13:12 CEST

Investigating

We are currently investigating Network disruption that is causing intermittent availability issues with the SuperOffice CRM Cloud
Posted Jul 07, 2025 - 12:53 CEST
This incident affected: SuperOffice CRM Cloud (Login services, Sales & Marketing Client, Service Client, WebTools, API and Apps).