Over the weekend, POS Portal experienced a server disk space constraint causing the Production API servers to be unresponsive to all API calls. This caused degraded performance on all systems connected with the POS Portal API servers. The duration of the outage was caused by a gap in the system monitoring. Once the issue was identified, it was resolved within 12 minutes.
The gap in the monitoring as been resolved and all other monitoring is being reviewed and updated as needed. The Development team will be engaged to determine how to prevent future constraints.
Incident Timeline:
Start: 2/16/19 7:18 AM PST
Resolved: 2/18/19 at 7:55 AM PST