Since implementing the second production fix yesterday, the platform has been stable and working now as expected. We are closing this incident.
Posted Oct 05, 2023 - 08:48 EDT
Update
8 AM EDT update - the cloud.gov support has deployed the additional fix to production and is now monitoring the system.
Posted Oct 04, 2023 - 07:59 EDT
Update
16:15 EDT update - the cloud.gov team is still seeing some spikes in traffic after the production roll-out. The team is working on an additional fix that will be deployed once it passes testing in lower environments. We will update this incident once this additional fix is deployed to production.
Posted Oct 03, 2023 - 16:17 EDT
Monitoring
The cloud.gov support team has deployed a fix to production and will be monitoring the system for the rest of the day.
Posted Oct 03, 2023 - 12:33 EDT
Update
7:45 EDT update - the cloud.gov support team is aware of another traffic spike yesterday evening and is working on a solution to the issue. Currently that solution is in deployment/testing in lower environments and the team expects to deploy the fix into production later on today.
Posted Oct 03, 2023 - 07:49 EDT
Identified
Twice today the cloud.gov platform experienced high amounts of internal traffic on the platform which caused some customer applications to experience HTTP 504 messages while accessing their applications. These where brief periods of time, around 4 minutes each time, and the platform recovered automatically. At no time did customers applications stop or go down on the platform.
At this time the cloud.gov support team has identified the issue, is monitoring for future events, and working on implementing a solution into production to mitigate future events. At this time the platform is fully available.
Posted Oct 02, 2023 - 15:22 EDT
This incident affected: cloud.gov customer applications (Applications) and cloud.gov customer access (API).