Resolved -
We monitored this over the weekend and did various checks today, and we believe we have resolved the incident. Customer Support has already contacted all customers who were impacted.
Sep 15, 14:44 CEST
Update -
Technical fixes are deployed, and all services are stable. We're maintaining monitoring through the weekend before final incident closure on Monday.
Detailed customer communication and a full postmortem will follow early next week.
Any weekend developments will be posted immediately.
Sep 12, 18:37 CEST
Update -
Latest Progress:
We have successfully deployed further performance optimizations and fixes to production.
Current Status:
- Database performance remains stabilized with the new optimizations
- Integration services are being closely monitored
- Real-time monitoring is active to ensure continued stability
Next Steps:
Our engineering team will continue monitoring system performance over the next few hours to confirm the effectiveness of our fixes and carry out various tests. We prepared additional safeguards if needed.
We will provide additional updates as soon as our monitoring confirms that it is stable across all services.
Sep 12, 15:51 CEST
Update -
Current Status:
We have successfully deployed an initial fix that has stabilized our systems. Database performance has returned to normal levels, and we are no longer experiencing the issues that caused integration disruptions.
Next Steps:
Our engineering team is deploying additional performance optimizations within the next 1-2 hours. These enhancements will provide long-term stability and allow us to safely restore all affected functionality.
Monitoring:
System performance metrics have returned to baseline levels. Our team maintains active monitoring to ensure continued stability throughout the optimization deployment.
We expect to provide another update within 2-3 hours following the completion of our performance enhancements. Thank you for your continued patience as we work toward full resolution.
Sep 12, 13:21 CEST
Monitoring -
Database Performance Issue Mitigated
We successfully mitigated a database performance issue that was causing service slowdowns.
Impact:
- Check runs experienced temporary slowdowns
- A small number of check runs timed out during the incident. If any check run was impacted, restart it and it should work normally
- There was no widespread service disruption
The underlying database performance issue has been mitigated
All services are operating normally
The issue is now resolved, and we continue monitoring system performance.
Sep 12, 12:10 CEST
Identified -
The issue has been identified and a fix is being implemented.
Sep 12, 10:59 CEST
Investigating -
We are currently experiencing performance issues affecting multiple GitGuardian services. Our engineering team is actively investigating and working to resolve these issues as quickly as possible.
Current Impact:
- Dashboard may load slowly or fail to load intermittently
- Public API response times are degraded, which may cause slower performance in ggshield and other integrations
What we're doing:
Our engineering team has been alerted and is actively investigating the root cause. We will provide updates as significant developments occur.
Sep 12, 10:39 CEST