Incident with Issues, API Requests and Pages

Incident Report for GitHub

Resolved

On April 23, 2025, between 07:00 UTC and 07:20 UTC, multiple GitHub services experienced degradation caused by resource contention on database hosts. The resulting error rates, which ranged from 2–5% of total requests, led to intermittent service disruption for users. The issue was triggered by heavy workloads on the database leading to connection saturation.
The incident mitigated when the database throttling activated which allowed the system to rebalance the connections. This restored the traffic flow to the database and restored service functionality.
To prevent similar issues in the future, we are reviewing the capacity of the database, improving monitoring and alerting systems, and implementing safeguards to reduce time to detection and mitigation.

Posted Apr 23, 2025 - 08:00 UTC

Update

A brief problem with one of our database clusters caused intermittent errors around 07:05 UTC for a few minutes. Our systems have recovered and we continue to monitor.

Posted Apr 23, 2025 - 07:47 UTC

Update

Issues is operating normally.

Posted Apr 23, 2025 - 07:43 UTC

Update

Actions is operating normally.

Posted Apr 23, 2025 - 07:42 UTC

Update

Pages is operating normally.

Posted Apr 23, 2025 - 07:42 UTC

Update

API Requests is operating normally.

Posted Apr 23, 2025 - 07:42 UTC

Update

Codespaces is operating normally.

Posted Apr 23, 2025 - 07:41 UTC

Update

Codespaces is experiencing degraded performance. We are continuing to investigate.

Posted Apr 23, 2025 - 07:23 UTC

Update

Actions is experiencing degraded performance. We are continuing to investigate.

Posted Apr 23, 2025 - 07:22 UTC

Investigating

We are investigating reports of degraded performance for API Requests, Issues and Pages

Posted Apr 23, 2025 - 07:17 UTC

This incident affected: API Requests, Issues, Actions, Pages, and Codespaces.