On August 12, 2025, between 13:30 UTC and 17:14 UTC, GitHub search was in a degraded state. Users experienced inaccurate or incomplete results, failures to load certain pages (like Issues, Pull Requests, Projects, and Deployments), and broken components like Actions workflow and label filters.
Most user impact occurred between 14:00 UTC and 15:30 UTC, when up to 75% of search queries failed, and updates to search results were delayed by up to 100 minutes.
The incident was triggered by intermittent connectivity issues between our load balancers and search hosts. While retry logic initially masked these problems, retry queues eventually overwhelmed the load balancers, causing failure. The query failures were mitigated at 15:30 UTC after throttling our search indexing pipeline to reduce load and stabilize retries. The connectivity failures were resolved at 17:14 UTC after the automated reboot of a search host, causing the rest of the system to recover.
We have improved internal monitors and playbooks, and tuned our search cluster load balancer to further mitigate the recurrence of this failure mode. We continue to invest in resolving the underlying connectivity issues.
Posted Aug 12, 2025 - 17:56 UTC
Update
Service availability has been mostly restored, but some users will continue to see increased request latency and stale search results. We are still working towards full recovery.
Posted Aug 12, 2025 - 17:07 UTC
Update
Service availability has been mostly restored, but increased load/query latency and stale search results persist. We continue to work towards full mitigation.
Posted Aug 12, 2025 - 16:33 UTC
Update
We are seeing partial recovery in service availability, but still see inconsistent experiences and stale search data across services. Investigation and mitigations are underway.
Posted Aug 12, 2025 - 15:48 UTC
Update
We are experiencing increased latency in our API layers and inconsistently degraded experiences when loading or querying issues, pull requests, labels, packages, releases, workflow runs, projects, and repositories, among others. Investigation is underway.
Posted Aug 12, 2025 - 15:20 UTC
Update
We are investigating reports of degraded performance in services backed by search. The team continues to investigate why requests are failing to reach our search clusters.
Posted Aug 12, 2025 - 14:53 UTC
Update
Packages is experiencing degraded performance. We are continuing to investigate.
Posted Aug 12, 2025 - 14:30 UTC
Investigating
We are investigating reports of degraded performance for API Requests, Actions, Issues and Pull Requests
Posted Aug 12, 2025 - 14:12 UTC
This incident affected: API Requests, Issues, Pull Requests, Actions, and Packages.