SEO

Barry Schwartz on November 12, 2021 at 9:43 am

For the past several days, 2 million or so of the domains hosted on SiteGround were potentially not being crawled and thus indexed by Google Search. There was some sort of “network issue between AWS Global Accelerator service and Google,” the company said and as of this morning, the issue was resolved.

When the issue began. Matt Tutt first reported about the issue this past Tuesday, November 9th. So the issue started sometime before November 9th, Matt suspects it started as early as Monday, November 8th.

Here is a screenshot showing how Google Search Console’s URL inspection tool was unable to access sites hosted on SiteGround. As you can see from the screenshot below, Google is reporting that it failed to crawl the page. Matt posted more debugging details on his post.

Confirmation. Then on November 10th, SiteGround confirmed the issue and said it is investigating. “We have escalated the issue to Google and we are working to troubleshoot and identify the cause of the problem,” the company said.

The issue. On November 11th, SiteGround confirmed the issue was between Amazon Web Services and Google. The company said “we traced it down to a network issue between AWS Global Accelerator service and Google. We’re collaborating with engineers from both teams to fix it.”

Resolved. Then a day later, on November 12th, SiteGround confirmed the issue was resolved and that it can take a bit more time for DNS to update but once the update is propagated properly, Google will once again be able to crawl sites hosted on SiteGround. The company said “We are glad to inform you that we have implemented a fix for the Google bot crawling issue experienced by some sites. Websites are already being crawled successfully. Please allow a few hours for the DNS changes to take effect.”

Google advice. John Mueller of Google posted some advice on Twitter on how Google deals with these outages. In short, don’t worry too much, the issues you may have experience from the outage will auto correct and “settle down.” There won’t be any “lasting effects” to the outage, John added. John posted several tweets, here is the first one if you want to click on it to read through the rest.

Why we care. If you are one of your clients are one of the two million domains hosted on SiteGround, you may have been impacted by this. That means any new or updated content or pages on your site was invisible to Google for most of the work week.

The issue is resolved and those pages should be crawled by Google going forward. But you may want to annotate your analytics and reporting if you were impacted by this crawling issue.

---