How To Control Googlebot’s Interaction With Your Website

Google’s Search Relations answered several questions regarding webpage indexing on the latest episode of the ‘Search Off The Record’ podcast.

[{"selector":"#anim-caf65539-ffdf-48dc-99f8-1c091beb2b4b","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

The topics discussed were how to block Googlebot from crawling specific sections of a page and how to prevent Googlebot from accessing a site altogether.

[{"selector":"#anim-c0a0f840-804b-4e5c-bd10-a63a1a58130f","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

Google’s John Mueller and Gary Illyes answered the questions examined in this article.

[{"selector":"#anim-628d4d81-ffb2-48a9-a639-35e998dbe918","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

Mueller says it’s impossible when asked how to stop Googlebot from crawling specific web page sections, such as “also bought” areas on product pages.

[{"selector":"#anim-f839dfe0-e55c-4136-a6e6-6e98668510ef","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}] [{"selector":"#anim-539da22d-a630-475a-a943-8c8e86317760","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

“The short version is that you can’t block crawling of a specific section on an HTML page,” Mueller said. He went on to offer two potential strategies for dealing with the issue, neither of which, he stressed, are ideal solutions.

[{"selector":"#anim-2cce377e-501a-4532-99d7-6164468ce968","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

Mueller suggested using the data-nosnippet HTML attribute to prevent text from appearing in a search snippet. Alternatively, you could use an iframe or JavaScript with the source blocked by robots.txt, although he cautioned that’s not a good idea.

[{"selector":"#anim-b8fc8978-c77a-498a-bc7b-197c9175cdd7","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

How To Control Googlebot’s Interaction With Your Website

Google's Search Relations team provides insights into controlling Googlebot's interactions with webpages on the latest 'Search Off The Record' podcast.

Brought to you by Trickyenough

Google’s Search Relations answered several questions regarding webpage indexing on the latest episode of the ‘Search Off The Record’ podcast.

Brought to you by Trickyenough

The topics discussed were how to block Googlebot from crawling specific sections of a page and how to prevent Googlebot from accessing a site altogether.

Brought to you by Trickyenough

Google’s John Mueller and Gary Illyes answered the questions examined in this article.

Brought to you by Trickyenough

Mueller says it’s impossible when asked how to stop Googlebot from crawling specific web page sections, such as “also bought” areas on product pages.

Blocking Googlebot From Specific Web Page Sections

Brought to you by Trickyenough

“The short version is that you can’t block crawling of a specific section on an HTML page,” Mueller said. He went on to offer two potential strategies for dealing with the issue, neither of which, he stressed, are ideal solutions.

Brought to you by Trickyenough

Mueller suggested using the data-nosnippet HTML attribute to prevent text from appearing in a search snippet. Alternatively, you could use an iframe or JavaScript with the source blocked by robots.txt, although he cautioned that’s not a good idea.

Brought to you by Trickyenough