Googlebot

Googlebot is the indexing robot used by Google to index web pages and documents to build a search for Google Search. There are two types of crawlers: Googlebot Desktop and Googlebot Mobile for mobile devices. Both simulate user in these two environments.

Googlebot by Google

According to Google itself on its developer website Advanced SEO section , "Googlebot is the generic name for Google's web crawler. This name is used for two different types of crawlers: one for computers and another for mobile devices that simulate a user's actions in each of these environments."

Your website will likely be crawled by both Googlebot Desktop and Googlebot Mobile. You can identify the Googlebot subtype by looking at the user agent string in the request.

However, both types of crawler obey the same product token (user agent token) in robots.txt . Therefore, it is not possible to selectively target Googlebot for smartphones or Googlebot for computers using robots.txt.

Post comment

Semantic Blog
Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognizing you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.