Googlebot
Googlebot is the indexing robot used by Google to index web pages and documents to build a search for Google Search. There are two types of crawlers: Googlebot Desktop and Googlebot Mobile for mobile devices. Both simulate user in these two environments.
Googlebot by Google
According to Google itself on its developer website Advanced SEO section , "Googlebot is the generic name for Google's web crawler. This name is used for two different types of crawlers: one for computers and another for mobile devices that simulate a user's actions in each of these environments."
Your website will likely be crawled by both Googlebot Desktop and Googlebot Mobile. You can identify the Googlebot subtype by looking at the user agent string in the request.
However, both types of crawler obey the same product token (user agent token) in robots.txt . Therefore, it is not possible to selectively target Googlebot for smartphones or Googlebot for computers using robots.txt.



Post comment