What is User-agent in the robots.txt? November 19, 2021 by admin –What is User-agent in the robots.txt?– –It names the website– –It names the page being crawled– –It names a web crawler– –None of the above– Related posts: URLs with a _______ status code should be included in a sitemap. URLs with a _______ status code should be included in a sitemap. What does the term “mobile first” mean? What is an example of a recommended area to block on your site? What is Crawl Rate Limit? What is one example of a site structure issue? What is one very important factor when optimizing for mobile? What is the Inclusion Ratio?