Google

AdsBot-Google is operated by Google. Ad-platform crawler that fetches landing pages for quality, policy, and creative checks. Blocking it typically breaks ad serving on the affected URLs.

AdsBot-Google-Mobile is operated by Google. Ad-platform crawler that fetches landing pages for quality, policy, and creative checks. Blocking it typically breaks ad serving on the affected URLs.

AdsBot-Google-Mobile-Apps is operated by Google. Brand-safety and ad-quality crawler, DoubleVerify, Moat, IAS, Criteo and similar, verifying that ads render in safe contexts. Allowed by default in most ad stacks.

APIs-Google is operated by Google. Ad-platform crawler that fetches landing pages for quality, policy, and creative checks. Blocking it typically breaks ad serving on the affected URLs.

Bard-AI is operated by Google. Crawls broadly across the public web to gather content for training new model versions. Site owners can opt out via robots.txt, though enforcement varies by operator.

Chrome Privacy Preserving Prefetch Proxy is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Chrome-Lighthouse is operated by Google. Site-monitoring, uptime, performance, or QA tooling, usually run by a third-party SaaS on behalf of someone who manages the site. Rarely affects rankings or visibility.

CloudVertexBot is operated by Google. Crawls broadly across the public web to gather content for training new model versions. Site owners can opt out via robots.txt, though enforcement varies by operator.

DuplexWeb-Google is operated by Google. Autonomously navigates websites, fills forms, and takes actions on behalf of a human user. Often runs a full headless browser and can perform any interaction a person could.

Feedfetcher-Google is operated by Google. Ad-platform crawler that fetches landing pages for quality, policy, and creative checks. Blocking it typically breaks ad serving on the affected URLs.

Gemini-AI is operated by Google. Crawls broadly across the public web to gather content for training new model versions. Site owners can opt out via robots.txt, though enforcement varies by operator.

Gemini-Deep-Research is operated by Google. Fetches a page in real time when an LLM is mid-answer to a user question. Traffic spikes correlate with question volume rather than crawl schedules, so it shows up unpredictably.

Gmail Image Proxy is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google Ad Manager Crawler is operated by Google. Brand-safety and ad-quality crawler, DoubleVerify, Moat, IAS, Criteo and similar, verifying that ads render in safe contexts. Allowed by default in most ad stacks.

Google AdsBot is operated by Google. Competitive, marketing, or threat-intelligence data collection. Often crawls competitor pages, ad creatives, or pricing pages on behalf of a buyer.

Google AdSense is operated by Google. Competitive, marketing, or threat-intelligence data collection. Often crawls competitor pages, ad creatives, or pricing pages on behalf of a buyer.

Google Adsense: Mediapartners-Google is operated by Google. Ad-platform crawler that fetches landing pages for quality, policy, and creative checks. Blocking it typically breaks ad serving on the affected URLs.

Google API is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google Area 120 Privacy Policy Fetcher is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google Calendar Importer is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google Cloud Function is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google Cloud Scheduler is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google Digital Asset Links is operated by Google. Maps link graphs, ranking signals, and audit data for SEO tooling. Typically run by SaaS platforms on behalf of marketers monitoring their own or competitor sites.

Google Docs is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google Favicon is operated by Google. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

Google Feed Fetcher is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google Firebase is operated by Google. Ad-platform crawler that fetches landing pages for quality, policy, and creative checks. Blocking it typically breaks ad serving on the affected URLs.

Google Image Proxy is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google Images is operated by Google. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

Google Inspection Tool is operated by Google. Site-monitoring, uptime, performance, or QA tooling, usually run by a third-party SaaS on behalf of someone who manages the site. Rarely affects rankings or visibility.

Google Page Renderer is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google Page Speed Insights is operated by Google. Site-monitoring, uptime, performance, or QA tooling, usually run by a third-party SaaS on behalf of someone who manages the site. Rarely affects rankings or visibility.

Google Partner Monitoring is operated by Google. Site-monitoring, uptime, performance, or QA tooling, usually run by a third-party SaaS on behalf of someone who manages the site. Rarely affects rankings or visibility.

Google Publisher Center is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google Read Aloud is operated by Google. Site-monitoring, uptime, performance, or QA tooling, usually run by a third-party SaaS on behalf of someone who manages the site. Rarely affects rankings or visibility.

Google Safety is operated by Google. Probes websites for vulnerabilities, exposed credentials, or compliance issues. Sometimes legitimate (your own pentest vendor) and sometimes hostile reconnaissance.

Google Schema Markup Testing Tool is operated by Google. Site-monitoring, uptime, performance, or QA tooling, usually run by a third-party SaaS on behalf of someone who manages the site. Rarely affects rankings or visibility.

Google Scholar is operated by Google. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

Google Search Console is operated by Google. Site-monitoring, uptime, performance, or QA tooling, usually run by a third-party SaaS on behalf of someone who manages the site. Rarely affects rankings or visibility.

Google Sheets is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google Site Verification is operated by Google. Probes websites for vulnerabilities, exposed credentials, or compliance issues. Sometimes legitimate (your own pentest vendor) and sometimes hostile reconnaissance.

Google Slides is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google Stackdriver Monitoring is operated by Google. Site-monitoring, uptime, performance, or QA tooling, usually run by a third-party SaaS on behalf of someone who manages the site. Rarely affects rankings or visibility.

Google StoreBot is operated by Google. Maps link graphs, ranking signals, and audit data for SEO tooling. Typically run by SaaS platforms on behalf of marketers monitoring their own or competitor sites.

Google Structured Data Testing Tool is operated by Google. Site-monitoring, uptime, performance, or QA tooling, usually run by a third-party SaaS on behalf of someone who manages the site. Rarely affects rankings or visibility.

Google Transparency Report is operated by Google. Probes websites for vulnerabilities, exposed credentials, or compliance issues. Sometimes legitimate (your own pentest vendor) and sometimes hostile reconnaissance.

Google Trust Services (DCV Check) is operated by Google. Site-monitoring, uptime, performance, or QA tooling, usually run by a third-party SaaS on behalf of someone who manages the site. Rarely affects rankings or visibility.

Google Videos is operated by Google. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

Google Web Preview is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google Web Snippet is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google-Ads-Conversions is operated by Google. Ad-platform crawler that fetches landing pages for quality, policy, and creative checks. Blocking it typically breaks ad serving on the affected URLs.

Google-AdWords-Express is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google-Adwords-Instant is operated by Google. Ad-platform crawler that fetches landing pages for quality, policy, and creative checks. Blocking it typically breaks ad serving on the affected URLs.

Google-Agent is operated by Google. Autonomously navigates websites, fills forms, and takes actions on behalf of a human user. Often runs a full headless browser and can perform any interaction a person could.

Google-AgentSpace is operated by Google. Autonomously navigates websites, fills forms, and takes actions on behalf of a human user. Often runs a full headless browser and can perform any interaction a person could.

Google-AP2-Agent is operated by Google. Autonomous browser agent specifically running shopping or checkout flows on behalf of a buyer. Volume is small today but growing fast as AI shopping assistants ship.

Google-BusinessLinkVerification is operated by Google. Probes websites for vulnerabilities, exposed credentials, or compliance issues. Sometimes legitimate (your own pentest vendor) and sometimes hostile reconnaissance.

Google-CloudVertexBot is operated by Google. Catalog crawler that indexes product feeds for AI-powered shopping experiences and answer engines. Allowing it is how products become discoverable inside agentic commerce.

Google-Display-Ads-Bot is operated by Google. Competitive, marketing, or threat-intelligence data collection. Often crawls competitor pages, ad creatives, or pricing pages on behalf of a buyer.

Google-Document-Conversion is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google-Extended is operated by Google. Crawls broadly across the public web to gather content for training new model versions. Site owners can opt out via robots.txt, though enforcement varies by operator.

Google-Firebase is operated by Google. Crawls broadly across the public web to gather content for training new model versions. Site owners can opt out via robots.txt, though enforcement varies by operator.

Google-Flights-Search-Spackle is operated by Google. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

Google-Gemini-CLI is operated by Google. AI-powered developer tool that fetches docs, code samples, and package metadata to ground code suggestions. Usually identifies itself with a Coding Agent or model-specific user-agent.

Google-InspectionTool is operated by Google. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

Google-Merchant is operated by Google. Catalog crawler that indexes product feeds for AI-powered shopping experiences and answer engines. Allowing it is how products become discoverable inside agentic commerce.

Google-NotebookLM is operated by Google. Fetches a page in real time when an LLM is mid-answer to a user question. Traffic spikes correlate with question volume rather than crawl schedules, so it shows up unpredictably.

Google-PageRenderer is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google-Pinpoint is operated by Google. Autonomously navigates websites, fills forms, and takes actions on behalf of a human user. Often runs a full headless browser and can perform any interaction a person could.

Google-Project-Mariner is operated by Google. Autonomously navigates websites, fills forms, and takes actions on behalf of a human user. Often runs a full headless browser and can perform any interaction a person could.

Google-Read-Aloud is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Google-Safety is operated by Google. Probes websites for vulnerabilities, exposed credentials, or compliance issues. Sometimes legitimate (your own pentest vendor) and sometimes hostile reconnaissance.

Google-Shopping is operated by Google. Catalog crawler that indexes product feeds for AI-powered shopping experiences and answer engines. Allowing it is how products become discoverable inside agentic commerce.

Google-Site-Verification is operated by Google. Maps link graphs, ranking signals, and audit data for SEO tooling. Typically run by SaaS platforms on behalf of marketers monitoring their own or competitor sites.

Google-Structured-Data-Testing is operated by Google. Maps link graphs, ranking signals, and audit data for SEO tooling. Typically run by SaaS platforms on behalf of marketers monitoring their own or competitor sites.

Google-Structured-Data-Testing-Tool is operated by Google. Site-monitoring, uptime, performance, or QA tooling, usually run by a third-party SaaS on behalf of someone who manages the site. Rarely affects rankings or visibility.

Google-TravelAds-Live is operated by Google. Ad-platform crawler that fetches landing pages for quality, policy, and creative checks. Blocking it typically breaks ad serving on the affected URLs.

Google-Trust-Services is operated by Google. Probes websites for vulnerabilities, exposed credentials, or compliance issues. Sometimes legitimate (your own pentest vendor) and sometimes hostile reconnaissance.

GoogleAgent-Mariner is operated by Google. Autonomously navigates websites, fills forms, and takes actions on behalf of a human user. Often runs a full headless browser and can perform any interaction a person could.

GoogleAgent-Search is operated by Google. Autonomously navigates websites, fills forms, and takes actions on behalf of a human user. Often runs a full headless browser and can perform any interaction a person could.

GoogleApps-DocumentScanner is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

GoogleAssociationService is operated by Google. Site-monitoring, uptime, performance, or QA tooling, usually run by a third-party SaaS on behalf of someone who manages the site. Rarely affects rankings or visibility.

Googlebot is operated by Google. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

Googlebot-IA is operated by Google. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

Googlebot-Image is operated by Google. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

Googlebot-Image-Smartphone is operated by Google. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

Googlebot-Mobile is operated by Google. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

Googlebot-News is operated by Google. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

Googlebot-Smartphone is operated by Google. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

Googlebot-Video is operated by Google. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

GoogleDocs is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

GoogleImageProxy is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

GoogleOther is operated by Google. Crawls broadly across the public web to gather content for training new model versions. Site owners can opt out via robots.txt, though enforcement varies by operator.

GoogleOther-Image is operated by Google. Crawls broadly across the public web to gather content for training new model versions. Site owners can opt out via robots.txt, though enforcement varies by operator.

GoogleOther-Video is operated by Google. Crawls broadly across the public web to gather content for training new model versions. Site owners can opt out via robots.txt, though enforcement varies by operator.

GoogleProducer is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

GoogleStackdriverMonitoring is operated by Google. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Jules is operated by Google. Autonomously navigates websites, fills forms, and takes actions on behalf of a human user. Often runs a full headless browser and can perform any interaction a person could.

Mediapartners is operated by Google. Ad-platform crawler that fetches landing pages for quality, policy, and creative checks. Blocking it typically breaks ad serving on the affected URLs.

Mediapartners-Google is operated by Google. Ad-platform crawler that fetches landing pages for quality, policy, and creative checks. Blocking it typically breaks ad serving on the affected URLs.

ProjectShield Url Check is operated by Google. Site-monitoring, uptime, performance, or QA tooling, usually run by a third-party SaaS on behalf of someone who manages the site. Rarely affects rankings or visibility.

ProjectShield-UrlCheck is operated by Google. Probes websites for vulnerabilities, exposed credentials, or compliance issues. Sometimes legitimate (your own pentest vendor) and sometimes hostile reconnaissance.

Storebot-Google is operated by Google. Crawls broadly across the public web to gather content for training new model versions. Site owners can opt out via robots.txt, though enforcement varies by operator.