ByteDance

Bytespider is operated by Bytedance. Crawls broadly across the public web to gather content for training new model versions. Site owners can opt out via robots.txt, though enforcement varies by operator.

Doubao-User is operated by Bytedance. Fetches a page in real time when an LLM is mid-answer to a user question. Traffic spikes correlate with question volume rather than crawl schedules, so it shows up unpredictably.

DoubaoBot is operated by Bytedance. Crawls broadly across the public web to gather content for training new model versions. Site owners can opt out via robots.txt, though enforcement varies by operator.

imageSpider is operated by Bytedance. Crawls broadly across the public web to gather content for training new model versions. Site owners can opt out via robots.txt, though enforcement varies by operator.

TikTok-Shop is operated by Bytedance. Catalog crawler that indexes product feeds for AI-powered shopping experiences and answer engines. Allowing it is how products become discoverable inside agentic commerce.

TikTokSpider is operated by Bytedance. Single-page fetch triggered by a user action, link previews, social unfurlers, RSS readers. Volume tracks shares and clicks rather than crawl schedules.

Toutiao is operated by Bytedance. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.

Trae is operated by Bytedance. AI-powered developer tool that fetches docs, code samples, and package metadata to ground code suggestions. Usually identifies itself with a Coding Agent or model-specific user-agent.