Operator
Archive.Org
Archive.Org runs 2 bots across 1 agent type. Each one is a separate user-agent so you can allow or block them independently.
Bot
HeritrixSearch Engine · Search index
Search Engine
Search index
Heritrix is operated by Archive.Org. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.
ia_archiverSearch Engine · Search index
Search Engine
Search index
ia_archiver is operated by Archive.Org. Traditional search-engine crawler that indexes pages for organic-search results. Blocking this almost always reduces visibility in the corresponding search engine.