Ernst K. Simader - Bot-Checker - Crawler-Übersicht

Ernst K. Simader

Bot-Checker - Crawler-Übersicht

Zur Zeit werden 91 User-Agent-Zeichenfolgen ausgewertet:
(wird von Zeit zu Zeit aktualisiert...)

Crawler-Übersicht:

Name Regulärer Ausdruck Bemerkung
afd \bafd\b|hamburg.*berall AfD
favicon google favicon FaviconBot
google googlebot GoogleBot
feedfetcher feedfetcher Google Feedfetcher
SiteExplorer siteexplorer SiteExplorer
wotbox www.wotbox.com/bot WotBox
publiclibrary publiclibraryarchive.org publiclibraryarchive
siteexplorer siteexplorer.info SiteExplorer
privacyaware privacyawarebot PrivacyAwareBot
safedns www.safedns.com SafeDNSBot
flippy flippybearbot* FlippyBearBot
semantic semanticbot SemanticBot
bing bingbot|bingpreview/1.0b BingBot
msn msnbot MSN Search
mfi mfibot MfiSoft
GoogleVerification google-site-verification Google-Site-Verification
uptime uptimebot UptimeBot
cc ccbot CommonCrawl
kumo stratagems kumo Stratagems Kumo
finbot finbot Financialbot
seoscanners spider@seoscanners.net SeoScanners
domaincrawler domaincrawler DomainCrawler
yahoo yahoo! slurp Yahoo Slurp
jobboerse jobboersebot JobboerseBot
linguee www.linguee.com/bot Linguee Bot
mega megaindex.ru MegaIndex
duck duckduckgo-favicons-bot DuckDuckGo
yacy yacybot YaCy
grouphigh grouphigh GroupHigh
smtbot smtbot SimilarTect SMTBot
sogou sogou web spider Sogou Web Spider
netestate netestate ne crawler NE Crawler
yandex yandexbot Yandex
nmap nmap scripting engine Nmap Scripting Engine
bubing bubing BUbiNG
dataprovider dataprovider\.com Dataprovider
backlink backlinkcrawler BacklinkCrawler
Bar barkrowler Barkrowler
007ac9 007ac9.*crawler SISTRIX 007ac9
blitz b-l-i-t-z-b-o-t BlitzBot
adscanner adscanner AdScanner
extlinks extlinksbot ExtLinksBot
kiev aihitbot Kiew AihitBot
NYU nyu internet census NYU Internet Census
maui mauibot MauiBot
linkedin linkedinbot LinkedIn
utel ahrefsbot Utel AhrefsBot
site sitebot Sitebot
speedy speedy.*spider Entireweb Speedy Spider
ezooms ezooms.bot|dotbot Ezooms dotnetdotcom.org
80legs www.80legs.com 80legs
shop ibusiness shopcrawler ShopCrawler
majestic mj12bot Majestic-12
memory memorybot MemoryBot
sistrix crawler.*sistrix|sistrix.*crawler Sistrix Crawler
plukkie www.botje.com/plukkie.htm Plukkie Robot
wbsearch wbsearchbot WBSearchBot
baidu baiduspider Baidu Spider
ps psbot PicSearch
pixray pixraybot Pixray-Seeker
icjobs icjobs iCjobs Stellenangebote
askpeter www.askpeter.info/bot.html AskPeterBot
digext digext.*dts agent DigExt MailCollector
semager semager-bots Semager Bot
checklink checklink Checklink
catch catchbot CatchBot Australia
cliqz cliqzbot Cliqzbot
semrush semrushbot Semrush
xovi xovibot XoviBot
bo \bobot\b obot
wonder wonderbot\/* WonderBot
ru ru_bot RuBot
archiveorg archive.org_bot Archive.org
nutch nutch Nutch
wevika wevikabot WeViKaBot
savetheworld savetheworldheritage SaveTheWorld
datenbutler \bmindupbot\b datenbutler.de
netcraft netcraftsurveyagent NetcraftSurveyAgent
semalt semalt.semalt.com/crawler.php semalt.com
openlinkprofiler openlinkprofiler.org/bot OpenLinkProfiler
meanpath meanpathbot meanPath
exa exabot ExaLead
oberforstmeister oberforstmeister Oberforstmeister
roger rogerbot Rogerbot
linkdex linkdexbot LinkDex
lightspeed lssrocketcrawler|lightspeedsystemscrawler Lightspeed
indy compatible.*indy library Indy Library
turnitin turnitinbot TurnitinBot
bigfinder bigfinder.de Bigfinder
netzcheck netzcheckbot NetzCheckBot
alexa ia_archiver Alexa Crawler