Legal

About Kitto-SiteCrawler

Last updated September 7, 2026 · Questions: [email protected]

If you found Kitto-SiteCrawler in your server logs, this page explains what it does and how to stop it.

What it does

Kitto-SiteCrawler reads the public pages of websites whose owners connected them to Kitto. It fetches HTML only (titles, headings, text and internal links) to understand what a site already covers. It never logs in, never submits forms and never fetches pages of sites that were not connected by their owner.

How to identify it

User-Agent: Kitto-SiteCrawler/1.0 (+https://kittoseo.com/crawler/). It respects robots.txt, fetches at most a few hundred pages per site, spaces requests out and never follows links to other domains.

How to block it

Add to your robots.txt:

User-agent: Kitto-SiteCrawler
Disallow: /

Or write to [email protected] with your domain and we will exclude it.