Free audit
Menu
Our bot

RateMySiteBot

RateMySiteBot is the reader behind ratemysite.ai audits. When an audit runs, it reads a site's home page and up to 25 other pages. This page shows what it reads and how to block it.

By . Updated .

What does one audit read?

One audit makes a small, fixed set of requests to your site:

  • The home page.
  • Your public files: robots.txt, sitemap.xml, llms.txt, llms-full.txt and ai.txt (at /.well-known/ai.txt).
  • A check that the http address redirects to https.
  • The Markdown version of the home page, if you offer one.
  • The site icon.
  • Then up to 25 more pages, 4 at a time, at most 8 seconds each and 45 seconds in all. To choose them it reads robots.txt, the sitemap and the home page links again.

What is not RateMySiteBot?

Page speed is measured by Google PageSpeed Insights, so Google's Lighthouse user agent fetches your page, not ours. DNS records are read through Cloudflare DNS, not from your server.

How often does it visit?

  • When a person asks for an audit. A finished audit is reused for 24 hours, so a site gets at most one new audit a day.
  • Once a week for sites with score emails turned on.
  • About once a month for the sites on the Top websites list.

How do I recognize it in my logs?

Requests carry this user agent:

Mozilla/5.0 (compatible; RateMySiteBot/1.0; +https://ratemysite.ai/bot)

They come from Cloudflare's network, and there is no fixed list of IP addresses to allow or block. The one exception is the site icon request, which does not carry the RateMySiteBot name.

How do I block it?

Add these lines to your robots.txt. They stop the scan of your other pages:

User-agent: RateMySiteBot
Disallow: /

Without a RateMySiteBot group, the bot follows your * group instead. The home page audit is different: a person requests it, like opening your site in a browser, so it does not follow robots.txt.

To have a site removed, write to hello@ratemysite.ai or use the contact page. We delete its existing reports and keep it out of rankings and scheduled audits.

Where can I read more?

For background, read the Google robots.txt introduction and the about page.

What else do people ask?

Is RateMySiteBot safe to allow?

It only reads public pages with ordinary GET requests, it never logs in, submits forms or changes anything, and one audit reads at most 25 pages beyond the home page and public files.

Does blocking RateMySiteBot hide my site from ratemysite.ai?

No. A robots.txt rule stops the scan of your other pages only. A person can still ask for an audit of your home page, and to remove your site from reports, rankings and scheduled audits you write to hello@ratemysite.ai.

Why did the bot visit my site?

Someone asked for an audit of your site, you turned on score emails for it, or it is on the Top websites list. Each audit reads the pages described above, once.

Can I get a list of its IP addresses?

No. Requests come from the Cloudflare network and no fixed list of addresses exists, so match the user agent string shown on this page instead.

See all 49 checks on your site

Run a free audit