Our crawler

MetricsTab-Bot

If you saw MetricsTab-Bot/1.0 in your logs, the site's owner has connected it to Metricstab, an SEO analytics tool. Here is everything the bot fetches, how often, and how to stop it.

Which sites it visits

Only sites whose owner added them to Metricstab through Google Search Console. It does not discover or crawl other sites.

What it fetches

  • robots.txt — about every 3 hours, so the owner is alerted if a change starts blocking search engines.
  • Pages in an SEO test the owner set up — up to 200 pages per test once a day, and the test's comparison pages at most once a week. It reads only the title, meta description, first heading and canonical link, to tell whether a page changed during the test.
  • Up to 20 of the site's most-visited pages — once a week, for an AI-readiness check.
  • Up to 50 of the site's top pages — only when the owner asks for a title check, at most once a day. It reads only the page title and the same few tags.
  • The site's sitemap files — once a week (or when the owner asks, at most once a day): the sitemaps Search Console knows, or /sitemap.xml, and up to 50 sitemaps listed in a sitemap index.

It fetches one page at a time, about one per second, and reads at most 512 KB of each page.

How to block it

The bot obeys robots.txt for every page it fetches. To block it entirely, add:

User-agent: MetricsTab-Bot
Disallow: /

It will still read robots.txt itself — that is how it learns the rule — but no other page.

Page-speed tests are not this bot

Metricstab's page-speed reports use Google PageSpeed Insights. Those visits come from Google's own servers with Google's Lighthouse user agent, not from MetricsTab-Bot, so the rule above does not cover them.

Questions

Email hello@metricstab.com and include the URL and time from your logs.