Screaming Frog SEO Spider

Screaming Frog SEO Spider is the technical-SEO workhorse — the tool professionals keep installed even when their suite's built-in audit exists. It crawls your site the way a search engine does, following links and rendering pages, then reports everything wrong: broken links, redirect chains, missing titles and meta descriptions, duplicate content, orphaned pages, hreflang errors and hundreds of other issues.

It's a desktop application, not a web dashboard — you point it at your site, it crawls, and you get dense, filterable tables of every issue it finds. That design is why it's powerful and why it has a learning curve: the data is exhaustive, but reading it well requires knowing what matters.

The pricing model is refreshingly old-school. The free version crawls up to 500 URLs and includes most core features. The paid licence — $279/year per user in the US, £199 in the UK, €245 in Europe — removes the URL cap and unlocks saved crawls, scheduling, JavaScript rendering depth, custom extraction and the Google integrations. Bulk discounts kick in at 5+ licences.

How Screaming Frog works

1. Enter your site's URL and configure the crawl (respect robots.txt, render JavaScript, set limits).
2. The Spider fetches every reachable page and extracts on-page and technical data.
3. Review issue reports — broken links, redirects, titles, canonicals, structured data, hreflang, duplicates.
4. Export data, connect Google Analytics/Search Console/PageSpeed data, or push findings into your workflow.
5. Schedule crawls for ongoing monitoring on the paid licence.

Key capabilities

Site crawling — follows links, respects robots.txt, handles JavaScript-rendered pages.

Issue detection — broken links, redirect chains, missing/duplicate titles and metas, canonicals, hreflang, orphan pages.

Crawl analysis — visualisations of site structure, crawl depth and internal linking.

Custom extraction — pull any element or attribute with XPath/CSS selectors.

JavaScript rendering — audit SPAs and JS-heavy sites accurately.

Google integrations — overlay Analytics, Search Console and PageSpeed Insights data on crawled URLs.

AI integrations — connect OpenAI, Gemini, Anthropic or local Ollama models for assisted analysis.

MCP server — expose crawl data to AI assistants and agents.

Scheduling and exports — recurring crawls, bulk exports, Data Studio reporting.

Where it earns its keep

The reports that matter most in practice: the redirect and canonical audits that catch migration mistakes before they cost rankings; the broken-link and orphan-page reports that quietly bleed crawl budget; the title/meta duplication checks that reveal templating gone wrong; and the structured-data validation that catches schema errors Google would otherwise ignore. None of these are glamorous — they're the unglamorous correctness work that separates sites that rank from sites that almost rank.

Who it's really for

Screaming Frog fits anyone whose job touches technical SEO: the consultant auditing a client's site before a retainer, the in-house marketer preparing a migration, the agency running monthly crawl checks across a portfolio. It's also the tool you reach for when a suite's built-in audit says "all good" but you suspect otherwise — its crawl depth and configurability find what templated dashboards smooth over.

It fits less well for non-technical users who want answers rather than data, and for teams that need the whole SEO workflow in one place — this is one job done to the highest standard.

Screaming Frog plans and pricing explained

Free — $0. Crawls up to 500 URLs per session, includes most audit features. The catch: you can't save or reopen crawls, and configuration, JS rendering, custom extraction and Google integrations are limited or locked. Enough for small sites and evaluation.

Paid licence — $279/year per user (USD), £199 (GBP), €245 (EUR). Unlimited crawling (bounded by your machine's memory and storage), saved crawls, full JS rendering, custom extraction, scheduling, Google integrations, AI integrations, MCP server and technical support. One licence per person; install on your own machines.

Bulk discounts — 5–9 licences $265/user/year, 10–19 $249, 20+ $235. Aimed at agencies.

Refunds — 14 days from purchase, full refund; no partial refunds after that. Licences last one year; auto-renew only if you choose it.

What you should know before buying

Screaming Frog is a specialist tool, not a suite — it does no keyword research, no rank tracking, no backlink analysis. Pair it with Ahrefs, Semrush or SE Ranking for those. Within its lane, though, it's the most thorough technical auditor available and the one agencies trust for client work.

The free tier's 500-URL cap is the real gate: most business sites exceed it quickly, so the licence is effectively required for ongoing use. And because it's a desktop app with dense tables, there's a genuine learning curve — budget a weekend to get fluent.

Buy it if technical SEO is part of your job — site migrations, audits, ongoing monitoring. Skip it if you just need occasional checks on a small site; free crawlers or your suite's built-in audit may cover you.