# CrawlGraph

> Free backlink intelligence powered by Common Crawl's open hyperlink graph (121.1M domains, 3.90B links). Paste any domain and see every site linking to it. No signup for the free tier. $99 lifetime, unlimited site queries.

CrawlGraph is a backlink-checker and link-gap tool for SEOs, indie founders, and researchers priced out of Ahrefs, Semrush, and Moz. It queries the Common Crawl hyperlink graph directly with DuckDB, so it costs $99 once instead of $99 to $140 per month. The graph is reindexed within days of each quarterly Common Crawl release; data spans 2017 through the latest 2026 release.

## What it does

- Backlink lookup: every referring domain for any target domain, ranked by authority.
- Gap analysis: the domains that link to your competitors but not to you, ranked by authority. The core outreach workflow. An unpaid run needs an email and returns the first 5 gaps; a paid plan returns the full list.
- Contact finder: for lifetime users, finds the outreach email for a target domain.
- API and MCP: a REST API (Bearer auth) and an npm MCP server (npx -y crawlgraph-mcp) usable from Claude, Cursor, and Cline.

## Core pages

- Homepage: https://crawlgraph.com
- Gap analysis (free): https://crawlgraph.com/gap-analysis
- API docs: https://crawlgraph.com/docs/api
- Blog: https://crawlgraph.com/blog
- MCP server: https://github.com/pucilpet/crawlgraph-mcp
- Open datasets: https://github.com/pucilpet/crawlgraph-datasets

## Topic hubs

Every blog post belongs to exactly one topic hub: an ordered reading path with its own introduction.

- The backlink guide, by topic (https://crawlgraph.com/seo-topics): the index of every hub below.
- backlinks explained (https://crawlgraph.com/seo-topics/backlinks-explained): what backlinks are and how search engines weigh them, plus every concept behind a backlink report: referring domains, anchor text, nofollow, and authority.
- competitor backlink analysis (https://crawlgraph.com/seo-topics/competitor-backlink-analysis): find the sites linking to your competitors and not to you: what a link gap is, how to run one for free, and how to turn the list into outreach.
- where backlink data comes from (https://crawlgraph.com/seo-topics/backlink-data): why no two backlink tools agree: the crawlers behind the paid indexes, the free data in search console and bing, and the open common crawl web graph.
- seo tool comparisons (https://crawlgraph.com/free-seo-tool-alternatives): head-to-head comparisons with the big backlink tools, list prices included, and where each one genuinely beats us.
- backlink studies (https://crawlgraph.com/blog/category/case-studies): original studies on the open graph: which domains link to every major tool in a category, and what that says about where links come from.
- seo for ai search (https://crawlgraph.com/seo-topics/seo-for-ai-search): what ai answers changed about search and what they did not: whether seo still works, what makes a page citable, and how to read your own traffic.
- backlink api (https://crawlgraph.com/seo-topics/backlink-api): backlink data in code: a free api tier with 15 lookups a month, an mcp server for ai assistants, and the raw common crawl graph to process yourself.

## Alternatives and comparisons

Nine pages for "free <tool> alternative" queries. Each one states what CrawlGraph replaces (referring domains, link gap, outreach contacts, API and MCP access) and what it does not (live re-crawl, rank tracking, keyword and PPC research), then shows the vendor's list price and the field where that vendor genuinely wins.

- All SEO tool alternatives, compared (https://crawlgraph.com/free-seo-tool-alternatives): the eight tools below in one table, by list price and by strength.
- Free Ahrefs alternative (https://crawlgraph.com/free-ahrefs-alternative): Ahrefs keeps the largest continuously re-crawled link index, daily rank tracking, and deep keyword research; CrawlGraph covers referring domains and gap analysis on open Common Crawl data.
- Free Semrush alternative (https://crawlgraph.com/free-semrush-alternative): Semrush keeps the all-in-one suite (PPC, social, content) and a huge keyword database; CrawlGraph covers the backlink and gap half only.
- Free Moz alternative (https://crawlgraph.com/free-moz-alternative): Moz keeps Domain Authority, link tracking, and scheduled site crawls; CrawlGraph reports its own 0-100 authority from Common Crawl instead.
- Free Majestic alternative (https://crawlgraph.com/free-majestic-alternative): the closest match in scope, since both are backlinks-only; Majestic keeps Trust Flow, Citation Flow, and a deeper historic index.
- Free Serpstat alternative (https://crawlgraph.com/free-serpstat-alternative): Serpstat keeps rank tracking, audits, keyword data, and cheap team seats; CrawlGraph covers backlinks and gap analysis.
- Free SpyFu alternative (https://crawlgraph.com/free-spyfu-alternative): SpyFu keeps PPC and ad-history intelligence, which CrawlGraph does not do at all; the overlap is backlink data.
- Free SE Ranking alternative (https://crawlgraph.com/free-seranking-alternative): SE Ranking keeps daily rank tracking and white-label agency reporting; CrawlGraph covers referring domains, gap analysis, and outreach contacts.
- Free Ubersuggest alternative (https://crawlgraph.com/free-ubersuggest-alternative): Ubersuggest keeps keyword research, a basic site audit, and rank tracking in one cheap login, and sells a lifetime license as well as a monthly plan; CrawlGraph covers referring domains, gap analysis, and outreach contacts on open Common Crawl data.

## Data studies (original, citable)

CrawlGraph publishes backlink-overlap studies: for a category of tools, which domains link to the whole set. The universal linkers are that niche's warmest outreach targets. Method: pull each tool's referring domains from Common Crawl ranked by authority, intersect, filter platform and CDN noise.

- Who links to every major SEO tool (https://crawlgraph.com/blog/who-links-to-every-seo-tool): 8 tools (Ahrefs, Semrush, Moz, Majestic, Serpstat, SE Ranking, SpyFu, Surfer). 9,687 unique linkers, 67 percent link to only one, 37 link to all eight, 98 percent of heavy overlappers are editorial. The universal linkers are SEO trade media and best-of roundups.
- Who links to every major CRM (https://crawlgraph.com/blog/who-links-to-every-crm): 8 CRMs (HubSpot, Zoho, Pipedrive, Freshworks, Close, Copper, Attio, Salesflare). 9,910 unique linkers, 78 percent link to only one, only 2 link to all eight (Zapier and RingCentral, both integration hubs). An integration-driven niche, unlike the media-driven SEO niche.
- SEO-tool link leaderboard (https://crawlgraph.com/seo-tools-link-leaderboard): sortable, with CSV and JSON download.

Datasets are CC-BY and downloadable as CSV and JSON at https://github.com/pucilpet/crawlgraph-datasets.

## Blog posts

- query the common crawl web graph yourself - backlinks from a 16 gb edge list (https://crawlgraph.com/blog/query-common-crawl-web-graph, 2026-09-10): three files, no api key, no cluster. the streaming pipeline that pulls one domain's referring domains out of the open graph on a laptop.
- what is link building - the honest, short version (https://crawlgraph.com/blog/what-is-link-building, 2026-09-10): getting other sites to link to yours, why the count stopped mattering, and the three approaches that still work sorted by how much of your time they waste.
- toxic backlinks and the disavow file - the unpopular, accurate version (https://crawlgraph.com/blog/toxic-backlinks-and-disavow, 2026-09-10): toxic is a vendor word, not a google one. what the score actually is, the narrow case where disavow is correct, and why the default answer is to do nothing.
- nofollow links - what they are, whether they help, and why some tools cannot see them (https://crawlgraph.com/blog/nofollow-links-explained, 2026-09-10): rel=nofollow became a hint in 2019, not a directive. what that changed, when to use sponsored and ugc instead, and the layer problem behind link-attribute reporting.
- domain authority vs domain rating vs trust flow - what the scores actually measure (https://crawlgraph.com/blog/domain-authority-vs-domain-rating, 2026-09-10): four vendors, four 0-100 numbers, none of them google's. what each one is computed from, why they never agree, and the only way to read them that survives contact with reality.
- anchor text, explained - what it is, what it signals, and where it actually pays (https://crawlgraph.com/blog/anchor-text-explained, 2026-09-10): the clickable words of a link are an independent description of your page. here is why that matters, what a natural profile looks like, and why internal anchors are the half worth working on.
- how to find keyword cannibalization in search console (with a real diagnosis) (https://crawlgraph.com/blog/keyword-cannibalization-search-console, 2026-09-10): the aggregate report cannot show it. query+page pairs can. here is the query that finds it, and 44 cannibalized queries found on our own site.
- what is a link gap - the set difference behind competitor link research (https://crawlgraph.com/blog/what-is-a-link-gap, 2026-09-10): a link gap is the domains linking to your competitors but not to you. here is the arithmetic, why the count of competitors changes the answer, and why two tools never agree.
- answer engine optimization: what actually makes a page citable (https://crawlgraph.com/blog/answer-engine-optimization, 2026-09-10): aeo, geo, and llm seo mostly describe the same job. here is the part that is genuinely different from ordinary seo, and what we ship on our own site.
- is seo dead? what the link graph actually shows (https://crawlgraph.com/blog/is-seo-dead, 2026-09-10): the honest answer from 3.90 billion links across 121.1 million domains: rankings still work, clicks are worth less, and the signal AI systems read has not changed.
- 3 domains link to every major landing page builder (https://crawlgraph.com/blog/who-links-to-every-landing-page-builder, 2026-09-07): we compared 8 landing page builders using the top 2,000 referring domains per subject from common crawl. 3 non-platform domains link to the full set.
- crawlgraph vs serpstat - how many modules do you actually open? (https://crawlgraph.com/blog/crawlgraph-vs-serpstat, 2026-09-05): serpstat at $69/mo is a fair buy if you use most of the suite. if you only ever open the backlink module, $99 once covers that one job forever.
- crawlgraph vs spyfu - different questions, different bills (https://crawlgraph.com/blog/crawlgraph-vs-spyfu, 2026-09-05): spyfu is a ppc intelligence tool at $39/mo. crawlgraph is backlink data for $99 once. two different jobs, and the cost lines cross in month three.
- 13 domains link to every major social media scheduler (https://crawlgraph.com/blog/who-links-to-every-social-media-scheduler, 2026-09-05): we compared 8 social media schedulers using the top 2,000 referring domains per subject from common crawl. 13 non-platform domains link to the full set.
- what is a backlink graph - nodes, edges, and where authority scores come from (https://crawlgraph.com/blog/what-is-a-backlink-graph, 2026-09-04): a backlink graph is the web drawn as nodes (sites or pages) and edges (links). every authority score is different math over a graph of that shape.
- how often do backlink tools crawl? why your new link is not showing up yet (https://crawlgraph.com/blog/how-often-backlink-tools-crawl, 2026-09-04): no tool sees the web live. a link must be crawled, indexed, then published before it appears. how to read your tool's date and confirm a link yourself.
- moz's crawler explained: dotbot, rogerbot, and the crawled flag (https://crawlgraph.com/blog/moz-crawler-explained, 2026-09-04): moz runs two crawlers: dotbot builds the link index behind DA, rogerbot runs site crawl in a moz pro campaign. what crawled means and why DA moves.
- the cheapest ahrefs alternative in 2026 (https://crawlgraph.com/blog/cheapest-ahrefs-alternative, 2026-09-04): two answers: of the seven tools here the cheapest subscription is spyfu at $39/mo, and the cheapest to own is $99 once, which beats ahrefs in one month.
- 12 domains link to every major help desk (https://crawlgraph.com/blog/who-links-to-every-help-desk, 2026-08-17): we compared 8 help desks using the top 2,000 referring domains per subject from common crawl. 12 non-platform domains link to the full set.
- 35 domains link to every major web host (https://crawlgraph.com/blog/who-links-to-every-web-host, 2026-08-10): we compared 8 web hosts using the top 2,000 referring domains per subject from common crawl. 35 non-platform domains link to the full set.
- 13 domains link to every major no-code tool (https://crawlgraph.com/blog/who-links-to-every-no-code-tool, 2026-08-07): we compared 8 no-code tools using the top 2,000 referring domains per subject from common crawl. 13 non-platform domains link to the full set.
- Common Crawl index API 2026: the current CC-MAIN index IDs (https://crawlgraph.com/blog/common-crawl-apr-jun-2026-release, 2026-07-24): The current index is cc-main-2026-apr-may-jun, built from CC-MAIN-2026-17, CC-MAIN-2026-21 and CC-MAIN-2026-25. Copy-paste API URLs for both the Common Crawl CDX index and crawlgraph.
- programmatic backlink research: 5 api recipes (https://crawlgraph.com/blog/backlink-api-recipes, 2026-07-12): get a free API key, pull one domain's referring sites, filter by authority, compare competitors, feed an automation, and connect hosted MCP, with copyable code for every step.
- 8 domains link to every major fintech app (and they are all press or capital) (https://crawlgraph.com/blog/who-links-to-every-fintech-app, 2026-07-06): we pulled the backlinks of 8 big fintech apps from common crawl. the 8 domains that link to all of them are finance trade press, newsletters, and the VC/startup databases. no integrations, no dev tools. the niche runs on trust and money.
- crawlgraph now has a free API tier (15 backlink calls a month, no card) (https://crawlgraph.com/blog/free-backlink-api, 2026-07-03): request a key on the docs page and it lands in your inbox. 15 backlink lookups a month on Common Crawl's open web graph, forever free. connect over hosted MCP or run the local npm server.
- only 5 domains link to every major AI agent framework (and the shelf is still being built) (https://crawlgraph.com/blog/who-links-to-every-ai-agent-framework, 2026-07-02): we pulled the backlinks of 8 AI agent frameworks from common crawl. only 5 domains link to all of them, and they are not tech media. the niche's link graph is dev platforms plus a brand-new crop of agent directories you can still get into.
- only 2 domains link to every major CRM (and that is the lesson) (https://crawlgraph.com/blog/who-links-to-every-crm, 2026-07-01): we pulled the backlinks of the 8 biggest CRMs from common crawl. only 2 domains link to all of them, both integration hubs. compare that to SEO tools, where 37 did. the shape of a niche's link graph tells you the strategy.
- the 37 domains that link to every major SEO tool (https://crawlgraph.com/blog/who-links-to-every-seo-tool, 2026-06-30): we pulled the backlinks of 8 major seo tools from common crawl. 37 domains link to all of them, and the list is the seo trade press plus the 'best seo tools' roundups. here is the playbook that falls out of it.
- watch: steal your competitor's backlinks (free 2-minute walkthrough) (https://crawlgraph.com/blog/watch-steal-competitor-backlinks, 2026-06-17): a short walkthrough of crawlgraph's free gap analysis: find every site linking to your competitor but not you, pull the contact email, and send the pitch - without an ahrefs or semrush subscription.
- only 6 domains link to every major email marketing tool (https://crawlgraph.com/blog/who-links-to-every-email-marketing-tool, 2026-06-05): we ran the same backlink-overlap study on the 8 biggest email tools. the overlap almost vanished - only 6 domains link to all of them. that collapse is itself the most useful finding.
- the 34 domains that link to every major ai writing tool (https://crawlgraph.com/blog/who-links-to-every-ai-writing-tool, 2026-06-05): we pulled the backlinks of the 8 biggest ai writing tools from common crawl. 34 domains link to all of them, and the list is almost entirely ai-tool directories and ai-news media. here is the playbook that falls out of it.
- the 27 domains that link to every major website builder (https://crawlgraph.com/blog/who-links-to-every-website-builder, 2026-06-04): we pulled the highest-authority backlinks of the 8 biggest website builders from common crawl. 9,189 domains link to at least one. only 27 link to all eight, and they reveal a very different playbook than you would guess.
- the 35 domains that link to every major project management tool (https://crawlgraph.com/blog/who-links-to-every-project-management-tool, 2026-06-03): we pulled the highest-authority backlinks of the 8 biggest project management tools from common crawl. 9,347 domains link to at least one of them. only 35 link to all eight, and that short list is the entire link-building playbook for the niche.
- find link building prospects with ai (https://crawlgraph.com/blog/find-link-prospects-with-ai, 2026-06-02): connect an ai assistant to a live backlink data source over mcp and let it run gap analysis, rank outreach targets, and draft angles - no spreadsheet wrangling.
- how to run a backlink gap analysis for free (https://crawlgraph.com/blog/backlink-gap-analysis-free, 2026-06-02): a gap analysis finds the domains that link to your competitors but not to you. that overlap is the highest-roi prospect list in link building. here is a free, step-by-step way to build one.
- free backlink gap analysis without ahrefs (2026) (https://crawlgraph.com/blog/free-backlink-gap-analysis-without-ahrefs, 2026-06-02): the domains that link to your competitors but not you are the best link prospects you have. here is how to find them for free, no ahrefs or semrush subscription.
- ubersuggest backlink checker vs crawlgraph - which one actually has the data (https://crawlgraph.com/blog/crawlgraph-vs-ubersuggest, 2026-06-01): ubersuggest runs backlinks as one report inside an seo suite, on its own index, capped by plan. crawlgraph does backlinks only: the full common crawl referring-domain list, for $99 once.
- crawlgraph vs majestic - two link-only tools, very different prices (https://crawlgraph.com/blog/crawlgraph-vs-majestic, 2026-06-01): majestic is the deepest pure-backlink index there is, with trust flow and a historic archive. crawlgraph is a quarterly common-crawl snapshot for $99 once. when each one is the right call.
- crawlgraph vs moz - domain authority, without the subscription (https://crawlgraph.com/blog/crawlgraph-vs-moz, 2026-06-01): moz gave the industry the DA score everyone quotes. crawlgraph gives you the same referring-domain picture from open data, once, for $99. here's the honest split.
- an honest take: when to use ahrefs vs when to use crawlgraph (https://crawlgraph.com/blog/ahrefs-vs-crawlgraph-honest, 2026-05-29): ahrefs is worth $129/month for continuous link-signal monitoring. crawlgraph is worth $99 once for outreach prospecting. here's why both are correct.
- google search console backlinks: what it shows (https://crawlgraph.com/blog/google-search-console-backlinks, 2026-05-23): gsc is the only free backlink data source straight from google - but it caps the report at ~1,000 referring domains, only shows data for sites you've verified, and surfaces a filtered subset of what google actually indexed. here's the full walkthrough plus how to get around the limits.
- where ahrefs and moz get their backlink data (https://crawlgraph.com/blog/where-ahrefs-gets-backlink-data, 2026-05-23): ahrefs, moz, semrush, and majestic each run their own proprietary crawler and ship a curated index - which is why no two backlink tools ever agree. here's the methodology, the size claims, and why common crawl is the only open alternative.
- bing webmaster tools backlinks (free, no signup) (https://crawlgraph.com/blog/bing-webmaster-backlinks, 2026-05-23): bing webmaster tools shows backlink data for free - and unlike google search console, it surfaces backlinks for sites you don't own through the similar sites report. here's the full walkthrough, the gsc comparison, and the export workflow.
- how long does google take to count a backlink? (https://crawlgraph.com/blog/how-long-google-counts-backlinks, 2026-05-02): "how long for google to count a backlink" is actually three questions: how long to crawl, how long to index, and how long for it to influence rankings. the answers are days, weeks, and months - and each phase has different bottlenecks.
- referring domain vs backlink - the difference (https://crawlgraph.com/blog/referring-domain-vs-backlink, 2026-05-02): a backlink is one link. a referring domain is one site - it can give you ten thousand links and still count as one. google weighs referring domain count more heavily than raw backlinks, and the ratio between them is itself a quality signal.
- common crawl release schedule (next crawl) (https://crawlgraph.com/blog/common-crawl-release-schedule, 2026-05-02): common crawl publishes new web crawls every 1-2 months. here's the recent release cadence, crawlgraph's current composite, and why the lag matters for backlink freshness.
- how to get backlinks for free in 2026 (https://crawlgraph.com/blog/how-to-find-backlinks-for-free-2026, 2026-05-02): free options for surveying a backlink profile - gsc, bing webmaster, crawlgraph's free tier - and what each one actually shows. for full lists, $99 once.
- crawlgraph vs semrush - focused vs all-in-one (https://crawlgraph.com/blog/crawlgraph-vs-semrush, 2026-04-28): semrush ships fifty things. crawlgraph ships one. here's why - for a real subset of seos - that's the point.
- ahrefs vs crawlgraph - what you actually pay for (https://crawlgraph.com/blog/ahrefs-vs-crawlgraph, 2026-04-22): a $99 lifetime license vs $129/mo. broken down by feature, by api call, by what the data really shows.
- common crawl, explained for SEOs (https://crawlgraph.com/blog/common-crawl-explained, 2026-04-08): the 3.90B-edge open dataset that quietly powers half the AI training runs - and how we turn it into rank intelligence.

## Pricing

- Free: 5 referring domains per query on the site, no signup. Plus a free API tier: 15 backlink API calls/month, key self-served by email at /docs/api, works with the MCP server, no card.
- Lifetime: $99 once, unlimited site queries, full results, CSV/JSON export, gap analysis (site + API + MCP), 1,000 API calls + 50 gap analyses/month.
- SEO Fix addon: $5/month optional, one AI-generated high-impact fix emailed monthly.

## Data source

Common Crawl hyperlink graph, an open dataset published quarterly at https://commoncrawl.org/web-graphs. CrawlGraph imports the vertices (about 850 MB) and edges (about 16 GB) files, queries them via DuckDB, and caches per-domain results in SQLite for 30 days.

## Contact

- Email: petteri@searchenginewizards.fi
- Site: https://crawlgraph.com