Technical SEO
Crawlability
What is Crawlability?
Crawlability is how easily search engine bots can discover and fetch the pages of a website. A crawlable page is linked from somewhere the bot can find, is not blocked by robots.txt, returns a successful status code, and does not require logging in, submitting forms or running complex JavaScript to reveal its links.
Crawlability is the first step of a chain: a page must be crawled before it can be indexed, and indexed before it can rank. Indexability, whether a crawled page is allowed and chosen to be added to the index, is the next step and is controlled by signals such as noindex and canonical tags.
Why it matters for SEO
If search engines cannot crawl a page, nothing else about its SEO matters. Crawl problems such as accidental robots.txt blocks, broken internal links, redirect chains or links only generated by JavaScript quietly stop good pages from ever appearing in search.
Example
A developer adds "Disallow: /blog" to robots.txt on a staging server, and the file is copied to production during a release. New blog posts stop appearing in Google. A robots.txt checker shows Googlebot is blocked from every /blog URL; removing the line and requesting recrawls fixes it.
Free tools for Crawlability
Put the theory to work
LazySEO researches keywords, writes SEO articles and publishes them to your site on a schedule.