Our link analyzer, available starting from the Growth plan, is a powerful tool to find broken links on your blog. Its design is quite simple. Our servers send out HTTP HEAD requests to fetch headers and we then validate the HTTP status to see if a link is broken or not.
Previously, we categorized links as follows::
200-299: OK300-399: Redirect>400: Broken
Now, we categorize them as follows:
200-299: OK300-399: Redirect404or connection issues: Broken- other codes: Risky (new)
Why?
As mentioned above, we use HTTP HEAD requests to see if the links are working. These requests are sent from our servers. However, some websites have firewalls blocking servers (bots) like ours. While there are dark patterns to go around them (for example, using browser-like headers, using browser automation, etc.), we decided not to do that and instead be a “good bot”.
This means that we cannot, in some cases, reliably check if a link is working or not. Previously all these links were shown as ‘broken’ when, in fact, they might have been ok (which was generally the case).
Ignoring Known Firewalls
In addition to introducing a new ‘risky’ bucket for links, our analyzer now automatically ignores any link that has a known firewall such as Cloudflare Challenge. It will be up to you to manually check these URLs.
