What is HTTP 403?
HTTP 403 Forbidden is a status code a server uses to say it understood a request but refuses to fulfil it. When an AI crawler receives this code, it cannot read the page content, no matter what robots.txt says.
The code is defined in the HTTP specification RFC 9110. It differs from 401, which asks for authentication, and from 404, which reports that the page does not exist. A 403 says: the page exists, but it will not be served to this request.
For AI visibility, a 403 is the clearest sign of a server-level block. The typical combination is a browser getting status 200 and a crawler getting 403. In that case a firewall, CDN or bot protection is filtering by user agent or IP address.
In our measurement across 346 reachable websites of German small businesses, 15.9 percent blocked the GPTBot user agent. On 12.7 percent, the same request with a browser identifier got status 200 at the same time. The second figure is the more telling one, because it only counts blocks targeted at the crawler.
What it means for your website
Fetch your page with a crawler user agent, for example with curl and its user agent option, and read the status code. A 403 alongside a 200 in the browser is a targeted block. The deeploupe GPTBot check shows both codes for GPTBot and ClaudeBot without a command line.
Related terms
Sources
Terms help you understand. Whether AI crawlers can reach your site is something you measure.
Check your website for freefree · no signup · no credit card