Key Takeaways
- A 401 error means the user is unauthorised and needs valid credentials.
- A 403 error means access is forbidden despite valid credentials.
- These errors are tied to user authentication and permissions.
- Fixing them often involves adjusting login settings or server permissions.
- A 403 error is typically due to server settings or insufficient user permissions.
- The 401 error may be resolved by providing the correct login details or token.
- A 403 error cannot be solved by simply re-authenticating; the issue lies with server permissions.
- Developers can fix a 401 error by ensuring proper access credentials are provided.
- To resolve a 403 error, it may be necessary to adjust the server configuration or permissions.
- Both errors can be avoided by ensuring correct authentication protocols and server configurations.
You may have encountered 401 and 403 error codes while trying to load a webpage. Both of these codes signal that your access is denied due to a lack of proper credentials.
These error codes may seem similar, as they fall under the same category of the 4xx status code. However, distinguishing them is essential so that if your site stumbled upon any of these codes by mistake, you can address them accordingly.
In this article, we’ll break down the differences between 401 vs 403 error codes, explore their impact on SEO, and guide you on how to fix them to improve your search performance. Let’s get into it!
HTTP Status Codes: An Introduction
Both 401 and 403 codes are part of the HTTP status codes. HTTP status codes are three-digit responses from a server to the browser, indicating whether a request was successful or not. These status codes are grouped into five categories, including:
-
- 1xx: Informational response
-
- 2xx: Successful
-
- 3xx: Redirection
-
- 4xx: Client error
-
- 5xx: Server error
In this case, both 401 and 403 codes fall under the 4xx client error category, signalling a denied access. Essentially, the client is blocked from loading the page because the necessary credentials or permissions are not in place.
401 vs 403 Error Codes: What are the Differences?
While 401 and 403 error codes fall under the same category, there are differences between them. Understanding the differences can help you better understand their causes and implement the correct solution when necessary.
We’ll further break them down below:
What is the 401 Unauthorised Code?
A 401 unauthorised code signals that access is denied due to a lack of valid credentials. Users may not have authentic login details such as username and password that are needed to access private resources.
The server will require valid authentication before the client (i.e., browser) can display protected resources. Provided that the client has valid credentials, the server will receive the request and verify the credentials included in the request.
In the case where a user has valid credentials, the server will process the request and return a 200 OK status code, and display the content.
If the opposite happens, or when the authentication is failed, the server will return with a 401 unauthorised code.
What Causes the 401 Unauthorised Code?
-
- Missing authentication credentials: This error code may be returned when the user fails to provide authentication credentials. As an example, a username and password may be needed, yet the user has not logged in to their account.
-
- Expired authentication tokens: Many websites use time-restricted tokens to authenticate users. When these tokens expire, the user can’t access the page, prompting them to re-authenticate.
-
- Incorrectly configured server settings: In some cases, server misconfiguration might happen. This will cause the server to fail at recognising valid authentication attempts, further returning a 401 error code.
-
- Access denied to specific IP addresses: Some servers are configured to deny access from specific IP addresses for security purposes. Users trying to access the site from blocked IP ranges or locations will automatically receive a 401 error code.
-
- Authentication method mismatch: Most servers are configured to accept specific authentication methods: basic, digest, or token-based. If the browser submits credentials using an unsupported method, the server will refuse the request.
What is the 403 Forbidden Code?
A 403 forbidden code is shown when the server refuses to grant the client access to protected resources, even when valid authentication credentials are provided.
Many servers restrict access to specific roles. This means a client’s credentials must not only be correct but also align with the server’s permission settings.
For example, if a user who is not an administrator tries to access an admin-only page, they will receive a 403 forbidden error, even if they are logged in with a valid username and password.
What Causes the 403 Forbidden Code?
-
- Insufficient permissions: Despite valid authentication credentials, the server may refuse the client’s request because their permissions do not align with the required access levels (e.g., resources are only accessible to administrators).
-
- IP address restriction: Servers may restrict certain IP addresses due to suspicious behaviour, such as repeated failed login attempts or unusual traffic patterns indicating brute-force hacking attempts.
-
- Directory listing denied: The server may be configured to disable directory listing by default when there is no index file, which prevents public access to sensitive files and directories.
-
- Misconfigured server settings: A misconfiguration, such as a corrupted .htaccess file, can inadvertently block access to the site, preventing even administrators from accessing certain pages.
-
- Security settings: Servers often have firewall settings that can block access based on specific rules or predetermined criteria.
-
- Blocked user agents: Servers may block certain user agents, such as browsers if they suspect they are being used by bots or for other suspicious activities.
-
- Geographic restrictions: Access to resources may be restricted for users in certain regions due to IP blocking, firewall configurations, or location-specific licensing agreements.
Is There Any Similarities Between 401 vs 403 Error Codes
We have discussed the differences between 401 and 403 error codes. However, there are also similarities between them, as they essentially fall under the same HTTP response code classification. What are they?
Access Denial
The primary similarity between 401 and 403 error codes is they both signify denied access to web resources. In this case, the server understands the request but refuses to grant the client access to the content.
As a result, the client encounters an error code, which indicates whether their authentication credentials are missing or invalid, or whether they lack the necessary permissions to access the resource despite being authenticated.
Response Expectations
Access denial due to missing or invalid credentials (401) or lack of permissions (403) both require a response for the server to proceed with allowing access to resources.
For 401 error codes, users are expected to provide valid authentication credentials, such as entering the correct username and password or using valid tokens. On the other hand, in the case of 403 codes, users should contact the system administrator or obtain the necessary permission.
Poor User Experience
Access denial to web resources due to 401 or 403 error codes can leave visitors feeling frustrated, as they are expecting to see the content they need. This frustration often results in visitors quickly exiting the site.
Such poor user experience contributes to increased bounce rates, ultimately impacting overall site performance and user engagement.
Security Measures
Both 401 and 403 status codes are implemented to protect private resources from public access. They ensure that only authenticated users and those with valid permissions can view or interact with the resources.
This prevents sensitive content from being accessed by unauthorised clients, reducing the risk of data alteration and security breaches that could compromise the system’s integrity or lead to data theft.
Impacts on SEO
While 401 and 403 codes are useful for protecting sensitive resources from malicious use or breaches, they may be implemented incorrectly. Misconfigurations in the settings are often the cause, resulting in search engine crawlers being unable to access resources and index them in search results.
401 vs 403 Error Codes: How Do They Affect SEO?
Incorrect implementation of 401 and 403 error codes can negatively affect your SEO performance. When pages intended to be publicly accessible return these error codes, several consequences may follow, including:
Increased Bounce Rates
Users who are unable to open your web content they believe they have access to, such as blogs and product pages may promptly exit your site, looking for alternatives elsewhere.
Imagine if there are hundreds or thousands of users failing to load your content.
This will lead to higher bounce rates, which can signal to search engines that your site does not provide a good user experience. You may also experience lower engagement rates and reduced time spent on your site.
Wasted Crawl Budget
Search engine bots essentially run a crawl on your web pages indiscriminately. This will also include pages that return 401 or 403 error codes. Since pages with these responses can’t be indexed in search results, you will end up wasting your crawl budget.
In turn, the crawling efficiency is disrupted. Important pages or newly published content on your site may not get crawled as frequently or thoroughly due to the presence of pages returning these error codes.
Indexing Issues
Both 401 and 403 response codes signify access denial and search engine bots are unable to index them. This will become an issue when important pages that are meant to be publicly accessible, return these codes by mistake.
As a result, your valuable content will not appear in search results, causing you to miss out on potential traffic from users searching for relevant queries that your content should be fulfilling.
Issues with Link Equity
Internal links play a crucial role in distributing link equity and passing authority from one page to another. However, when you link to pages within your site that return 401 or 403 error codes, this will obstruct the flow.
These errors prevent link equity from being passed as intended, which can weaken the authority of the linked pages and ultimately impact the overall SEO performance of your site.
Drop in Rankings
A combination of potential SEO issues caused by 401 and 403 error codes will eventually lead to a drop in ranking. Search engines may consider a site irrelevant due to high bounce rates, limited crawling, and indexing, as well as disrupted link equity.
To avoid a decline in rankings, it’s important to address these issues promptly. You need to make sure that your web pages are free from 401 or 403 errors, which are configured by mistake, to maintain a smooth user experience and optimise your site’s performance in search results.
Monitoring 401 and 403 Error Codes on Your Site
Before you can fix 401 or 403 error codes, you must first monitor your site for their presence. Identifying these issues early will allow you to take informed actions before they hinder your SEO performance.
You can use tools like Google Search Console (GSC) or an SEO audit tool to track these issues. Below is a step-by-step guide on how to do this.
Step-by-Step Guide to Identify 401 and 403 Errors Using Google Search Console
Using GSC will help you identify blocked pages due to 401 or 403 error codes, as it provides reports on crawling and indexing errors. Pages that are listed down indicate that Google bots are facing issues when trying to crawl them, resulting in an inability to index in search results.
Here’s how to look out for pages that return 401 or 403 client errors:
-
- Go to the Indexing report
-
- Open the Pages tab
-
- Scroll down to the Why pages aren’t indexed section
Within the Why pages aren’t indexed section, you will see reasons why indexing issues are affecting your web pages. If 401 or 403 error codes are present, you can identify them there.
Step-by-Step Guide Using an SEO Audit Tool (e.g., Screaming Frog)
Besides Google Search Console, you can also use an SEO audit tool, such as Screaming Frog to identify 401 and 403 error codes within your site.
To begin, you should first run a crawl of your website. Enter your site’s URL in the “Enter URL to spider” box and click start.
After the crawl finishes, you should be able to view all existing issues and opportunities. As you intend to focus on identifying client error codes, you can navigate to the ‘Response Codes’ tab and filter for ‘Client Error 4XX’.
Additionally, you can also identify any of your pages that link to broken pages (i.e., pages that return client error codes). To do so, click on a URL in the upper window and click on the ‘Inlinks’ tab in the bottom section.
Fixing 401(Unauthorised) and 403 (Forbidden) Error Codes
After obtaining reports on your 401 (unauthorised) and 403 (forbidden) pages, you need to consider whether they need to be indexed. This is important because the approach to pages intended for indexing versus those that aren’t will be different.
Fixing 401 Unauthorised Request
401 error codes are returned when the user fails to provide the authentication credentials that the server requests. Whether these pages should be indexed by search engines depends on your specific requirements.
If you want them to be indexed, you need to resolve the restrictions to allow search engine crawlers to access the content so it can be indexed and visitors can have access. Here’s what you may need to perform:
-
- Remove authentication requirements for pages meant for public access
-
- Adjust security settings that may be blocking search engine bots from accessing your pages.
-
- Implement 301 redirections if certain pages have moved and no longer need authentication,
-
- Check server and firewall configurations that may cause access restrictions.
If you don’t want these 401 pages to be indexed but search engine bots are still attempting to crawl them, here’s what you need to do to avoid wasting your crawl budget:
-
- Set up disallow using a robot.txt file prompting search engine bots to stop crawling attempts.
-
- Apply a noindex meta tag to prevent these pages from being visible in search results.
-
- Remove links pointing to these 401 pages to keep your internal linking structure clean.
Fixing 403 (Forbidden) Error Codes
403 forbidden error codes, indicating restricted access due to a lack of permissions, may or may not be worth fixing. This boils down to whether or not you want the affected pages to be indexed on search results.
If your goal is to make these pages publicly visible and drive traffic, the 403 errors might have been returned by mistake. To resolve this, you will need to investigate several potential issues by checking:
-
- .htaccess file: A misconfigured .htacess file could lead to unintentional 403 error codes. Consider disabling your existing .htaccess file and generate a new one to fix this issue.
-
- CMS plugins: Certain plugins may cause 403 forbidden issues. Make sure that all of your plugins are up-to-date; you can also deactivate those that are no longer needed or compatible.
-
- File permissions: Your file permissions may have been set up as private. If you want certain files to be publicly accessible, you need to grant permission for search engine bots to crawl and index.
-
- Malware: Malware infections may cause your pages to return 403 error codes by mistake. Make sure to remove them if you find any.
-
- Your hosting: if the issues persist after taking corrective actions, there might be problems with your hosting.
Otherwise, if you wish to keep 403 pages private and unindexed, blocking the pages in robots.txt will do. You can use the disallow command to stop certain URLs from being crawled by search engine bots, preventing a waste of crawl budget.
Final Thoughts
401 and 403 error codes essentially exist to deny access to private resources from public access. However, when pages that are intended for public access return these codes, it can create a poor user experience and negatively impact SEO performance.
Here at Roots Digital, we work closely with our clients to solve issues like 401 and 403 error codes as part of our comprehensive SEO services and website maintenance services. By fixing these errors, websites are optimised for both users and search engines.
If you’re looking for expert support to keep your SEO and website running smoothly, we are here to help. Schedule a call today!
FAQs
When to use 401 and 403 error codes?
401 error codes are used to request authentication credentials for accessing private resources. Meanwhile, 403 error codes are often used to limit access to users with valid permissions.
Is 403 forbidden an IP ban?
An IP ban is just one reason why pages may return 403 forbidden error codes. Other factors can include invalid permissions, denied directory listings, and firewall settings, among others.
