Avoid site indexing on Google ! How can I stop my search engines from scrolling through a WordPress site?
Recently, one of our users asked us how to get your WordPress site hidden from the search engines, or in other words stop the site’s index? There are many scenarios for stopping the listing of the website in the search results. In this article, we review these scenarios.
Why and who wants to stop search engines?
For most websites, search engines are the largest source of traffic. You might ask why someone wants to block search engines?
At the start, many people do not know how to create a local development environment or an official site.
In all of the above conditions, you probably do not want to list your search engines.
A common misconception is that if I do not link my domain, so it’s probably search engines will never find my website. This is completely wrong.
There are many ways that search engines can find that the website is linking to another one. For example:
- Your domain name has already been for someone else and still links to some of your links to your website.
- Some search results for a domain site can show your link on them.
- There are literally thousands of pages with a list of domain names, your site will appear in one of them.
There are so many things happening on the web, and many of them are not under your control.
Block search engines from the WordPress site design page
WordPress has an inbuilt feature that lets you remove search engines from your site. To access this feature, select the Reading option in the Settings section and mark the Serch engine visibility checkbox.
By selecting this box, WordPress adds the following line to the header of the site:
<meta name = ‘robots’ content = ‘noindex, follow’ />
WordPress also edits your site’s robots.txt file and adds the following lines:
These lines of robots (web crawlers) do not want to list your pages. However, accepting this request or ignoring it entirely depends on search engines. While search engines pay attention to this, some random pages or images from your site may be listed.
How to make sure your site is not displayed in search results?
Default WordPress settings guide search engines to not list your website. However, search engines may list your website, page, file, or image.
The only way to ensure that search engines do not list is to protect your entire WordPress site from the server level through encryption.
This means that when search engines or someone tries to access your website, they will be asked to provide username and password before they reach WordPress. So when they enter the system, the error page 401 is displayed and the robots go away.
WordPress site encryption ways
- Encrypt the site using cPanel
If your web hosting provider offers Cpanel access to manage your account, you can protect your entire site using Cpanel. All you have to do is login to your Cpanel dashboard and then click on the “Password Protect Directories” icon in the “Security” section.
Then select the folder where you installed WordPress, which is usually public_html.
If you have installed several WordPress sites in the public_html directory, you should click on the folder icon to select a folder to create a password for your website.
On the next page, you must enter a name for the protected folder and click the Save button.
Your information will be saved and the cpanel will reload the page. After that, by entering a username and password, you must add an authorized user.
You have successfully added password protection to your WordPress site.
At this time, whenever a user or website searches for your search engine, you will be prompted to enter the username and password you previously created to view the site.
- Encrypt the site using the plugin
If you use another solution to manage web hosting, sometimes you do not have access to cPanel.
In this case, you can use WordPress plugins to maintain your website’s password.
The most common solutions are:
The WordPress Maintenance Plugin is used on over 800,000 websites. This plugin has full license and access features that you can use to hide your website from the perspective of everybody, including search engines.
- Password protected
A very simple way to protect your WordPress website is to create a password (no need to create a user).