Get started

sandywp / plugins / wp-robots-txt

WP Robots Txt

5.0 out of 5 stars. 5.0 21 reviews

WP Robots Txt Allows you to edit the content of your robots.txt file.

Launch a sandbox with this plugin → No signup.
Ready in seconds.

v1.3.6 40,000+ installs WP 5.3.0+ PHP 7.0+

By George Pattichis · Read the readme on WordPress.org

your-sandbox.sandywp.com/wp-admin
WP Robots Txt running inside a SandyWP sandbox

What you get inside

1 screen

  • A view of the admin option

screenshot 1 of 1
A view of the admin option

A view of the admin option

About this plugin

WordPress, by default, includes a simple robots.txt file that’s dynamically generated from within the WP application. This is great, but how do you easily change the content?

Enter WP Robots Txt, a plugin that adds an additional field to the “Reading” admin page where you can do just that. No manual coding or file editing required!

Simply visit https://your-site.com/wp-admin/options-reading.php and you can control the contents of your https://your-site.com/robots.txt

Changelog

Questions

I totally screwed up my `robots.txt` file. How can I restore the default version?

Delete all the content from the Robots.txt Content field and save the privacy options.

Could I accidently block all search bots with this?

Yes. Be careful! That said, robots.txt files are suggestions. They don’t really block bots as much as they suggest that bots don’t crawl portions of a site. That’s why the options on the Privacy Settings page say “Ask search engines not to index this site.”

Does discouraging search engines change robots.txt?

No. Since WordPress 5.3 that setting adds a noindex meta tag. It does not add Disallow: / to robots.txt. Your saved rules are still served. Add Disallow: / yourself if you want crawlers to skip the site.

Why is my edited robots.txt not used?

WordPress only serves the virtual robots.txt when there is no physical robots.txt file in the site root. Delete or rename that file to use this plugin.

Where can I learn more about `robots.txt` files?

Here is a general guide by Google and here is the WordPress SEO documentation.

5.0

21 reviews

  • 5 ★ 20
  • 4 ★ 1
  • 3 ★ 0
  • 2 ★ 0
  • 1 ★ 0

Ratings come from WordPress.org. SandyWP does not collect its own reviews.

sandywp / also on the shelf

Try another plugin

  • XML Sitemap & Google News

    4.4 · 100K+ installs

    Take control of your WordPress core XML Sitemap and add a Google News Sitemap.

    Try demo

  • Virtual Robots.txt

    4.3 · 40K+ installs

    Virtual Robots.txt automatically creates a robots.txt file for your site. Your robots.txt file can be easily edited from the…

    Try demo

  • Head Meta Data

    4.8 · 10K+ installs

    Adds a custom set of <meta> tags to the <head> section of all posts & pages.

    Try demo

Spin up a real WordPress site in seconds.

Test plugins, build a demo, hand a client a link — then squash it and start again. No local setup, no Docker.

Create a site → See pricing