@chrisbdm We’ve identified several concerns in your message and responded to each below:
Is there a way for Yoast to recreate a robots.txt file?
There isn’t a feature in the plugin to delete the robots.txt file, it only allows you to create a robots.txt file and edit an existing robots.txt file. If for some reason you want to completely delete the robots.txt file, it would need to be done via a file manager or FTP application.
It doesn’t look like the typical Yoast example so how can I change this? Is there a way to re-run the create robots.txt file or do I just manually edit the text to something like the one below and save it?
You can edit the robots.txt file by performing the following steps:
- In WordPress, go to Yoast SEO>Tools
- Click on ‘File Editor’
- Under ‘robots.txt’ beneath ‘Edit the content of your robots.txt:’ is a field from which you can directly edit your robots.txt file
- After making any changes to the file, click ‘Save changes to robots.txt’
If you do not see the ‘File Editor’ option in Yoast SEO>Tools, this may mean that file editing is disabled on your WordPress installation. If file editing is disabled, please enable file editing or edit the file through FTP.
We also have more information on this and editing the robots.txt file here: How to edit robots.txt with Yoast SEO.
Also, would changing this effect anything else such as with Google Search Console which has already indexed pages etc?
The robots.txt file contains directives intended to instruct search engine bots, including Googlebot, how you would like your site to be crawled by them. That is to say that any changes you make to this file are to affect Googlebot. The specific directives you are wanting to include don’t affect which areas or pages to crawl though, but rather the crawl-delay directive instructs search engine bots to change how frequently they’ll request pages on your site.
You can read more information on the robots.txt file and the use of the crawl-delay directive in the guide here: The ultimate guide to robots.txt.
