Your robots.txt file tells search engines which parts of your site to crawl and which to leave alone. Limecube serves it at
yoursite.com/robots.txt and gives you a simple editor for it.Finding it#
Go to SEO Manager › robots.txt.
Editing it#
It's a plain text box, one directive per line. Type your rules and click Apply. Changes take effect immediately, and search engines pick up the new file on their next crawl. Use View live file to see what's published. Leave the box empty to allow everything: Limecube falls back to a permissive default.
Ready-made examples#
| What you want | What to write |
|---|---|
| Allow everything (the default) | User-agent: *Disallow: |
| Block a single folder | User-agent: *Disallow: /private/ |
| Block a specific bot | User-agent: BadBotDisallow: / |
| Point crawlers at your sitemap | Sitemap: https://yoursite.com/sitemap.xml |
A wrong robots.txt can hide your whole site:
Disallow: / under User-agent: * blocks everything. Double-check before you Apply, and test with the robots.txt report in Google Search Console.robots.txt controls crawling, not indexing. To keep a page out of search results, use noindex on the Page indexation tab instead. Blocking a page in robots.txt can actually leave it in Google's index without a description.