Your robots.txt file tells search engines which parts of your site to crawl and which to leave alone. Limecube serves it at
yoursite.com/robots.txt and gives you a simple editor for it.Finding it
Go to SEO Manager › robots.txt.
Editing it
It's a plain text box, one directive per line. Type your rules and click Apply. Changes take effect immediately, and search engines pick up the new file on their next crawl. Use View live file to see what's published. Leave the box empty to allow everything: Limecube falls back to a permissive default.
Ready-made examples
| What you want | What to write |
|---|---|
| Allow everything (the default) | User-agent: *Disallow: |
| Block a single folder | User-agent: *Disallow: /private/ |
| Block a specific bot | User-agent: BadBotDisallow: / |
| Point crawlers at your sitemap | Sitemap: https://yoursite.com/sitemap.xml |
A wrong robots.txt can hide your whole site:
Disallow: / under User-agent: * blocks everything. Double-check before you Apply, and test with the robots.txt report in Google Search Console.robots.txt controls crawling, not indexing. To keep a page out of search results, use noindex on the Page indexation tab instead. Blocking a page in robots.txt can actually leave it in Google's index without a description.
Was this helpful?
Last updated:
Related Articles
XML Sitemap
Limecube builds and updates your XML sitemap automatically at /sitemap.xml. See where it is, how to submit it to Google, and how noindex removes a page from it.
3 min read
Page Indexation (Noindex & Nofollow)
Tell search engines which pages to index and which links to follow, across your whole site, from the SEO Manager's Page indexation tree. Noindex also drops a page from your sitemap.
5 min read
Google Search Console
Set up Google Search Console to monitor your site in Google: verify it (via Analytics, a meta tag in your head code, or DNS), then submit your sitemap.
4 min read
Still need help?
Submit a Support Ticket