Search Results
All Results

Robots.txt Generator

Direct search engine bots on where they can and cannot go on your site.

1. Global Settings
2. Restricted Paths
3. File Preview

                
                
            

        


        

            
Robots.txt FAQ

            

                

                    
What is a Robots.txt file?

                    
It is a simple text file placed in your website's root directory that tells search engine crawlers which pages or sections of your site they should not visit.

                

                

                    
Does it hide my pages from users?

                    
No. Robots.txt is only a request for web robots. Regular users can still access these pages if they have the link.

                

                

                    
Where should I upload it?

                    
The file must be uploaded to the root of your domain. For example: yoursite.com/robots.txt.