A robots.txt file is a simple text file that tells search engine crawlers which parts of your web application they are allowed to access and index.
It’s like a set of instructions that says, “Hey, search engines! Here’s what you can look at and use, and here’s what you should leave alone.”
Elegant is configured right out of the box to generate a robots.txt for your web application automatically.
Below is a snippet of our robots.txt for this website.
# * User-agent: * Allow: / # Host Host: https://www.elegantframework.com # Sitemaps Sitemap: https://www.elegantframework.com/sitemap.xml
During local development, a robots.txt file will not generated by your application.
npm run dev will not run certain build steps as sitemap and RSS feed generation, and robots.txt creation.
To override this behavior locally, simply run
npm run build and navigate to your
Once you have deployed your Elegant application to your domain, you can view your robots.txt file by navigating to
/robots.txt at the root of your domain.
For example: elegantframework.com/robots.txt.