Fichier Robots.txt File

To create your robots.txt as a template, first set the enableRobotsTXT value to true in your configuration file. By default, this option generates a robots.txt with the following content, which tells search engines that they are allowed to crawl everything:

User-agent: *

Ordre de Recherche du Modèle Robots.txt

The lookup order for the robots.txt template is as follows:

• /layouts/robots.txt
• /themes/<THEME>/layout/robots.txt

{{% notice note %}} If you do not want Hugo to create a default robots.txt or leverage the robots.txt template, you can hand code your own and place the file in static. Remember that everything in the static directory is copied over as-is when Hugo builds your site. {{% /notice %}}

Exemple de Modèle Robots.txt

The following is an example robots.txt layout:

{{% code file=“layouts/robots.txt” download=“robots.txt” %}}

User-agent: *

{{range .Data.Pages}}
Disallow: {{.RelPermalink}}
{{end}}

{{% /code %}}

This template disallows all the pages of the site by creating one Disallow entry for each page.