mirror of
https://github.com/gohugoio/hugo.git
synced 2025-08-19 21:21:39 +02:00
Merge commit '5be51ac3db225d5df501ed1fa1499c41d97dbf65'
This commit is contained in:
@@ -2,13 +2,9 @@
|
||||
title: robots.txt template
|
||||
linkTitle: robots.txt templates
|
||||
description: Hugo can generate a customized robots.txt in the same way as any other template.
|
||||
categories: [templates]
|
||||
categories: []
|
||||
keywords: []
|
||||
menu:
|
||||
docs:
|
||||
parent: templates
|
||||
weight: 170
|
||||
weight: 170
|
||||
weight: 180
|
||||
aliases: [/extras/robots-txt/]
|
||||
---
|
||||
|
||||
@@ -20,8 +16,6 @@ enableRobotsTXT = true
|
||||
|
||||
By default, Hugo generates robots.txt using an [embedded template].
|
||||
|
||||
[embedded template]: {{% eturl robots %}}
|
||||
|
||||
```text
|
||||
User-agent: *
|
||||
```
|
||||
@@ -37,24 +31,23 @@ You may overwrite the internal template with a custom template. Hugo selects the
|
||||
|
||||
## robots.txt template example
|
||||
|
||||
{{< code file=layouts/robots.txt >}}
|
||||
```text {file="layouts/robots.txt"}
|
||||
User-agent: *
|
||||
{{ range .Pages }}
|
||||
Disallow: {{ .RelPermalink }}
|
||||
{{ end }}
|
||||
{{< /code >}}
|
||||
```
|
||||
|
||||
This template creates a robots.txt file with a `Disallow` directive for each page on the site. Search engines that honor the Robots Exclusion Protocol will not crawl any page on the site.
|
||||
|
||||
{{% note %}}
|
||||
To create a robots.txt file without using a template:
|
||||
|
||||
1. Set `enableRobotsTXT` to `false` in the site configuration.
|
||||
1. Create a robots.txt file in the `static` directory.
|
||||
|
||||
Remember that Hugo copies everything in the [`static` directory][static] to the root of `publishDir` (typically `public`) when you build your site.
|
||||
> [!note]
|
||||
> To create a robots.txt file without using a template:
|
||||
>
|
||||
> 1. Set `enableRobotsTXT` to `false` in the site configuration.
|
||||
> 1. Create a robots.txt file in the `static` directory.
|
||||
>
|
||||
> Remember that Hugo copies everything in the [`static` directory][static] to the root of `publishDir` (typically `public`) when you build your site.
|
||||
|
||||
[embedded template]: {{% eturl robots %}}
|
||||
[site configuration]: /configuration/
|
||||
[static]: /getting-started/directory-structure/
|
||||
{{% /note %}}
|
||||
|
||||
[site configuration]: /getting-started/configuration/
|
||||
|
Reference in New Issue
Block a user