First
[anni] / docs / administration / CLI_tasks / robots_txt.md
1 # Managing robots.txt
2
3 {! backend/administration/CLI_tasks/general_cli_task_info.include !}
4
5 ## Generate a new robots.txt file and add it to the static directory
6
7 The `robots.txt` that ships by default is permissive. It allows well-behaved search engines to index all of your instance's URIs.
8
9 If you want to generate a restrictive `robots.txt`, you can run the following mix task. The generated `robots.txt` will be written in your instance [static directory](../../../configuration/static_dir/).
10
11 === "OTP"
12
13     ```sh
14     ./bin/pleroma_ctl robots_txt disallow_all
15     ```
16
17 === "From Source"
18
19     ```sh
20     mix pleroma.robots_txt disallow_all
21     ```