Merge branch 'master' into graceful-queues

docs/content/doc/advanced/config-cheat-sheet.en-us.md

@@ -186,7 +186,7 @@ Values containing `#` or `;` must be quoted using `` ` `` or `"""`.
 - `STATIC_ROOT_PATH`: **./**: Upper level of template and static files path.
 - `STATIC_CACHE_TIME`: **6h**: Web browser cache time for static resources on `custom/`, `public/` and all uploaded avatars.
 - `ENABLE_GZIP`: **false**: Enables application-level GZIP support.
-- `LANDING_PAGE`: **home**: Landing page for unauthenticated users  \[home, explore\].
+- `LANDING_PAGE`: **home**: Landing page for unauthenticated users \[home, explore, organizations, login\].
 - `LFS_START_SERVER`: **false**: Enables git-lfs support.
 - `LFS_CONTENT_PATH`: **./data/lfs**: Where to store LFS files.
 - `LFS_JWT_SECRET`: **\<empty\>**: LFS authentication secret, change this a unique string.

docs/content/doc/advanced/customizing-gitea.en-us.md

@@ -44,12 +44,6 @@ environment variable; this can be used to override the default path to something
 
 **Note:** Gitea must perform a full restart to see configuration changes.
 
-## Customizing /robots.txt
-
-To make Gitea serve a custom `/robots.txt` (default: empty 404), create a file called
-`robots.txt` in the `custom` folder (or `CustomPath`) with
-[expected contents](http://www.robotstxt.org/).
-
 ## Serving custom public files
 
 To make Gitea serve custom public files (like pages and images), use the folder

docs/content/doc/advanced/search-engines-indexation.en-us.md

@@ -0,0 +1,39 @@
+---
+date: "2019-12-31T13:55:00+05:00"
+title: "Advanced: Search Engines Indexation"
+slug: "search-engines-indexation"
+weight: 30
+toc: true
+draft: false
+menu:
+  sidebar:
+    parent: "advanced"
+    name: "Search Engines Indexation"
+    weight: 60
+    identifier: "search-engines-indexation"
+---
+
+# Search engines indexation of your Gitea installation
+
+By default your Gitea installation will be indexed by search engines.
+If you don't want your repository to be visible for search engines read further.
+
+## Block search engines indexation using robots.txt
+
+To make Gitea serve a custom `robots.txt` (default: empty 404) for top level installations,
+create a file called `robots.txt` in the [`custom` folder or `CustomPath`]({{< relref "doc/advanced/customizing-gitea.en-us.md" >}})
+
+Examples on how to configure the `robots.txt` can be found at [https://moz.com/learn/seo/robotstxt](https://moz.com/learn/seo/robotstxt).
+
+
+```txt
+User-agent: *
+Disallow: /
+```
+
+If you installed Gitea in a subdirectory, you will need to create or edit the `robots.txt` in the top level directory.
+
+```txt
+User-agent: *
+Disallow: /gitea/
+```