From c47177323168c016a4fea915d37b5e7208c41a9e Mon Sep 17 00:00:00 2001 From: Nick Sweeting Date: Tue, 2 Apr 2019 17:27:49 -0400 Subject: [PATCH] Updated Configuration (markdown) --- Configuration.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/Configuration.md b/Configuration.md index e1a18d5..75c1928 100644 --- a/Configuration.md +++ b/Configuration.md @@ -99,6 +99,15 @@ Some text to display in the footer of the archive index. Useful for providing s *Related options:* [`TEMPLATES_DIR`](#templates_dir) +--- +#### `URL_BLACKLIST` +**Possible Values:** [`None`]/`".*\.exe$"`/`"((.+\.)?youtube\.com)|(amazon\.com)'... + +A regex expression used to exclude certain URLs from the archive. You can use if there are certain domains, extensions, or other URL patterns that you want to ignore whenever they get imported. Blacklisted URLs wont be included in the index, and their page content wont be archived. + +*Related options:* +[`FETCH_MEDIA`](#FETCH_MEDIA), [`FETCH_GIT`](#FETCH_GIT), [`GIT_DOMAINS`](#GIT_DOMAINS) + --- ## Archive Method Toggles