mirror/archivebox
1
0
Fork 0
mirror of https://github.com/pirate/ArchiveBox.git synced 2025-08-27 00:05:27 +02:00
Code Issues Releases Wiki Activity

Updated Usage (markdown)

Nick Sweeting
2021-07-05 20:57:36 -04:00
parent 3115681499
commit 6b75c03434
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
Usage.md

@@ -165,7 +165,7 @@ Those numbers are from running it single-threaded on my i5 machine with 50mbps d
Storage requirements go up immensely if you're using `FETCH_MEDIA=True` and are archiving many pages with audio & video. Storage requirements go up immensely if you're using `FETCH_MEDIA=True` and are archiving many pages with audio & video.
You can run it in parallel by manually splitting your URLs into separate chunks: You can try to run it in parallel by manually splitting your URLs into separate chunks (though this may not work with `database locked` errors on slower filesystems):
```bash ```bash
archivebox add < urls_chunk_1.txt & archivebox add < urls_chunk_1.txt &
archivebox add < urls_chunk_2.txt & archivebox add < urls_chunk_2.txt &