Use wget to download / scrape a full website

preview_player
Показать описание
********************************************
Using wget to download/scrape an entire site
********************************************

Example 1: Basic Example

Example 2: Basic Template

wget \
--recursive \
--no-clobber \
--page-requisites \
--html-extension \
--convert-links \
--restrict-file-names=unix \
--no-parent \

Example 3: More Advanced Template
wget \
--recursive \
--no-clobber \
--page-requisites \
--html-extension \
--convert-links \
--restrict-file-names=unix \
--wait=5 \
--limit-rate=20K \
--user-agent=Mozilla \
--level=10 \
--no-parent \

Other options
--random-wait
Рекомендации по теме