Downloading an Entire Web Site with wget

wget command line

If you ever need to download an entire Web site, perhaps for off-line viewing, wget can do the job—for example:

$ wget \
     --recursive \
     --no-clobber \
     --page-requisites \
     --html-extension \
     --convert-links \
     --restrict-file-names=windows \
     --domains website.org \
     --no-parent \
         www.website.org/tutorials/html/

This command downloads the Web site www.website.org/tutorials/html/.

The options are:

  • --recursive: download the entire Web site.

  • --domains website.org: don't follow links outside website.org.

  • --no-parent: don't follow links outside the directory tutorials/html/.

  • --page-requisites: get all the elements that compose the page (images, CSS and so on).

  • --html-extension: save files with the .html extension.

  • --convert-links: convert links so that they work locally, off-line.

  • --restrict-file-names=windows: modify filenames so that they will work in Windows as well.

  • --no-clobber: don't overwrite any existing files (used in case the download is interrupted and resumed).

Load Disqus comments

Corporate Patron

Linode Logo

 

Pulseway Logo

Limited Time Offer

September Cover

 

Take Linux Journal for a test drive. Download our September issue for FREE.

Topic of the Week

Cloud

The cloud has become synonymous with all things data storage. It additionally equates to the many web-centric services accessing that same back-end data storage, but the term also has evolved to mean so much more.