Wget is free and cross-platform command-line tool created by the GNU Project for downloading files from the web. It downloads files via protocols like FTP, SFTP, HTTP, and HTTPS. It supports multiple files downloading, downloading in the background, resuming downloads, recursive downloading, limiting the bandwidth for downloads and viewing headers. Wget is available to install in Linux, Unix and Windows system.
In this article, we will show you how to install and use the Wget command. We will also use some useful Wget command examples.
Install Wget
Wget tool comes pre-installed in most Linux distributions. You can check whether it is installed on your system, open Terminal or CMD, type wget and press enter. It will print bellow output.
laravelcode@Lenovo-G50-70:~$ wget
wget: missing URL
Usage: wget [OPTION]... [URL]...
Try `wget --help' for more options.
wget: missing URL
Usage: wget [OPTION]... [URL]...
Try `wget --help' for more options.
If wget command is not installed in your system, you can easily install it directly from the command-line.
To install wget on Ubuntu or Debian, execute the following command:
sudo apt-get install wget
To install wget on Fedora or CentOS, use the bellow command:
sudo yum install wget
Download files via wget
The basic syntax to download files is as bellow:
wget [options] [url]
Where [options] are how you want to download the files and [url] is the remote file path.
There are many ways you can download files via wget. Here is some useful options you can use to download files over the internet.
Download single file
This is the basic wget command that is used most. For example, download latest Ubuntu 18.04 LTS ISO file using bellow command.
wget http://releases.ubuntu.com/18.04/ubuntu-18.04.4-desktop-amd64.iso
Download file with different name
You can also download file with different name. Pass -O option in the command with new name. In the bellow example, file will be saved as ubuntu.iso.
wget -O ubuntu.iso http://releases.ubuntu.com/18.04/ubuntu-18.04.4-desktop-amd64.iso
Download file to different folder
You can download file direct to other folder by passing -P option with directory path. In the bellow command file will be stored in Documents folder.
wget -P ~/Documents http://releases.ubuntu.com/18.04/ubuntu-18.04.4-desktop-amd64.iso
Download multiple files
You can download multiple files at a once. For that, create new txt file and put all files link in that txt file. Here is my text document file which download multiple files at a once.
files.txt
http://releases.ubuntu.com/18.04/ubuntu-18.04.4-desktop-amd64.iso
https://download.fedoraproject.org/pub/fedora/linux/releases/31/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-31-1.9.iso
https://download.fedoraproject.org/pub/fedora/linux/releases/31/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-31-1.9.iso
After text document created, you can use -i to get all the files stored in your text file:
wget -i ~/Downloads/files.txt
Download resumable file
While downloading large files, it is important to download with resume.For that pass -c option to every time when resume the download.
wget -c http://releases.ubuntu.com/18.04/ubuntu-18.04.4-desktop-amd64.iso
Download file in the background
You can also download the file in the background using -b option, so you can use Terminal while downloading.
wget -b http://releases.ubuntu.com/18.04/ubuntu-18.04.4-desktop-amd64.iso
You can check current download progress in wget-log file of current directory.
nano wget-log
Download file with limit speed
Sometimes you also want to download file with limit speed. Pass --limit-rate option with maximum limit speed like this. In the bellow example, file will download with maximum speed of 100KB/S.
wget --limit-rate=100k http://releases.ubuntu.com/18.04/ubuntu-18.04.4-desktop-amd64.iso
Download zip, tar file and extract to specified directory
You can also download tar or zip file and extract to specified directory. Run the bellow command to download tar file with extracted to specified folder
wget -q -O - https://website.com/path/file.tar.gz | tar -xzf - --strip 1 -C ~/Downloads/file
Download file with increase retry attempts
You can add retry attempts --tries
wget --tries=50 http://releases.ubuntu.com/18.04/ubuntu-18.04.4-desktop-amd64.iso
Download file over FTP
The Wget command can also use with FTP. You’ll only need to specify the username and password as in this wget example:
wget --ftp-user=USERNAME --ftp-password=PASSWORD ftp://website.com/path/file.zip
Download whole website
You can also download whole website in your local system. Use the bellow command to download the full website:
wget --mirror --convert-links --page-requisites --no-parent -P Downloads/website/ https://website.com
In the above command,
--mirror download with recursive strategy
--convert-links All links converts to offline
--page-requisites download all files
-P Download content to specified directory
--convert-links All links converts to offline
--page-requisites download all files
-P Download content to specified directory
Download files over HTTPS site without checking SSL certificate
If you want to download the file over HTTPS without checking SSL certificate, run the bellow command. This is useful in case invalid certificate.
wget --no-check-certificate https://website.com/path/file.zip
Set your user agent
You can also set your User-Agent by self while downloading files. Pass -U option to add your User-Agent. It is useful in arbitrary string to be set for the user agent.
wget -U YOUR-USER_AGENT https://website.com/path/file.zip
View Response Header
To view response header of the file, pass -S --spider option along with command.
wget -S --spider https://www.google.com
Conclusion
This way, you can use Wget command-line tool for download files from the internet. There are many other tricks you can use. To get more details about Wget visit the GNU wget Manual page.
If you have any query, please do let us know in the bellow comment section.
