Tag: wget

If your server support both IPv4 and IPv6 and you need to force download a file using IPv4, you can use

wget -4 URL_HERE

To force IPv6 connection, use

wget -6 URL_HERE

Example

[root@li2136-233 ~]# wget -4 https://rpms.remirepo.net/enterprise/remi-release-7.rpm
--2020-09-22 04:45:30--  https://rpms.remirepo.net/enterprise/remi-release-7.rpm
Resolving rpms.remirepo.net (rpms.remirepo.net)... 195.154.241.117
Connecting to rpms.remirepo.net (rpms.remirepo.net)|195.154.241.117|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 20732 (20K) [application/x-rpm]
Saving to: ‘remi-release-7.rpm’

100%[==================================================================================================================================================================>] 20,732      74.2KB/s   in 0.3s   

2020-09-22 04:45:32 (74.2 KB/s) - ‘remi-release-7.rpm’ saved [20732/20732]

[root@li2136-233 ~]# 

from wget help, here are the options

See wget

wget is used to download files. When you use wget to run cronjob, it creates a lot of files. To avoid this, you can use the wget option -qO-

If you want to use wget, you can use -O option, that speicify file to save. You can use -O /dev/stdout, so downloaded file content will be written to stdout.

wget -O /dev/stdout http://checkip.amazonaws.com

You can use -q to hide the download progres message (quite).

wget -q -O /dev/stdout http://checkip.amazonaws.com

-O /dev/stdout can be replaced with

-O -

Or

-O-

Example

wget -qO- http://checkip.amazonaws.com

See wget

Mirroring a site with wget
wget print content to screen
wget force IPv4/6 download

Download Multiple Files

To download multiple files from FTP server, run

wget ftp://backup2.serverok.in/backup1/*.tar.gz --ftp-user=USER --ftp-password=PASSWORD

Download all files in a folder

wget -r -l1 --no-parent http://cloud.serverok.in/public/

Download Entire Website

wget \
 --recursive \
 --no-clobber \
 --page-requisites \
 --html-extension \
 --convert-links \
 --restrict-file-names=windows \
 --domains example.com \
 --no-parent \
     www.example.com

Or

wget --mirror --convert-links --adjust-extension --page-requisites --no-parent https://domain.com/

Download with Limit Rate

wget -c --limit-rate=60k http://dn.serverok.in/1.mkv

This limits the download rate to 60.0KB/s

Back to Linux Commands

To mirror a web site for local viewing, you can use

wget --mirror --convert-links --html-extension --wait=2 -o log http://www.domain.com

–convert-links = After downloading, convert the links suitable for local viewing.