Skip to main content

wget utility



wget utility is the best option to download files from internet. wget can pretty much handle all complex download situations including large file downloads, recursive downloads, non-interactive downloads, multiple file downloads etc.

1. Download Single File with wget

The following example downloads a single file from internet and stores in the current directory.
$wget http://www.openss7.org/repos/tarballs/strx25-0.9.2.1.tar.bz2

2. Download and Store With a Different File name Using wget -O

$wget -O taglist.zip
http://www.vim.org/scripts/download_script.php?src_id=7701

3. Continue the Incomplete Download Using wget -c

Restart a download which got stopped in the middle using wget -c option as shown below.
$wget -c http://www.openss7.org/repos/tarballs/strx25-0.9.2.1.tar.bz2

4. Download in the Background Using wget -b

For a huge download, put the download in background using wget option -b as shown below.
$wget -b http://www.openss7.org/repos/tarballs/strx25-0.9.2.1.tar.bz2

5. Download Multiple Files / URLs Using Wget -i

First, store all the download files or URLs in a text file as:
$cat > download-file-list.txt
URL1
URL2
URL3
URL4
Next, give the download-file-list.txt as argument to wget using -i option as shown below.

$wget -i download-file-list.txt

6. Download a Full Website Using wget –mirror

Following is the command line which you want to execute when you want to download a full website and made available for local viewing.

$wget --mirror -p --convert-links -P ./LOCAL-DIR WEBSITE-URL

7. Reject Certain File Types while Downloading Using wget –reject

You have found a website which is useful, but don’t want to download the images you can specify the following.

$wget --reject=gif WEBSITE-TO-BE-DOWNLOADED

8. Quit Downloading When it Exceeds Certain Size Using wget -Q

When you want to stop download when it crosses 5 MB you can use the following wget command line.

$wget -Q5m -i FILE-WHICH-HAS-URLS

9. Download Only Certain File Types Using wget -r -A

You can use this under following situations:
  • Download all images from a website
  • Download all videos from a website
  • Download all PDF files from a website 
$wget -r -A.pdf http://url-to-webpage-with-pdfs/

10. FTP Download With wget

You can use wget to perform FTP download as shown below.
Anonymous FTP download using Wget

$wget ftp-url


Labels:

Comments

Post a Comment

Popular posts from this blog

Writing a Bash Shell Script

Bash shell scripts in Linux are text files that contain a series of commands that can be executed by the Bash shell. Bash (Bourne Again Shell) is a popular shell in Linux and UNIX systems, and shell scripts are used to automate tasks, configure systems, or perform a sequence of operations. How to Write a Bash Shell Script Create a New File: You can create a new script using any text editor like nano , vim , or gedit . gedit myscript.sh Write the Script: A basic shell script begins with a "shebang" ( #!/bin/bash ) to specify the interpreter that will be used to execute the script. The rest of the file contains the commands to be run. Example of a simple script: #!/bin/bash # This is a comment echo "Hello, World!" # Print "Hello, World!" #!/bin/bash : Specifies that the script will be executed using the Bash shell. echo "Hello, World!" : A command that prints the string "Hello, World!" to the terminal. Comments: Any line starting ...
Post a Comment
Read more

Different syntax for writing arithmetic expressions in bash shell

#!/bin/bash echo "Enter two numbers" read a b s=`expr $a + $b` echo "Sum1=$s" s=$[$a+$b] echo "sum2=$s" ((s=$a+$b)) echo "sum3=$s" ((s=a+b)) echo "sum3=$s" let s=$a+$b echo "sum4=$s" let s=a+b echo "sum4=$s" Note:bash shell support only integer arithmetic.zsh support operations on real numbers.We can use bc in bash shell to do real arithmetic. Eg: echo "$a*$b"|bc # where a and b are real Mathematical Operators With Integers Operator Description Example Evaluates To + Addition echo $(( 20 + 5 )) 25 - Subtraction echo $(( 20 - 5 )) 15 / Division echo $(( 20 / 5 )) 4 * Multiplication echo $(( 20 * 5 )) 100  % Modulus echo $(( 20 % 3 )) 2 ++ post-increment (add variable value by 1) x=5 echo $(( x++ )) echo $(( x++ )) 5 6 -- post-decrement (subtract variable value by 1) x=5 echo $(( x-- )) 4 ** Exponentiation x=2 y=3 echo $(( x ** y )) 8
Post a Comment
Read more

Flow Control in shell script - if and case statements

Shell script usually consist of sequence of commands that starts at the first line and continues line by line until it reaches the end. Most programs do much more than this. They make decisions and perform different actions depending on conditions. The shell provides several commands that we can use to control the flow of execution in our program. In this lesson, we will look at the following:  test if case exit   test The test command is used most often with the if command to perform true/false decisions. The command is unusual in that it has two different syntactic forms: # First form test expression # Second form [ expression ] The test command works simply. If the given expression is true, test exits with a status of zero; otherwise it exits with a status of 1. Common numerical test conditions -gt   Greater than -lt    Less than -ge  Greater than or equal to -le    Less than or equal to -eq   Equal to -ne...
Post a Comment
Read more