Skip to main content

wget utility



wget utility is the best option to download files from internet. wget can pretty much handle all complex download situations including large file downloads, recursive downloads, non-interactive downloads, multiple file downloads etc.

1. Download Single File with wget

The following example downloads a single file from internet and stores in the current directory.
$wget http://www.openss7.org/repos/tarballs/strx25-0.9.2.1.tar.bz2

2. Download and Store With a Different File name Using wget -O

$wget -O taglist.zip
http://www.vim.org/scripts/download_script.php?src_id=7701

3. Continue the Incomplete Download Using wget -c

Restart a download which got stopped in the middle using wget -c option as shown below.
$wget -c http://www.openss7.org/repos/tarballs/strx25-0.9.2.1.tar.bz2

4. Download in the Background Using wget -b

For a huge download, put the download in background using wget option -b as shown below.
$wget -b http://www.openss7.org/repos/tarballs/strx25-0.9.2.1.tar.bz2

5. Download Multiple Files / URLs Using Wget -i

First, store all the download files or URLs in a text file as:
$cat > download-file-list.txt
URL1
URL2
URL3
URL4
Next, give the download-file-list.txt as argument to wget using -i option as shown below.

$wget -i download-file-list.txt

6. Download a Full Website Using wget –mirror

Following is the command line which you want to execute when you want to download a full website and made available for local viewing.

$wget --mirror -p --convert-links -P ./LOCAL-DIR WEBSITE-URL

7. Reject Certain File Types while Downloading Using wget –reject

You have found a website which is useful, but don’t want to download the images you can specify the following.

$wget --reject=gif WEBSITE-TO-BE-DOWNLOADED

8. Quit Downloading When it Exceeds Certain Size Using wget -Q

When you want to stop download when it crosses 5 MB you can use the following wget command line.

$wget -Q5m -i FILE-WHICH-HAS-URLS

9. Download Only Certain File Types Using wget -r -A

You can use this under following situations:
  • Download all images from a website
  • Download all videos from a website
  • Download all PDF files from a website 
$wget -r -A.pdf http://url-to-webpage-with-pdfs/

10. FTP Download With wget

You can use wget to perform FTP download as shown below.
Anonymous FTP download using Wget

$wget ftp-url


Labels:

Comments

Post a Comment

Popular posts from this blog

Writing a Bash Shell Script

What Are Shell Scripts? In the simplest terms, a shell script is a file containing a series of commands. The shell reads this file and carries out the commands as though they have been entered directly on the command line. The shell is somewhat unique, in that it is both a powerful command line interface to the system and a scripting language interpreter. As we will see, most of the things that can be done on the command line can be done in scripts, and most of the things that can be done in scripts can be done on the command line. Writing Your First Script And Getting It To Work To successfully write a shell script, you have to do three things: Write a script Give the shell permission to execute it Put it somewhere the shell can find it Writing A Script A shell script is a file that contains ASCII text. To create a shell script, you use a text editor . A text editor is a program, like a word processor, that reads and writes ASCII text files. There are...
Post a Comment
Read more

Basic Linux Commands For Beginner's

Basic Linux Commands for Beginners Linux is an Operating System’s Kernel. You might have heard of UNIX. Well, Linux is a UNIX clone. But it was actually created by Linus Torvalds from Scratch. Linux is free and open-source, that means that you can simply change anything in Linux and redistribute it in your own name! There are several Linux Distributions, commonly called “distros”. A few of them are: Mint Ubuntu Linux Red Hat Enterprise Linux Debian Fedora Kali Linux is Mainly used in Servers. About 90% of the Internet is powered by Linux Servers. This is because Linux is fast, secure, and free! The main problem of using Windows Servers are their cost. This is solved by using Linux Servers. Forgot to mention, the OS that runs in about 80% of the Smartphones in the World, Android, is also made from the Linux Kernel. Yes, Linux is amazing! A simple example of its security is that most of the viruses in the world run on Windows, but not on Linux...
4 comments
Read more

Shell

Definition of the Shell Shell is an interactive environment which provides an interface to an Operating System. It gathers input from user and execute the commands. Bourne shell(sh)- 1977 The Bourne shell was introduced. The Bourne shell(sh), by Stephen Bourne at AT&T Bell Labs for V7 UNIX, remains a useful shell today (in some cases, as the default root shell). The Bourne shell was developed after working on an ALGOL68 compiler, so its grammar is more along the lines of Algorithmic Language (ALGOL) than other shells. The source code was developed in C. The Bourne shell served two primary goals: Executing UNIX/Linux commands for the operating system,i.e, command line interpreter Writing reusable scripts that could be invoked through the shell,i.e, scripting In addition to replacing the Thompson shell, the Bourne shell offered many other advantages over its predecessors such as control flows, loops, and variables into scripts, providing a more functional language to...
Post a Comment
Read more