cURL & wget

cURL is a software package which consists of command line tool and a library for transferring data using URL syntax.

cURL supports various protocols like, DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP.

Examples:

  • curl http://www.centos.org
  • curl -O URL1 -O URL2     // multiple file downloads
  • curl -L http://www.google.com     // follow HTTP location headers.
  • curl -C – -O http://www.gnu.org/software/gettext/manual/gettext.html
    // continue & resume download
  • curl -z 21-Dec-11 http://www.example.com/yy.html     // download a file if modified after given date.
  • curl -u username:password URL   // pass http authentication in curl
  • curl -u ftpuser:ftppass -O ftp://ftp_server/public_html/xss.php     // download files from ftp server
  • curl dict://dict.org/d:bash   // get definition of a word with DICT protocol
  • curl -x proxysever.test.com:3128 http://google.co.in   // use proxy for file download
  • curl –mail-from blah@test.com –mail-rcpt foo@test.com smtp://mailserver.com    // send mail with SMTP protocol

 

wget utility is the best option to download files from internet. wget can pretty much handle all complex download situations including large file downloads, recursive downloads, non-interactive downloads, multiple file downloads etc.

Examples:

 

Two interesting blog posts that I have come across for cURL and wget. The definitions and examples in this post are taken from these.

cURL: http://www.thegeekstuff.com/2012/04/curl-examples/
wget: http://www.thegeekstuff.com/2009/09/the-ultimate-wget-download-guide-with-15-awesome-examples/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s