Here’s a core utility that will surely be of use for Linux or Mac users: cURL. In my own description, it’s sorta like
wget, but different. Description from the website:
curl is a tool to transfer data from or to a server, using one of the supported protocols (HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, DICT, TELNET, LDAP or FILE). The command is designed to work without user interaction.
Also, it might be good to note this as well now that I brought up
Curl is not a wget clone. That is a common misconception. Never, during
curl’s development, have we intended curl to replace wget or compete on its
market. Curl is targeted at single-shot file transfers.
I took some time to play around with it a bit and found that it’s quite easy to use and the documentation and man pages are nicely laid out. For example, here’s how I used
curl to download an image from a password protected website:
curl http://www.thewebsite.com/image.jpg --user theusername:thepassword > image.jpg
I wanted to see if I could download multiple files at a time, so I became a little fancier and threw in a
for loop with the
seq command for counting. Since I knew all of my images were taken in order (100.jpg -> 110.jpg), I could use something like:
for i in $(seq 100 110); do curl http://www.thewebsite.com/$i.jpg --user theusername:thepassword > $i.jpg; done
I’m sure that this hasn’t even scratched the surface for what curl can do, so be sure to check out the man pages and have fun with it!