7

[Bash] Use wget to Fetch Webpages

 2 years ago
source link: http://siongui.github.io/2016/01/02/bash-use-wget-to-fetch-webpages/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

Question

Fetch webpages http://www.aia.or.th/prayerXX.htm, where XX are numbers from 00 to 99.

Answer

wget-webpages.sh | repository | view raw 

#!/bin/bash
# wget webpages from http://www.aia.or.th/prayerXX.htm
# Google search: bash print number with leading zeros
# run this script with the following command on Ubuntu 15.10
# $ bash wget-webpages.sh
# the following command is not working:
# $ sh wget-webpages.sh

for i in {01..99}
do
  wget "http://www.aia.or.th/prayer$i.htm" -P prayer
  sleep .3
done

GNU wget is used to fetch the webpages. I tried to run the script:

$ sh wget-webpages.sh

but did not work. Need to run with the following command:

$ bash wget-webpages.sh

My development environment is Ubuntu Linux 15.10.

References:

[1]Google search: bash print number with leading zeros

[2]GNU Wget Manual (or $ man wget in command line in Linux console)

[3]sleep(3): sleep for specified number of seconds - Linux man page (or $ man sleep in command line in Linux console)


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK