wget to download multiple folders from https url

Question

I am using following line of wget to download dataset.

wget -e robots=off -m -np -R .html,.tmp -nH --cut-dirs=3 \
                 https://ladsweb.modaps.eosdis.nasa.gov/archive/orders/50672578/ \
                 --header "Authorization: Bearer <your appKey>" -P <target dir>

Now, I want to download multiple folders from same URL (i.e. https://ladsweb.modaps.eosdis.nasa.gov/archive/orders/50672578/; https://ladsweb.modaps.eosdis.nasa.gov/archive/orders/50672579/; https://ladsweb.modaps.eosdis.nasa.gov/archive/orders/50672580/ etc)

I am using for loop in shell script for a same as following:

for i in 50672578 50672579 50672580 
do
    wget -e robots=off -m -np -R .html,.tmp -nH --cut-dirs=3 \
                 https://ladsweb.modaps.eosdis.nasa.gov/archive/orders/$i/ \
                 --header "Authorization: Bearer <your appKey>" -P <target dir> 
done

But this gives me the following error:

Proxy request sent, awaiting a response... 400 Bad Request 2018-11-12 12:18:34 ERROR 400: Bad Request.

Individually it is working fine but not via for-loop.

Kindly request to indicate my mistakes.

Thank you in advance

gopy gopy · Accepted Answer · 2018-11-13T07:32:12

There is no need to use a for loop you can express the list of urls as follows:

https://ladsweb.modaps.eosdis.nasa.gov/archive/orders/{50672578,50672579,50672580}/

wget to download multiple folders from https url

2 Answers