download pdf files that are href links using ruby mechanize

0
votes

Using Ruby Mechanize I have successfully submitted input values to a form and am able to get the resultant page based on the search criteria. The resultant page has pdf files as ahref links that i need to download. Attribute href has value:

href='xxx.do?FILENAME=path/abc.pdf&SEARCHTEXT=aaa&ID=123_4

where SEARCHTEXT is the text entered as input originally. When i manually click the link pdf opens in a new window having url as http://someip:8080/xxx/temp/123_4 which is the same ID seen in the href attribute. The actual filename however is different and is of the form xxx.123_2_.doc. My below code returns 0 byte file -

scraper.pluggable_parser.pdf = Mechanize::FileSaver
File.open('n1pdf.pdf', 'wb'){|f| f << scraper.get(alink).body}

where alink=http://someip:8080/xxx/temp/123_4 If i use 

    File.open("new.pdf", "w") do |f|
    uri = URI(alink)
    f << Net::HTTP.get(uri)
    end

I get HTTP not found error. I am not sure if i am doing this correct. Is ID a session id that is generated dynamically since all pdf files on the resultant page have this ID with _1/2/3 as filename(or url).

Please note that whenever i manually click and open a pdf file and then hardcore that in my code the file downloads but does not when my code dynamically extracts the ID value and assigns to alink. Not sure if this is related to cookies. Kindly help. Thank You.

1
rubyweb-scrapingmechanize

1 Answers

0
votes

Make sure it's the right absolute url:

uri = scraper.page.uri.merge(a[:href])
puts uri # just check to be sure
File.open('n1pdf.pdf', 'wb'){|f| f << scraper.get(uri).body}