Headless browser testing with download functionality?

7
votes

I've been looking for a solution to do headless testing in osx. But I need the ability to save files returned by the server.

I've tested selenium, phantomjs, casperjs and have looked into anything I could find online.

none of them supports downloading. am I missing something? are there any headless browser/testing frameworks that support downloads?

2
seleniumphantomjscasperjsheadless-browser
CasperJS has a download function, so it supports downloading. PhantomJS supports downloading because CasperJS is built on top of it. What is the problem that you're having? – Artjom B.
@ArtjomB. what do you mean download function? the file is a result of an http post so it's sent as a Content-Disposition: attachment; filename=FILENAME in the header – Mars
Yes, it's true that PhantomJS isn't saving a file, when it receives such a request, but the file in question can be downloaded such a case from an matching event handler. – Artjom B.
possible duplicate of downloading a file that comes as an attachment in a POST request response in PhantomJs – Artjom B.
yes! but you cannot call resource.url on a header attachment. @ArtjomB. because the resource url is just an html page. and thus not the file that needs to be saved – Mars

2 Answers

4
votes

What you can do is:

  • start a virtual display (see Xvfb)
  • start up Firefox browser with preferences configured to automatically save csv files

Working example in python with additional comments (using pyvirtualdisplay xvfb wrapper):

from os import getcwd
import time

from pyvirtualdisplay import Display
from selenium import webdriver

# start the virtual display
display = Display(visible=0, size=(800, 600))
display.start()

# configure firefox profile to automatically save csv files in the current directory
fp = webdriver.FirefoxProfile()
fp.set_preference("browser.download.folderList", 2)
fp.set_preference("browser.download.manager.showWhenStarting", False)
fp.set_preference("browser.download.dir", getcwd())
fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "text/csv")

browser = webdriver.Firefox(firefox_profile=fp)
browser.get('http://www.nationale-loterij.be/nl/onze-spelen/lotto/resultaten')

# check the option
browser.find_element_by_id('corporatebody_3_corporategrid93961a8f9b424ed6bd0697df356d9483_1_rblType_0').click()

# click the link
browser.find_element_by_name('corporatebody_3$corporategrid93961a8f9b424ed6bd0697df356d9483_1$btnDownload').click()

# hardcoded delay for waiting a file download (better check for the downloaded file to appear on the disk)
time.sleep(2)

# quit the browser
browser.quit()

# stop the display
display.stop()

See also:

1
votes

I use on OSX selenium + wget command to perform downloads.

Here it is a sample of code:

new_driver = webdriver.Firefox()
new_driver.get(url)
for element in new_driver.find_elements_by_tag_name('img'):
    os.system('wget ' + element.get_attribute('src').rstrip('\n'))