Aws Lambda Ruby Crawler selenium chrome driver: 'unknown error: unable to discover open window in chrome'

2
votes

I want to deploy my crawler to 'Aws Lambda Environment'. I was tested my code in lambda-ci (https://github.com/lambci/docker-lambda/tree/master/ruby2.5). and the result is success.

But when i try to deploy my code in aws lambda, but i received this error. 'unknown error: unable to discover open window in chrome' enter image description here

My Project structure is

  /project
    lambda_hanlder.rb
    vender/

and i uploaded headless-chrome, chromedriver on aws lambda layer.

headless-chrome version is v1.0.0-55 (https://github.com/adieuadieu/serverless-chrome)

chromedriver version is 2.4.2 (https://chromedriver.storage.googleapis.com/2.42/chromedriver_linux64.zip)

and this is my ruby code setup

Selenium::WebDriver::Chrome::Service.driver_path = '/opt/bin/chromedriver'
    options = Selenium::WebDriver::Chrome::Options.new(args: ['--headless', '--no-sandbox', '--disable-dev-shm-usage',
                                                              '--disable-gpu', '--disable-gpu-sandbox'])
    options.binary = '/opt/bin/headless-chromium'
    options.add_argument('window-size=1920x1080')
    options.add_argument('"user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"')

and my gem file is

GEM
  remote: https://rubygems.org/
  specs:
    archive-zip (0.12.0)
      io-like (~> 0.3.0)
    childprocess (3.0.0)
    chromedriver-helper (2.1.1)
      archive-zip (~> 0.10)
      nokogiri (~> 1.8)
    ethon (0.12.0)
      ffi (>= 1.3.0)
    ffi (1.12.2)
    io-like (0.3.1)
    logger (1.4.2)
    mini_portile2 (2.4.0)
    multi_json (1.14.1)
    nokogiri (1.10.8)
      mini_portile2 (~> 2.4.0)
    rubyzip (2.2.0)
    selenium-webdriver (3.142.7)
      childprocess (>= 0.5, < 4.0)
      rubyzip (>= 1.2.2)
    typhoeus (1.3.1)
      ethon (>= 0.9.0)

I do not know what is the problem. Does that problem is (headless, chromedriver) version compatibility? But In the lambda-ci-ruby enviroment, i didn't receive those error. How can i solve this problem?

1
rubyaws-lambdaselenium-chromedriver
Have you found a solution for you issue? I am getting the same error. The thing is that it runs without errors in a docker container. But it does not work in AWS. - Eugene Kovalev
Yea i found solution for this issue. The problem was chromedriver options. options = Selenium::WebDriver::Chrome::Options.new options.add_argument("--headless") options.add_argument("--disable-gpu") options.add_argument("--disable-application-cache") options.add_argument("--disable-infobars") options.add_argument("--no-sandbox") options.add_argument('--disable-dev-shm-usage') options.add_argument("--hide-scrollbars") options.add_argument("--enable-logging") options.add_argument("--single-process") options.add_argument("--ignore-certificate-errors") - Hoonki
options.add_argument("--homedir=/tmp") options.add_argument("--log-level=0") So when i try to execute my code with this options, it works! If you need any help, contact me or comment here. - Hoonki

1 Answers

1
votes

adding

options.add_argument("--single-process")

resolved the issue for me.

I have used:

Driver: 2.41

Chromium :  68.0.3440.84