BrowserMob Proxy and Webdriverio minimal, empty HAR file

Question

I was wondering if anyone has any idea as to what could be the problem or additional steps I could take in resolving the following issue.

I have a requirement to capture the network traffic so I can write some selenium tests verifying certain request headers.

The problem is when trying to use Webdriver.io + Browsermob proxy the HAR file created contains very minimal information.

I have tried using the C# bindings of the proxy and that resulted in the same issue as the Javascript ones. The only bindings that I got to work which returned data in the HAR file were the Java ones. There's a business requirement though that I must use Webdriver.io to implement this.

I have checked a number of other Questions and Answers from stack overflow but to no avail. Most have not been answered and the ones that have been have not worked for me just yet.

The code I am currently running is very similar to this one with the difference being the URL I'm trying to get the information from.

The difference between my Java code which worked and the Javascript and C# ones was that the Java one sets up the proxy programmatically whereas the other two expect the proxy to be already running and just connect to it.

Even with that in mind, when I start the C# or Javascript tests the proxy registers the new connection. The selenium server also starts up fine. Both cleanup with no issues according to their respective consoles once the tests are finished.

Are there any other ways to potentially debug this? Or even alternatives to capture the network traffic (Must work cross browser - already have a solution which works for chrome using chrome-remote-interface but we saw suggestions to use browsermob proxy for cross browser network capture).

Thanks for your time

Something I forgot to mention which is pretty important. Below is the npm package I am using for the browsermob-proxy :

https://www.npmjs.com/package/browsermob-proxy

Pan Pan · Accepted Answer · 2018-09-04T22:05:24

Okay so, I figured out why I wasn't getting any data. I hope this saves others some time and hassle.

The problem is the way browsermob proxy handles localhost. I instead switched the proxy to use my IPV4 address and it started capturing all the HAR data.

See the code below:

//BroswerMobProxy + webdriver.io
//npm package used https://www.npmjs.com/package/browsermob-proxy


var webdriverio = require('webdriverio');

//proxy settings, host is IPV4 address
var Proxy = require('browsermob-proxy').Proxy
, fs = require('fs')
, proxy = new Proxy({host: 'Put IPV4 Address Here', proxyPort: 8081 , selHost: 'Put IPV4 Address Here'});
;

//convenience method that 
proxy.cbHAR({captureHeaders: true, captureContent: true, captureBinaryContent: true }, doSeleniumStuff, function(err, data) {

  if (err) {
    console.error('ERR: ' + err);
  } else {

    /* Make sure har results are in the correct shape
    *  for any further processing
    */
    var harResultsString = JSON.stringify(data);
    var harResultsJson = JSON.parse(harResultsString);

    //Write HAR file
    fs.writeFileSync('DemoFile.json', harResultsJson, 'utf8');

    //Print to console
    console.log(harResultsJson);
  }
});

//webdriver.io options
const opts = {
  desiredCapabilities:{
    browserName: 'chrome',
    proxy: {
      proxyType: 'MANUAL',
        httpProxy: String(proxy.host)+":"+String(proxy.proxyPort),
        sslProxy:  String(proxy.host)+":"+String(proxy.proxyPort),
        socksProxy:String(proxy.host)+":"+String(proxy.proxyPort),
        socksVersion: 4,
      autodetect: false
     },
     acceptSslCerts: true,
     acceptInsecureCerts: true
  },
  host: 'Put IPV4 Address here',
  port: 4444,
  protocol: 'http',
  coloredLogs: true,
  proxy: 'http://'+String(proxy.host)+":"+String(proxy.proxyPort),
}

function doSeleniumStuff(proxy, cb) {
  var browser = webdriverio.remote(opts);
  // console.log(browser.options);
  browser
    .init()
    .url('http://yahoo.com.au')
    .getTitle().then(function(title) {
      console.log('Title was: ' + title);
    })
    .end().then(cb).catch(e => console.log(e));
}

BrowserMob Proxy and Webdriverio minimal, empty HAR file

1 Answers