TooLongFrameException happen when use Selenium with BrowserMob

Question

I am use Selenium with BrowserMob to perform the testing, I use requestfilter and responsefilter to modify the request header and response contents just like this:

public void proxySettings(){

    proxy.addResponseFilter((response, contents, messageInfo) -> {
        if((messageInfo.getOriginalRequest().toString().contains("/draft/footer"))&&(response.getStatus().code()==200)){
            contents.setTextContents(FileUtil.read("JsonPool//draft_footer.json"));
        }
    });
    proxy.addRequestFilter(new RequestFilter() {
        @Override
        public io.netty.handler.codec.http.HttpResponse filterRequest(io.netty.handler.codec.http.HttpRequest request,HttpMessageContents contents, HttpMessageInfo messageInfo) {
            if(messageInfo.getOriginalRequest().toString().contains("v1/doc"))
            {
                request.headers().set("Authorization", "Basic cHJpbmNpcGVDHHEHDNvbTpwd2Q0dGVzdCE=");
            }
            return null;
        }
    });
}

After I start proxy, and Selenium runs the test. somehow an exception shows and the running is stoped.

[LittleProxy-0-ProxyToServerWorker-4] ERROR org.littleshoot.proxy.impl.ProxyToServerConnection - (AWAITING_INITIAL) [id: 0x23fdf062, /192.168.1.103:49953 => www.html5rocks.com/216.239.38.21:80]: Caught an exception on ProxyToServerConnection io.netty.handler.codec.TooLongFrameException: HTTP content length exceeded 2097152 bytes.
at io.netty.handler.codec.http.HttpObjectAggregator.decode(HttpObjectAggregator.java:218) ~[browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.handler.codec.http.HttpObjectAggregator.decode(HttpObjectAggregator.java:57) ~[browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89) ~[browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:318) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:304) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:318) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:304) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [browsermob-dist-2.1.0-beta-4.jar:?]
at org.littleshoot.proxy.impl.ProxyConnection$ResponseReadMonitor.channelRead(ProxyConnection.java:738) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:318) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:304) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:276) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:263) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:318) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:304) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [browsermob-dist-2.1.0-beta-4.jar:?]
at org.littleshoot.proxy.impl.ProxyConnection$BytesReadMonitor.channelRead(ProxyConnection.java:692) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:318) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:304) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) [browsermob-dist-2.1.0-beta-4.jar:?]
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112) [browsermob-dist-2.1.0-beta-4.jar:?]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_40]

[ProxyToServerConnection io.netty.handler.codec.TooLongFrameException: HTTP content length exceeded 2097152 bytes.]

I did some research, and find that is http content is too big, but I don't know where to set this content length. I just want this exception can be resolved.

Jason Hoetger Jason Hoetger · Accepted Answer · 2016-04-02T23:45:34

You'll have to set the maximum request/response size yourself in order to handle messages larger than 2097152 bytes.

In short, add your request/response filter like this:

proxyServer.addFirstHttpFilterFactory(new RequestFilterAdapter.FilterSource(filter, 16777216)); // or whatever buffer size you want

There's a bit more information about this issue here: https://github.com/lightbody/browsermob-proxy/issues/372

TooLongFrameException happen when use Selenium with BrowserMob

1 Answers