How to have load balancer for nifi handlehttprequest processor

Question

I have 3 node Nifi cluster running(1.5 version). I have built the pipeline with handlehttprequest and intermediate set of processor for transformation/cleaning the incoming data followed by handlehttpresponse. I have provided the http end point to client applications. There are various client applications which would be posting the data through the rest end point provided by nifi. I had issue with Handlehttprequest running with execution mode as all nodes. Nifi UI becomes unresponsive after some time and I could see below error message in the nifi-app.log and I could see error message in the processor(handlehttprequest).

Changing the execution to primary node, solves the issue.

So need help with below: Handlehttprequest can't work on multinode cluster environment? If so, how to make use of multiple nodes to handle high throughput of incoming data posted by client applications? how to put load balancer for handlehttprequest endpoint and distribute the incoming data to multiple nodes on the cluster?

error message at processor level:

HandleHttpRequest[id=74ee1128-2de6-3979-a818-2b598186f7aa]       HandleHttpRequest[id=74ee1128-2de6-3979-a818-2b598186f7aa] failed to process due to org.apache.nifi.processor.exception.ProcessException: Failed to initialize the server; rolling back session: Failed to initialize the server

==========================

Nifi-app.log

2018-08-03 14:45:22,551 INFO [Timer-Driven Process Thread-6] org.eclipse.jetty.server.Server jetty-9.4.3.v20170317
2018-08-03 14:45:22,551 ERROR [Timer-Driven Process Thread-3] o.a.n.p.standard.HandleHttpRequest HandleHttpRequest[id=74ee1128-2de6-3979-a818-2b598186f7aa] Failed to process session due to org.apache.nifi.processor.exception.ProcessException: Failed to initialize the server: {}
org.apache.nifi.processor.exception.ProcessException: Failed to initialize the server
        at org.apache.nifi.processors.standard.HandleHttpRequest.onTrigger(HandleHttpRequest.java:488)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.BindException: Cannot assign requested address
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Net.java:433)
        at sun.nio.ch.Net.bind(Net.java:425)
        at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)

==============================================

Any help in providing the direction is appreciated.

Thanks, Vish

From the log, I think it is not able to bind to the provided port. Maybe the port which you are giving is already in use in one of your nodes. Try giving different port or check in all the nodes whether the port which you have provided is in use. — Sivaprasanna Sethuraman
Thanks @SivaprasannaSethuraman for the inputs. I am using one of hostname and port 8080 as endpoint in handlehttprequest and when the processor is not running I do not see that port being used on any of the hosts and when i start the processor in execution mode (all nodes), I can see mutiple threads open on coordinator node and cross nproc limit set on that node and nifi UI goes unresponsive. If i run handlehttprequest only on primary node, all looks fine. — vishwanatha handadi
What port is NiFi itself running on? NiFi defaults to 8080 so unless you changed that then you would have to assign a different port to HandleHttpRequest — Bryan Bende
Thanks @BryanBende for the response. Nifi is running on 8090 nifi.web.http.port=8090 is what is set in nifi.properties. so it is not the same port. — vishwanatha handadi
Did you specify anything for the hostname property in HandleHttpRequest? — Bryan Bende

Bryan Bende Bryan Bende · Accepted Answer · 2018-08-03T14:55:14

This error generally means there is an issue with the server binding to the host or port. Since it sounds like the port is available it is probably an issue with the hostname.

In a cluster you would have to leave the hostname property blank in HandleHttpRequest so it will bind to the address of each node, or you could use a dynamic expression like ${hostname}.

For load balancing, it should be the same as any other configuration to put a load balancer in front of a web application. There are some articles that cover it already:

https://pierrevillard.com/2017/02/10/haproxy-load-balancing-in-front-of-apache-nifi/

How to have load balancer for nifi handlehttprequest processor

error message at processor level:

Nifi-app.log

1 Answers