Is there a way to have a proxied request respond straight to the requestor?

1
votes

So let's say I have the following (obviously simplified) architecture: I have 10 servers running the same REST API endpoint. I have an intermediate API which fields requests, and then forwards it to one of the servers (a load balancer).

Now let's imagine that this is a really big, streaming response. As such, I obviously don't want the data to have to go back through the load balancer -- because wouldn't this bog down and defeat the purpose of the load balancing server?. What would be the proper way to implement a load balancing system that would delegate a request to node but not force the response back through the load balancing server?

Further, are there any REST frameworks on the JVM that implement this?

1
serverload-balancing
What you're after here is called load-balancing. I've edited your question to make the wording very clear. – user562566
Yeah I mean I imagine that this is something people have long dealt with... are there any well known ways to deal with it when you know that you have a lot of data to stream? I imagine one way would be to make every request two hits...the first is to give you a backend, the second then goes straight to the backend. (and thanks for the edit) – A Question Asker
Yes this is something that probably anyone who has ever worked with even a slightly successful web service has had to deal with. I just made the edits to help make the question clearer, I'll wait for someone with more experience to post an answer. In the meantime, en.wikipedia.org/wiki/… – user562566
Cool, thank you. In this particular case, the issue isn't as much that the connection is long lived as much as there is a lot of data to transfer. I could definitely chunk it up into a ton of smaller requests, but if all the data goes through the load balancer there are going to be problems. It seems like this is a problem any time where the amount of work done on the rest endpoint isn't very high... or you just have a huge numbers of servers in the backend. – A Question Asker

1 Answers

1
votes

What you are looking for is called DSR (direct server return). you can attempt to google it a bit. AFAIK most hardware load balancers have this option.
The question is what load balancer are you using? Is it hardware, ELB on AWS, HAProxy? For example: http://blog.haproxy.com/2011/07/29/layer-4-load-balancing-direct-server-return-mode/

If you're not really into load balancers, you could attempt to set this up in 2 stages: first - the client hits the API and gets the ip of the server, second the client talks to the servers. The hard part will be not to overload some servers when leaving others idle (both initial setup and rebalancing workloads as time goes by)