service discovery, load balancing and connection pooling approach

Question

There are two approaches that can be used for service interaction when having SOA for large systems deployed on cloud like AWS.

1. Have each service cluster behind internal elb. client makes a connection pool with corresponding elb and elb does round-robin balancing.

2. going with service discovery approach like netflix eureka.

Currently we are using 1st approach where each service cluster is behind internal elb and clients communicate via elbs so each client instance has to maintain only 1 pool i.e. with the elb endpoint.

I have following doubts regarding 2nd apporach.

1. Is there a benefit in moving to service discovery and smart client architecture where service client knows all service instances (via eureka service or equivalent) and does internal load balancing?
2. In above case how does connection pooling work? Currently each client instance has to maintain exactly 1 connection pool i.e. with the corresponding service's elb. But with rich client each client will have all the service instance endpoints to directly communicate to. Making connection on each request will not be efficient and having so many connection pools (1 for each service instance) for each client is a overkill I guess.

Need inputs/suggestions on above two questions.

Answer 1

First question.

Yes there is. First, you can do better failure recovery - for example, retry failed requests to another node without showing any errors to client. Next, you can do better balancing than ELB offers. Next, you can automatically add/remove nodes to/from cluster w/o altering ELB configuration. This is very useful if your nodes have healthchecks. More importantly, software balancer can do this fast.

Second question.

Have connection pool per node. i.e. [api method in client code] -> [software balancer] -> [node connection pool] -> [node connection] -> [use this connection to make request]