How could I control max requests per a backend instance

Question

I wanted to know how many requests are allowed to be processed at the same time on a backend instance.
So, I tried to run the following code.

Serverside code

class BackendProcess(webapp2.RequestHandler):
    def post(self):
        id = uuid.uuid4().hex
        time.sleep(30)
        self.response.out.write(id)

backends.yaml

backends:
- name: backstream
  class: B1
  instances: 1
  options: public

Clientside Code

$('#backendtest').on('click', function(){
                $.ajax({
                    type: 'POST',
                    dataType: 'text',
                    url: "{{backend_url}}" + '/test',
                    success: function(response){
                        console.log(response);
                    }
                   });
            });

I opened 8 tabs on a chrome browswer and clicked the button '#backendtest' on each tabs.
Then, the responses of first 6 tabs were returned almost at the same time but the responses of 7th and 8th tabs were returned 30 seconds later after the response of the 1th tab.
This means that a backend instance could handle only 6 requests at the same time.
So, I added one more instance like the following.

backends:
- name: backstream
  class: B1
  instances: 2
  options: public

But still the responses of 7th and 8th tabs were returned 30 seconds later after the response of the 1th tab.
How does it come?????

Here is the log.

2013-05-05 05:50:31.220 /test 200 30035ms 0kb Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
I 2013-05-05 05:50:31.217 Saved; key: appstats:001200, part: 27 bytes, full: 1565 bytes, overhead: 0.000 + 0.002; link: http://xxxxxxxxx.appspot.com/_ah/sta
2013-05-05 05:50:29.423 /test 200 30186ms 0kb Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
I 2013-05-05 05:50:29.420 Saved; key: appstats:099400, part: 27 bytes, full: 1565 bytes, overhead: 0.000 + 0.003; link: http://xxxxxxxxx.appspot.com/_ah/sta
2013-05-05 05:50:08.984 /test 200 30034ms 0kb Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
I 2013-05-05 05:50:08.982 Saved; key: appstats:078900, part: 27 bytes, full: 1565 bytes, overhead: 0.000 + 0.003; link: http://xxxxxxxxx.appspot.com/_ah/sta
2013-05-05 05:50:07.217 /test 200 30189ms 0kb Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
I 2013-05-05 05:50:07.214 Saved; key: appstats:077000, part: 27 bytes, full: 1567 bytes, overhead: 0.000 + 0.154; link: http://xxxxxxxxx.appspot.com/_ah/sta
2013-05-05 05:50:05.181 /test 200 30212ms 0kb Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
I 2013-05-05 05:50:05.179 Saved; key: appstats:075100, part: 27 bytes, full: 1565 bytes, overhead: 0.000 + 0.002; link: http://xxxxxxxxx.appspot.com/_ah/sta
2013-05-05 05:50:03.214 /test 200 30102ms 0kb Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
I 2013-05-05 05:50:03.211 Saved; key: appstats:073200, part: 27 bytes, full: 1565 bytes, overhead: 0.000 + 0.002; link: http://xxxxxxxxx.appspot.com/_ah/sta
2013-05-05 05:50:00.965 /test 200 30036ms 0kb Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
I 2013-05-05 05:50:00.962 Saved; key: appstats:070900, part: 27 bytes, full: 1565 bytes, overhead: 0.000 + 0.002; link: http://xxxxxxxxx.appspot.com/_ah/sta
2013-05-05 05:49:59.014 /test 200 30035ms 0kb Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
I 2013-05-05 05:49:59.012 Saved; key: appstats:069000, part: 27 bytes, full: 1565 bytes, overhead: 0.000 + 0.003; link: http://xxxxxxxxx.appspot.com/_ah/sta

I also tried this 8 requests test on fronted instances and the result was the same.
So, this is not only a backend phenomenon.

I would like to increase the number of requests which could be processed at once, especially on a backend instance. — Nigiri
If it is not possible, I'm ready to increase the number of instances. But it seems not working with my test. — Nigiri

Nigiri Nigiri · Accepted Answer · 2013-05-06T11:07:10

It was because of the backends option "public".
After I changed it to "resident" as removing the options syntax, the load balancing between instances worked fine.
I guess the load balancing doesn't work between "public" backends instances.

How could I control max requests per a backend instance

1 Answers