Cosmos DB - Why the requests are not getting throttled?

2
votes

I have created a "Free" Cosmos DB account (SQL API) in my Azure Subscription. In that account, I have created a database with shared throughput (400 RU/s) and that database has a container which will utilize database's throughput (there is only one container in the database).

I am now trying to insert some items in my container using batch. Each batch contains exactly 100 items (each item is less than 1K) and I am trying to create 5000 entities (i.e. 50 batches). Each batch insert is consistently consuming 780.95 RUs.

Considering the throughput configured is only 400 RU/s, I was expecting that some of my requests to be throttled however to my surprise all of my requests were successful. I also monitored the total time taken (both end-to-end and time taken by server) and they were respectively 30 seconds and 3 seconds approximately. The total throughput consumed was 39047.50.

enter image description here

I also checked the monitoring to ensure that the service is not throttling any requests (in case SDK is retrying throttled requests) but there also did not find any throttled requests.

enter image description here

Here's the log for 1st 10 requests (remaining requests are more or less the same):

Overall time taken (end-to-end): 30605 ms
Overall time taken (request): 2733.6059999999993 ms
Overall RUs consumed: 39047.49999999999
===========================================
Batch #    Status Code    Request Charge Time (E2E) (ms)    Time (Request) (ms)
001        OK             780.95         7018               56.262
002        OK             780.95         531                53.618
003        OK             780.95         474                51.65
004        OK             780.95         504                54.668
005        OK             780.95         488                56.367
006        OK             780.95         487                50.501
007        OK             780.95         513                54.459
008        OK             780.95         486                55.089
009        OK             780.95         483                65.06

I am curious to know as to why my requests are not getting throttled?

I read about RU debt (here and here) however the math does not add up. If I consider 30 seconds as the total time period, then RUs available to me is 12000 (400 RU/s x 30 seconds) and if I consider 3 seconds as the total time period, then RUs available to me is 1200 (400 RU/s x 3sec). Both of these numbers are way less than 39000 RUs consumed by my application and my requests should have been throttled but none of the requests got throttled.

Any insights into this will be highly appreciated.

1
azureazure-cosmosdbazure-cosmosdb-sqlapi
The SDK does some level of retries automatically, which you can check in the cloud if you look at the "Total Requests" metric and split by status code. However, this would still not explain how you get 39000RUs in 30s. You can check the "provisioned throughput" metric to make sure you only had 400 provisioned throughout, and no autoscaling happened. – Alex AIT
Alex - I did check that and there were no throttling errors. All requests succeeded with 200 status code. Also double checked the provisioned throughput. It is indeed manual 400 RU/s. Thanks. – Gaurav Mantri

1 Answers

0
votes

That's not how free tier works. It simply makes the first 400 RU/s free for use.

This is fairly common misunderstanding. We are working on a throughput cap feature that would rate limit free tier and other accounts. For now you will need to monitor.