Expensive use of storage account from Azure Functions

13
votes

I'm running a single Azure Function on the consumption plan. I've picked the consumption plan for the serverless feature as well as to minimize cost. The function consumes messages from a service bus topic and writes some output to blob storage.

Keeping the function running for the last 30 days is around $10. That's very acceptable, since the function has a lot of messages to consume. Writing the output to blob storage is around $20. Also acceptable. What I don't understand is, that the charge for the Function's underlying storage account is around $70 for the same period. The consumption is primarily hitting File Write Operation Units and File Protocol Operation Units. The storage account is created as locally redundant general purpose v1.

Anyone able to explain what's going on here? When looking at the storage account, there's a few blobs. I believe the problem is with tables storage. When inspecting the storage account, there are tables looking like this:

$MetricsCapacityBlob $MetricsHourPrimaryTransactionBlob AzureWebJobsHostLogs201804

I've disabled logging in my function, by removing the AzureWebJobsDashboard app setting. After doing so, the AzureWebJobsHostLogs* tables no longer seems to receive new rows. But the $Metrics* tables still receive new data. I have no clue if writes to these tables are causing all of the file write activity I see in the Costs Management view in the Portal, though.

What's going on here? Is maintaining these tables from serverless code really required and does it sound normal that the price for table access is x7 the price of the function itself?

2
azureazure-functions
Please check the account kind for this storage account? If it is General Purpose Storage V2, then the pricing for such accounts is significantly higher than your V1 accounts. – Gaurav Mantri
It's Locally-redundant general purpose v1 storage. – ThomasArdal
Do you mean that you are seeing additional entries getting created in Table Storage? If all you are seeing as the ones that existed before you made the change, then I don't understand the question. – David Ebbo
I didn't make any change. The function has always been running on the consumption plan. I just didn't look at the cost until now. I totally understand that there is a cost related to maintain the storage, but I didn't expect it to be that expensive compared to running the function itself. – ThomasArdal
How many times does your function run per month? – kamil-mrzyglod

2 Answers

3
votes

You should go to Metrics in Azure Portal for this storage account and check the patterns of how the File storage transactions are consumed. If it's consistently high, it's something with your application (e.g. too much logging to file).

In my case, it appears to be a bug in Azure Functions, and I filed a bug here.

The function starts consuming tens of thousands of read and write transactions after any code change, however minor. So basically each code change or deployment costs me perhaps around $0.20, and it could be more in your case.

This is easy to see in the Metrics diagram because it looks like a huge spike in transactions.

So the solution is: don't write logs to the filesystem and don't deploy often.

enter image description here

1
votes

It is interesting and unusual that your storage cost is so much higher. I think the dashboard logging is a likely culprit, so it would be good understand if you see a drop over the next few days with it turned off.

I would spend a bit more time in the cost analysis section of the Azure Portal to see if you can get more details about exactly which aspect of your storage usage is driving the majority of the cost. i.e. is it about table operations, blob operations, etc. This screenshot shows the Cost History view with a breakdown per meter. Note the tooltip in this screenshot:

enter image description here

The $Metrics tables are not written by Azure Functions, they are generated by Azure Storage itself. I would be surprised if these metrics were contributing significantly to your overall cost. But if you want to experiment, I think you can disable them through this UX:

image

To give you a baseline on what sort of ratio of storage costs to functions execution cost is expected, you might want to take a look at the cost write up I did in this blog post: https://blogs.msdn.microsoft.com/appserviceteam/2017/09/19/processing-100000-events-per-second-on-azure-functions/

You'll notice that the storage costs were less than functions, and that includes a significant number of storage operations due to event hubs processing requiring checkpoints written to storage. I'll note that these tests were run with dashboard logging off (again making me suspect that as the main cost driver). So no, it is NOT normal for your storage costs to be 7x your functions cost!