Getting a partition from Azure Table Storage

Question

In my Table storage, there are 10.000 elements per partition. Now I would like to load a whole partition into memory. However, this is taking very long. I was wondering if I am doing something wrong, or if there is a way to do this faster. Here is my code:

        public List<T> GetPartition<T>(string partitionKey) where T : TableServiceEntity
        {
            CloudTableQuery<T> partitionQuery = (from e in _context.CreateQuery<T>(TableName)
                                                 where e.PartitionKey == partitionKey
                                                 select e).AsTableServiceQuery<T>();

            return partitionQuery.ToList();
        }

Is this the way it is supposed to be done or is their anything equivalent to the batch insertion for getting elements out of the table again?

Thanks a lot, Christian

EDIT

We have all the data also available in blob storage. That means, one partition is serialized completely as byte[] and saved in a blob. When I retrieve that from blob storage and afterwards deserialize it, it is way faster than taking it from the table. Almost 10 times faster! How can this be?

Sandrino Di Mattia Sandrino Di Mattia · Accepted Answer · 2012-11-05T09:26:38

In your case I think turning off change tracking could make a difference:

context.MergeOption = MergeOption.NoTracking;

Take a look on MSDN for other possible improvements: .NET and ADO.NET Data Service Performance Tips for Windows Azure Tables

Edit: To answer your question why a big file in blob storage is faster, you have to know that the max amount of records you can get in a single request is 1000 items. This means, to fetch 10.000 items you'll need to do 10 requests instead of 1 single request on blob storage. Also, when working with blob storage you don't go through WCF Data Services which can also have a big impact.

Getting a partition from Azure Table Storage

2 Answers