Azure table Storage continuation

Question

So, Microsoft decided to send diagnostic data to Azure table storage. I'm trying to query this storage and send it to another location for analytics via C# SDK. I can query just fine pull the hundreds of thousand of record, but it appears that the last continuation token they send will always receive a null response. Even if more data gets sent into table storage, my continuation token doesn't work, still gets a null continuation token and null data back.

Has anyone done anything like this? How can I continue "syncing" azure table data if the continuation tokens they send are broken?

           public static List<PerfMonEntity> GetEventData(ref TableContinuationToken contToken)
    {
        CloudStorageAccount storageAccount = CloudStorageAccount.Parse(ConfigurationManager.AppSettings["StorageConnectionString"]);
        CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
        CloudTable eventLogsTable = tableClient.GetTableReference("WADPerformanceCountersTable");
        TableQuery<PerfMonEntity> query = new TableQuery<PerfMonEntity>();

        var l = new List<PerfMonEntity>();
        var segment = eventLogsTable.ExecuteQuerySegmented(query, contToken ?? new TableContinuationToken());
        foreach (PerfMonEntity wadCounter in segment)
        {
            l.Add(wadCounter);
        }
        contToken = segment.ContinuationToken;

        if (contToken == null)
        {
            Console.WriteLine("contToken is NULL!");
            return null;
        }

        Console.WriteLine("partkey: {0}", contToken.NextPartitionKey ?? "");
        Console.WriteLine("rowkey: {0}", contToken.NextRowKey ?? "");

        return l;
    }

-=-=-=-=-=-

        while (num < loop)
        {
            List<PerfMonEntity> eleList = AzurePerfTable.GetEventData(ref contToken);
            if (eleList != null)
                returnedList.AddRange(eleList);
            else
                num = loop;
            num += 1;

            if (contToken != null)
                AZContinuationToken.SetContToken(contToken);

            Console.WriteLine("returnedlistsize: {0}", returnedList.Count<PerfMonEntity>());
        }

Tamra Myers - Microsoft Tamra Myers - Microsoft · Accepted Answer · 2015-03-10T00:24:24

The continuation token is null when there is no more data to return. When it's non-null, it means that there are additional entities to return in the next page. You can check for null to determine when you've retrieved the last page and then exit the loop.

Try writing your logic along these lines:

CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));

CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

CloudTable eventLogsTable = tableClient.GetTableReference("WADPerformanceCountersTable");
TableQuery query = new TableQuery();

Console.WriteLine("List perf counter results in pages:");

TableContinuationToken token = null;

do
{
    var segment = eventLogsTable.ExecuteQuerySegmented(query, token, null, null);

    foreach (var wadCounter in segment)
    {
        Console.WriteLine(wadCounter.PartitionKey);
        Console.WriteLine(wadCounter.RowKey);
        Console.WriteLine(wadCounter.Timestamp);
    }

    token = segment.ContinuationToken;
}
while (token != null);

Azure table Storage continuation

1 Answers