Azure Table Storage Exception Information

Question

Finding debugging Azure table storage incredibly frustrating. Until I learn about the various limitations, trying to find the cause of incredibly informative DataServiceQueryException with a code of InvalidInput is difficult to say the least.

After much scouring the web, the post at http://blogs.msdn.com/b/partlycloudy/archive/2009/12/16/development-storage-logging.aspx shows how to turn on logging. This populates the error log file with much more, very useful information. It is probably not a good idea to leave this logging on permanently, and inspecting this file every time I have a problem is not an ideal situation.

There are many posts about using Fiddler to look at the actual requests and responses, but I'm unable to get this working properly. I've configured the connection string to connect through the Fiddler proxy (I had to manually add the ipv4.fiddler host name to the hosts file, otherwise it wouldn't resolve - something Fiddler should do automatically). I can see the connections for deployment, but not connections for any queries. I've tried running Fiddler as administrator, but still get the same results.

Why is this so hard? Am I missing something? Is there an option to return the exception information that is logged in the error log file, rather than the rubbish InvalidInput exception messages? Any ideas why Fiddler is not playing the game?

Answer 1

Typically, you will get the invalid inputs error messages if you are trying to use an operation that is not supported at the service. Keep in mind that the full set of LINQ operations (sorting, min, max, etc.) are not available at the service (though, they can be computed locally). Your first step in troubleshooting this should be to look at the actual operations your are trying. If you are using something other than .Select() or .Where(), there is a good chance it is not currently supported.

Edit: As soon as I wrote this, I ran into a similar error. Since I already know which operations are supported, I thought I would show you the code I used to troubleshoot:

void Main()
{
    var acct = CloudStorageAccount.DevelopmentStorageAccount;

    var client = acct.CreateCloudTableClient();
    var ctx = client.GetDataServiceContext();
    ctx.IgnoreMissingProperties = true;

    var table = "tl36f6e92d94954f168ade0be6a547c0ce";

    //build query
    var q = ctx.CreateQuery<Foo>(table)
        .Where(e => e.RowKey.CompareTo(2) < 0) //this query fails
        .Take(10);

    //Dump URI to inspect
    ((DataServiceQuery)q).RequestUri.Dump();

    //dump results
    q.Dump();
}


[System.Data.Services.Common.DataServiceKey("PartitionKey", "RowKey")]
class Foo
{
    public string PartitionKey { get; set; }
    public string RowKey { get; set; }
    public string Whatever { get; set; }
}

I dropped this into LINQPad (greatest tool ever) and just looked at the URI. It became clear that the comparison I was making was expecting a string and was being compared as an integer. I have of course simplified the actual query, but my point stands. I use this tool all the time to model my queries first and inspect what is actually happening.

Answer 2

If you add "ipv4.fiddler" to your hosts file, it will resolve to localhost prior to hitting Fiddler, and therefore ignore the proxy setting as it is a local address. The answer is not to add "ipv4.fiddler" to hosts. There is a solution to the DNS resolution problem that you have encountered though...

When you start Fiddler, it sets the WinINET proxy for the logged on user to 127.0.0.1:8888. However, the Dev Fabric sets the IIS App Pool to run as NETWORK SERVICE and so this does not know of the WinINET proxy config from Fiddler, and therefore doesn't route HTTP requests through Fiddler. Therefore, we get DNS resolution failure when using the Azure storage connection string: "UseDevelopmentStorage=true;DevelopmentStorageProxyUri=http://ipv4.fiddler".

So try running this from the VS command prompt:

bitsadmin /util /SETIEPROXY NETWORKSERVICE MANUAL_PROXY 127.0.0.1:8888 NULL

so that NETWORK SERVICE now uses Fiddler as a proxy, resolves ipv4.fiddler to localhost, and traces all the HTTP traffic to and from the dev storage emulator.

Answer 3

I just got this error and wanted to throw my 2 cents in, hopefully saving someone else a few minutes... I had a nullable Guid in my POCO and thus far, none of the records had filled that value w/ anything and so when I called:

_repository.Find(f => f.PartitionKey == request.CompanyPartitionKey() && f.MyNullGuid == null);

this threw the same error because until an instance comes through w/ a non-nullable value, table storage didn't put this column in at all... So I was getting "Invalid Input" through nothing that I had done, but because Table Storage had never put my nullable column in until a record warranted doing so.

The fix: I changed my null guid to non-null and changed my check to Guid.Empty... Ugh...

Answer 4

I have written an extension class to Azure StorageException that extracts all of the useful information embedded inside it by the Azure Storage Service (Table, Blob, Queue, ...)

https://www.nuget.org/packages/AzureStorageExceptionParser/

Usage:

Try

{

//Make a request to Azure Table Storage (Blob, Table, Queue, ..)

}

catch (StorageException storageException)

{

//GetHttpStatusCode extension method gives you the HttpStatusCode embedded inside the StorageException

int? httpStatusCode = storageException.GetHttpStatusCode();

// GetFailedOperationIndex extension method gives you the index of the failed operation in an Azure Table Batch operation

int failedOperationIndex = storageException.GetFailedOperationIndex();

}