How to encode a URI to safely store in the RowKey of a TableServiceEntity?

Question

I would really, really like to store a URI as the RowKey value in Azure Table storage. According to the documentation, RowKeys can't contain characters commonly found in URIs (/, \, #, ?).

The solution seems simple: Just encode the URI. But that doesn't work. For whatever reason, any values containing the sequence %2f (the encoded value for a forward-slash) can be inserted, but not queried even though '%2f' does not contain any forbidden characters.

Okay, so how about base64 encoding? Nope. It produces the occasional forward-slash character, which is not allowed.

So is there a method of encoding a string (a URI) that can be reliably stored as the RowKey in Azure Table? Preferably, but not necessarily, something human-readable.

Answer 1

Replacing / with _ after using base64 encoding should work. Got this suggestion from Encoding and decoding a string that may have slashes in it

Answer 2

A couple possiblities:

1. Convert the bytes using some specific encoding (eg, UTF-8), and then encode the bytes as hex. Use that as the key. However, some URIs can be very long, and the key will be even longer, so you might run into the max length of a RowKey.
2. Hash the URI (eg, with MD5). Hex encode those bytes and use them as the key. Note that there is a chance of collisions, but your PartitionKey may remove that chance. Since hashes are constant length you shouldn't have problems with the maximum RowKey length.

I'm sure there are other ideas, I'll be interested to hear them.

Answer 3

Off topic but related: I'm creating tables dynamically and have code that may help you if you need it, though the rules for tables and PK/RK are quite different. "^[A-Za-z][A-Za-z0-9]{2,62}$"."

Perhaps you can use this to inspire your own solution

Decode a string

        string edit1 = host
            .Replace("qqu", "_")
            .Replace("qqh", "-")
            .Replace("qqp", ".")

            // NOTE: qqn is reserved leading sequence

            .Replace("qqt", "qqu")
            .Replace("qqo", "qqp")
            .Replace("qqg", "qqh")
            ;

        if (edit1.StartsWith("qqn"))
        {
            edit1 = edit1.Substring(3, edit1.Length);
        }
        if (edit1.StartsWith("qq"))
        {
            edit1 = edit1.Substring(2, edit1.Length);
        }

Method to Encode a string

            string edit1 = this.originalName.ToLower().Trim()
                .Replace("qqu", "qqt")
                .Replace("qqp", "qqo")
                .Replace("qqh", "qqg")

                // NOTE: qqn is reserved leading sequence

                .Replace("_", "qqu")
                .Replace("-", "qqh")
                .Replace(".", "qqp");

            string test = "qq";
            if (edit1.StartsWith(test))
                return test + "n" + edit1;

            test = "0";
            if (edit1.StartsWith(test))
                return "qq" + edit1;

            test = "1";
            if (edit1.StartsWith(test))
                return "qq" + edit1;

            test = "2";
            if (edit1.StartsWith(test))
                return "qq" + edit1;

            test = "3";
            if (edit1.StartsWith(test))
                return "qq" + edit1;

            test = "4";
            if (edit1.StartsWith(test))
                return "qq" + edit1;

            test = "5";
            if (edit1.StartsWith(test))
                return "qq" + edit1;

            test = "6";
            if (edit1.StartsWith(test))
                return "qq" + edit1;

            test = "7";
            if (edit1.StartsWith(test))
                return "qq" + edit1;

            test = "8";
            if (edit1.StartsWith(test))
                return "qq" + edit1;

            test = "9";
            if (edit1.StartsWith(test))
                return "qq" + edit1;

            test = "0";
            if (edit1.StartsWith(test))
                return "qq" + edit1;

Answer 4

Replace all the illegal row key characters in the URL (\, /, ?) with characters that are illegal in URLS (eg <, >, %)

Answer 5

public static string Encode(string rawText)
    {
        return Regex.Replace(rawText, @"[/\?\:@\&=\+,\$]", delegate(Match m)
        {
            switch (m.Value)
            {
                case "/": return "{#sl}";
                case "?": return "{#qm}";
                case ":": return "{#cl}";
                case "@": return "{#at}";
                case "&": return "{#am}";
                case "=": return "{#eq}";
                case "+": return "{#pl}";
                case ",": return "{#cm}";
                case "$": return "{#dl}";

                default: return m.Value;
            }
        });
    }

public static string Decode(string encodedText)
    {
        return Regex.Replace(encodedText, @"\{#[a-z]{2}\}", delegate(Match m)
        {
            switch (m.Value)
            {
                case "{#sl}": return "/";
                case "{#qm}": return "?";
                case "{#cl}": return ":";
                case "{#at}": return "@";
                case "{#am}": return "&";
                case "{#eq}": return "=";
                case "{#pl}": return "+";
                case "{#cm}": return ",";
                case "{#dl}": return "$";

                default: return m.Value;
            }
        });
    }