Azure Web Jobs suddenly stopped working properly

Question

I wrote a program named "blobsUploader" that uploads a csv file to a blob container every night at 11pm.

Whenever a new csv file arrives to the blobs container, a new message in a queue named "blobsAdressQueue" appears with the address of the new blob (csv file).

This evokes the Web Job which reads the csv file and stores all it's data in Azure Table named "myDataTable".

The whole process worked great but suddenly from the past month or two, every night when a new csv is uploaded, there is an error with the Web Job process and the message from "blobsAddressQueue" moves to "blobsAddressQueue-poison" which means a message that has exceeded the maximum number of delivery attempts to the application.

I uploaded now a csv from June 2018, that worked for sure. However, now the message with the address the this blob is in "blobsAddressQueue-poison".

When I check the logs I can see 5 failing calls:

When I go into one of the attempts and open "Toggle Output", that is what I get: Which is very weird because this file was read a in June 2018!!! with no any problems! I did not change anything in my code or the csv file since then.

If more information is needed in order to answer the problem, pls let me know.

Is the CsvHelper an external reference, like a NuGet package? And if it is, have you updated this package? Because apparently something changed, otherwise you would be having this issue now. — rickvdbosch
@rickvdbosch yes it is. I am afraid to update this package so I won't make more mess. Also, if everything worked until now with the old package and nothing else was updated, how could it help to update csvHelper ? — jrz
On top of that, I just tried another csv from 2017 (their structure is exactly the same) and it worked well.. @rickvdbosch — jrz
Agree with @rickvdbosch, its your assemblies that changed and likely has nothing to do with the webjobs not working. — ElvisLives
@I.zv I posted an answer weeks ago. Do you have a chance to take a look? If it's helpful, coudl you mark it as an answer so others would see it easily? — axfd

axfd axfd · Accepted Answer · 2019-01-15T10:12:53

This issue is not related to webjob, but CsvHelper library you reference to. I've checked the source code and found a field will be considered as bad data when it contains quote and the field is not quoted (escaped).

Source code:

/// <summary>
/// Gets or sets the function that is called when bad field data is found. A field
/// has bad data if it contains a quote and the field is not quoted (escaped).
/// You can supply your own function to do other things like logging the issue
/// instead of throwing an exception.
/// Arguments: context
/// </summary>
Action<ReadingContext> BadDataFound { get; set; }

The solution is

Modify the problematic fields in csv file

or

Ingnore the bad data by setting BadDataFound to null:

csv.Configuration.BadDataFound = null;

Sample code：

    static void Main(string[] args)
    {
        using (var reader = new StreamReader(@"C:\Users\toml\Desktop\test.csv"))
        using (var csv = new CsvReader(reader))
        {
            csv.Configuration.BadDataFound = null;
            var records = csv.GetRecords<Foo>();

            foreach(var item in records)
            {
                Console.WriteLine(item.Name);
            }
        }

        Console.ReadKey();
    }
}

public class Foo
{
    public int Id { get; set; }
    public string Name { get; set; }
}

Invalid CSV sample:

    Id,Name
    1,one"
    2,two

Valid CSV sample:

    Id,Name
    1,"one""
    2,two

Azure Web Jobs suddenly stopped working properly

1 Answers