Reading multiple classes from single csv file using CsvHelper

Question

I've been using Josh Close' CsvHelper a bit recently to parse CSV files, I quite like the fluent api for class mapping.

I'm trying to map a csv file which contains multiple record types, the file structure is

C,Comment,Timestamp
I,Class1,Header1,Header2
D,Class1,Data1,Data2
D,Class1,Data1,Data2
...
I,Class2,Header1,Header2,Header3
D,Class2,Data1,Data2,Data3
D,Class2,Data1,Data2,Data3
...
C,Checksum

Is this something which can be handled by CsvHelper? I've writen a custom parser which basically works but all it really does is filter out the Header and Data fields for a specific class - I'd really like to be able to do something like

csv.Configuration.RegisterClassMap<Class1>();
csv.Configuration.RegisterClassMap<Class2>();

var data1 = csv.GetRecords<Class1>().ToList();             
var data2 = csv.GetRecords<Class2>().ToList();

And read the file in one pass? Is this possible or am I using the wrong parser?

Regards Dave

I've been parsing text files like this for 40 years. If I can't parse the file nobody can. I can do this lots of ways with a single pass. You have to use a method that doesn't use a header row since you have multiple sections with different headers and each section may contain a different number of columns. You need to use a GroupBy that sorts the data on the first two columns (Type [C,I,D] and Class). — jdweng
That doesn't really answer the question though, can CsvHelper parse this file? — David Waterworth
You may not need help on this anymore, but how do you know which record should map to which class? For example, are all rows distinct lengths, is there a discriminator column, etc? — drneel
Hi @drneel, yes there's a discriminator column. The 2nd column is the class type. The first is the row type (I is headers, D is data) — David Waterworth

drneel drneel · Accepted Answer · 2016-02-18T13:52:14

There is a way to do this; you just have to do it manually.

You manually read the csv file row by row
Inspect the first column for the discriminator that will indicate that you need to map to a Class object.
Inspect the second column for the class to map to.

Map the entire row to that given class.

public static void ReadMultiClassCsv()
{
    var class1Data = new List<Class1>();
    var class2Data = new List<Class2>();

    using (StreamReader reader = File.OpenText(@"C:\filename.csv"))
    using (var csvReader = new CsvReader(reader))
    {
        //1.  You manually read the csv file row by row
        while (csvReader.Read())
        {
            var discriminator = csvReader.GetField<string>(0);

            //2.  Inspect the first column for the discriminator that will indicate that you need to map to a Class object.
            if (discriminator == "D")
            {
                var classType = csvReader.GetField<string>(1);

                //3.  Inspect the second column for the class to map to.
                switch (classType)
                {
                    //4.  Map the entire row to that given class.
                    case "Class1":
                        class1Data.Add(csvReader.GetRecord<Class1>());
                        break;
                    case "Class2":
                        class2Data.Add(csvReader.GetRecord<Class2>());
                        break;
                    default:
                        break;
                }
            }
        }
    }
}

Reading multiple classes from single csv file using CsvHelper

1 Answers