FireDac TFDBatchMoveTextReader not bringing in the full field value

Question

I have a CSV file I would like to import with the FireDAC BatchMove family of components (TextReader and DataSetWriter).

Sample CSV Data

Vehicle,State,Toll distance (mi),Distance (mi),Time (hours)
Tr226,VA,0.0,8679.9,142.5
Tr114,VA,0.0,7227.2,151.5

Sample Code

  {FDBatchMove, FDBatchMoveTextReader, FDBatchMoveDataSetWriter, and FDMemTable are declared in the Private declarations of my form}
  FDBatchMove := TFDBatchMove.Create(nil);
  FDBatchMoveTextReader := TFDBatchMoveTextReader.Create(nil);
  FDBatchMoveDataSetWriter := TFDBatchMoveDataSetWriter.Create(nil);

  FDMemTable := TFDMemTable.Create(nil);

  FDBatchMoveTextReader.FileName := 'Y:\Shared\VehicleShort.csv';
  FDBatchMoveDataSetWriter.DataSet := FDMemTable;

  FDBatchMove.Reader := FDBatchMoveTextReader;
  FDBatchMove.Writer := FDBatchMoveDataSetWriter;

  FDBatchMove.Analyze := [taDelimSep, taHeader, taFields];

  FDBatchMove.AnalyzeSample := 10;

  FDBatchMove.Execute;

After executing the above block, I loop over the FDMemTable and display the contents of the first field:

  FDMemTable.First;
  while not FDMemTable.Eof do
  begin
    ShowMessage(FDMemTable.Fields[0].AsString);
    FDMemTable.Next;
  end;

And I get the following answers. Notice the last character is cut off:

Tr22
Tr11

If I modify the CSV data to include longer vehicle names like Truck226 and Truck114 I get the same problem with the last character cut off.

Truck22
Truck11

I'm a bit at a loss for what can be done here. I need to create the BatchMove components at runtime, and the demos that came with Delphi don't go into enough depth for me to solve the problem on my own.

Interesting. I decided to create a brand new txt file and copied the data from my SO post and it worked. So that tells me there's some fundamental difference between the files themselves. I tried changing the encoding, but that doesn't seem to work. If I just copy/paste the data (using notepad++) into a new file, it works. — FLDelphi

Branko Branko · Accepted Answer · 2019-10-24T04:36:47

String fields in csv are not enclosed with delimiter (double or single quote) so you must set FDBatchMoveTextReader.DataDef.Delimiter := #0;

FireDac TFDBatchMoveTextReader not bringing in the full field value

2 Answers