NimbleCSV : Elixir

Question

I'm trying to use NimbleCSV library for a personal project but I'm having some problems...

NimbleCSV.define(MyParser, separator: ",", escape: "\"")

defmodule Siren do
  def parseCSV do
    IO.puts("Let's parse CSV file!")
    File.stream!("name.csv")
  |> MyParser.parse_stream
  |> Stream.map(fn [name, team, position, height, weight, age] ->
    %{name: name, team: team, position: position, height: String.to_integer(height), weight: String.to_integer(weight), age: String.to_integer(age)}
    end)
  |> Enum.map(&IO.puts(&1))
  end
end

Like you can see above I'm using Stream, but when I launch my Mix task it's crashed :

➜  siren mix siren
Compiling 1 file (.ex)
Let's parse CSV file!
** (NimbleCSV.ParseError) unexpected escape character " in " \"Team\", \"Position\", \"Height(inches)\", \"Weight(lbs)\", \"Age\"\n"
    deps/nimble_csv/lib/nimble_csv.ex:427: MyParser.separator/5
    deps/nimble_csv/lib/nimble_csv.ex:360: anonymous fn/4 in MyParser.parse_stream/2
    (elixir 1.10.3) lib/stream.ex:902: Stream.do_transform_user/6
    (elixir 1.10.3) lib/stream.ex:1609: Enumerable.Stream.do_each/4
    (elixir 1.10.3) lib/enum.ex:3383: Enum.map/2
    (mix 1.10.3) lib/mix/task.ex:330: Mix.Task.run_task/3
    (mix 1.10.3) lib/mix/cli.ex:82: Mix.CLI.run_task/2

Here my CSV file:

"Name", "Team", "Position", "Height(inches)", "Weight(lbs)", "Age"
"Adam Donachie", "BAL", "Catcher", 74, 180, 22.99
"Paul Bako", "BAL", "Catcher", 74, 215, 34.69
"Ramon Hernandez", "BAL", "Catcher", 72, 210, 30.78
"Kevin Millar", "BAL", "First Baseman", 72, 210, 35.43
"Chris Gomez", "BAL", "First Baseman", 73, 188, 35.71
"Brian Roberts", "BAL", "Second Baseman", 69, 176, 29.39
"Miguel Tejada", "BAL", "Shortstop", 69, 209, 30.77
"Melvin Mora", "BAL", "Third Baseman", 71, 200, 35.07
"Aubrey Huff", "BAL", "Third Baseman", 76, 231, 30.19
"Adam Stern", "BAL", "Outfielder", 71, 180, 27.05
"Jeff Fiorentino", "BAL", "Outfielder", 73, 188, 23.88
"Freddie Bynum", "BAL", "Outfielder", 73, 180, 26.96
"Nick Markakis", "BAL", "Outfielder", 74, 185, 23.29
"Brandon Fahey", "BAL", "Outfielder", 74, 160, 26.11
"Corey Patterson", "BAL", "Outfielder", 69, 180, 27.55

The problem must come from my escape character that I defined before but I don't understand why ? What is an escape character here ? For me is the double quotes surrounding each string in CSV's rows.

Aleksei Matiushkin Aleksei Matiushkin · Accepted Answer · 2020-07-04T17:13:29

CSV states for Comma-Separated Values and it’s the format that has its own RFC4180. One cannot put spaces whenever they want to. Change the input to the one shown below and everything would work fine. The issue is spaces after commas, or, put it in other words, escape character not immediately follow the delimiter.

"Name","Team","Position","Height(inches)","Weight(lbs)","Age"
"Adam Donachie","BAL","Catcher",74,180,22.99
"Paul Bako","BAL","Catcher",74,215,34.69
"Ramon Hernandez","BAL","Catcher",72,210,30.78
"Kevin Millar","BAL","First Baseman",72,210,35.43
"Chris Gomez","BAL","First Baseman",73,188,35.71
"Brian Roberts","BAL","Second Baseman",69,176,29.39
"Miguel Tejada","BAL","Shortstop",69,209,30.77
"Melvin Mora","BAL","Third Baseman",71,200,35.07
"Aubrey Huff","BAL","Third Baseman",76,231,30.19
"Adam Stern","BAL","Outfielder",71,180,27.05
"Jeff Fiorentino","BAL","Outfielder",73,188,23.88
"Freddie Bynum","BAL","Outfielder",73,180,26.96
"Nick Markakis","BAL","Outfielder",74,185,23.29
"Brandon Fahey","BAL","Outfielder",74,160,26.11
"Corey Patterson","BAL","Outfielder",69,180,27.55

NimbleCSV comes with a default implementation, NimbleCSV.RFC4180 and it is exactly what you use, so you don’t need yo define your own parser, use the default one.

defmodule Siren do
  def parseCSV do
    IO.puts("Let's parse CSV file!")

    File.stream!("name.csv")
    |> NimbleCSV.RFC4180.parse_stream()
    |> Stream.map(fn [name, team, position, height, weight, age] ->
      %{name: name, team: team, position: position,
        height: String.to_integer(height),
        weight: String.to_integer(weight),
        age: String.to_float(age) # NOTE float here!
      }
    end)
    |> Enum.to_list()
    |> IO.inspect()
  end
end
#⇒ [
#  %{
#    age: 22.99,
#    height: 74,
#    name: "Adam Donachie",
#    position: "Catcher",
#    team: "BAL",
#    weight: 180
#  },
#  ...
# ]

NimbleCSV : Elixir

1 Answers