OCaml: Issue manipulating string read from file

Question

I am trying to read a file, line by line in OCaml. Each line in the file represents a string I want to parse, in the correct format expected by the Parsing tool. I am saving each line in a list structure.

I an finding an issue parsing the string contained in each element of the list. I am using OCamllex and Menhir as parsing tools.

• If I try to use print_string to print the contents of the list at every element, I get the correct file contents.

• If I try to pass a string that I defined within the program to the function, then I get the desired output.

• However, if I try to parse the string which I have just read from the file, I get an error: Fatal error: exception Failure ("lexing empty token")

Note: that all of this has been tested against the same string.

Here is a snippet of the code:

let parse_mon m = Parser.monitor Lexer.token (from_string m)

let parse_and_print (mon: string)=
  print_endline (print_monitor (parse_mon mon) 0)

let get_file_contents file =
  let m_list = ref [] in
    let read_contents = open_in file in
      try
        while true; do
          m_list := input_line read_contents :: !m_list
        done; !m_list
      with End_of_file -> close_in read_contents; List.rev !m_list

let rec print_file_contents cont_list = match cont_list with
  | [] -> ()
  | m::ms -> parse_and_print m

let pt = print_file_contents (get_file_contents filename)

Answer 1

I agree with kne, hard to say without seeing the file, but what you can do is trying to isolate the line that causes the trouble by doing :

let rec print_file_contents cont_list = match cont_list with
| [] -> ()
| m::ms -> 
  try parse_and_print m
  with Failure _ -> print_string m

Answer 2

Ocamllex throws an exception Failure "lexing: empty token" when a text in the stream doesn't match any scanner pattern. Therefore, you will need to match with "catch-all" patterns such as ., _, or eof.

{ }
rule scan = parse
  | "hello" as w { print_string w; scan lexbuf }
  (* need these two for catch-all *)
  | _ as c       { print_char c; scan lexbuf }
  | eof          { exit 0 }

Answer 3

Without seeing your grammar and file I can only offer a wild guess: Could it be that the file contains an empty line at the end? Depending on the .mll that might result in the error you see. The reason being that get_file appends new lines to the front of the list and print_file_contents only looks at the head of that list.