OCaml: new_line equivalent before 3.11

Question

I'm trying to compile Libra toolkit on a machine running Ubuntu Hardy with OCaml 3.10, I can't upgrade the OS nor update OCaml, and I don't know anything about OCaml. There is only one line that gives me an unbound value error because it uses the new_line function, which was introduced in OCaml 3.11 (http://caml.inria.fr/pub/docs/manual-ocaml/libref/Lexing.html), could someone tell me how to change it to be compatible with OCaml 3.10? It's the line near the end of this code:

{
open MnParseTypes;;
open MnParser;;
(* Raised when parsing ends *)
exception Eof;;

module L = Lexing
let linenum lexbuf = lexbuf.L.lex_curr_p.L.pos_lnum

let line = ref 1;;

let keywords = Hashtbl.create 10
let _ = 
  List.iter2 (Hashtbl.add keywords)
    ["mn"; "features"; "tree"; "table"; "w"; "eof"]
    [Tmn; Tfeatures; Ttree; Ttable; Tweight; EOF];;
}

let digits = ['0'-'9']+
let identifier = ['a'-'z' 'A'-'Z']+

rule lexer = parse
(* eat blank characters *)
    [' ' '\t'] {lexer lexbuf}
(* | "Feature list:" {lexer lexbuf} *)
  | '{' {Tlbrace}
  | '}' {Trbrace}
  | '(' {Tlparen}
  | ')' {Trparen}
  | ('-')? "inf" {Tfloat( float_of_string(L.lexeme lexbuf))}
  | identifier {
      let x = String.lowercase (Lexing.lexeme lexbuf) in
      try Hashtbl.find keywords x
      with Not_found -> 
        failwith((Lexing.lexeme lexbuf) 
          ^ ": unknown identifier on line " ^ string_of_int (linenum lexbuf))}
  | digits {Tint (int_of_string (L.lexeme lexbuf))}
  | ('-')? digits ('.' digits)? (['e' 'E'] ['+' '-']? digits)? 
      {Tfloat( float_of_string(L.lexeme lexbuf))}
  | '+' 'v' (digits as var) '_' (digits as value) 
      {Tcond(true, int_of_string var, int_of_string value)}
  | '-' 'v' (digits as var) '_' (digits as value) 
      {Tcond(false, int_of_string var, int_of_string value)}
  | 'v' (digits as var) '_' (digits as value) 
      {Tvar( int_of_string var, int_of_string value)}
  | ['\n' '\r']+ {L.new_line lexbuf; TEOL}   (* THIS GIVES THE ERROR *)
  | eof {EOF}
  | _ {failwith((L.lexeme lexbuf) ^ 
       ": mistake on line " ^ string_of_int lexbuf.L.lex_curr_p.L.pos_lnum)}

This will help, it's actually the function that does what I want, but I don't know how to insert it in my code: caml.inria.fr/mantis/view.php?id=3930 — user2059990
I tried using this: (* let start_next_line lexbuf = () *) let start_next_line lexbuf = let lcp = lexbuf.lex_curr_p in lexbuf.lex_curr_p <- { lcp with pos_lnum = lcp.pos_lnum + 1; pos_bol = lcp.pos_cnum;} } and then | ['\n' '\r']+ {start_next_line lexbuf; TEOL} but I get the error Fatal error: exception Data.Eof sometimes — user2059990
you really should include these in your question body if it's relevant. — didierc
That Data.Eof issue seems unrelated to your problem: apparently your code seems correct (it does compile, and is the same as the one in lexing), so I would try to see if the data you feed it with is correct, or report to the maintainer. Note that it's not production ready code. — didierc
The data is correct and it works on Libra 0.5, but it causes that error with the change in gasche's answer. In fact, Libra has another file that works in this way. IDK if this is worth reporting, I'm trying to make the source compatible with an older version of OCaml, it's not useful unless someone is in the exact same situation as me. The source is in Libra 0.5.0, and there are two files like this: src/mn/mnLexer.mll (line 46) and src/bn/cnLexer.mll (line 48), and the one which includes the function is src/bn/bifLexer.mll (line 27). — user2059990

gasche gasche · Accepted Answer · 2013-05-12T07:29:37

In the directory of the OCaml sources (from SVN or a relase tarball), the source of the module Foo of the standard library will be in stdlib/foo.{ml,mli} (.mli is the interface file, .ml the implementation file). Looking at stdlib/lexing.ml gives you:

let new_line lexbuf =
  let lcp = lexbuf.lex_curr_p in
  lexbuf.lex_curr_p <- { lcp with
    pos_lnum = lcp.pos_lnum + 1;
    pos_bol = lcp.pos_cnum;
  }

You can implement this in your code as well, using open Lexing to have the field names in scope, or using lexbuf.Lexing.lex_curr_p, and { lcp with Lexing.pos_lnum = lcp.Lexing.pos_lnum ... instead.

Edit: as you probably don't plan to hack the OCaml code yourself, let's give you the full thing:

let new_line lexbuf =
  let lcp = lexbuf.Lexing.lex_curr_p in
  lexbuf.Lexing.lex_curr_p <- { lcp with
    Lexing.pos_lnum = lcp.Lexing.pos_lnum + 1;
    Lexing.pos_bol = lcp.Lexing.pos_cnum;
  }

add this at the top of the file that uses new_line (if it says Lexing.new_line, turn it into new_line), and you should be fine.

OCaml: new_line equivalent before 3.11

2 Answers