Ocaml lwt read stdout from other process

Question

I'm trying to build a new frontend in Ocaml for a terminal based application. The main idea is the spawn a new process with Lwt:

let cmd = shell "./otherterminalapp" in
let p = open_process_full cmd;

And then later write stuff to the process' stdin, to execute commands in the external app.

 Lwt_io.write_line p#stdin "some command" >>= (fun _ -> Lwt_io.flush p#stdin)

When I read the result from the command back in with Lwt_io.read_line_opt. How do I read till there aren't any lines left? The problem I'm encountering is that my program just hangs at a certain point. When I read with read_line_opt, while I reached the end it seems like it's just waiting for the process to redirect new output.

How can I approach this?

A concrete example of what I'm trying to do: (The terminal based application is ocamldebug)

Program source code:

open Lwt
open Lwt_unix
open Lwt_process
let () =
  let run () =
    let cmd = shell "ocamldebug test.d.byte" in
    let dbgr = open_process_full cmd in
    (((((((Lwt_io.write_line dbgr#stdin "info modules") >>=
            (fun _  -> Lwt_io.flush dbgr#stdin))
           >>= (fun _  -> Lwt_io.read_line_opt dbgr#stdout))
          >>=
          (fun s  ->
             (match s with
              | Some l -> print_endline l
              | None  -> print_endline "nothing here! ");
             Lwt_io.read_line_opt dbgr#stdout))
         >>=
         (fun s  ->
            (match s with
             | Some l -> print_endline l
             | None  -> print_endline "nothing here! ");
            Lwt_io.read_line_opt dbgr#stdout))
        >>=
        (fun s  ->
           (match s with
            | Some l -> print_endline l
            | None  -> print_endline "nothing here! ");
           Lwt_io.read_line_opt dbgr#stdout))
       >>=
       (fun s  ->
          (match s with
           | Some l -> print_endline l
           | None  -> print_endline "nothing here! ");
          Lwt_io.read_line_opt dbgr#stdout))
      >>=
      (fun s  ->
         (match s with
          | Some l -> print_endline l
          | None  -> print_endline "nothing here! ");
         Lwt.return ()) in
  Lwt_main.run (run ())

If you would normally run ocamldebug with test.d.byte, you get the following in your terminal:

    OCaml Debugger version 4.03.0

(ocd) info modules
Loading program... done.
Used modules: 
Std_exit Test Pervasives CamlinternalFormatBasics
(ocd)

When I execute the above program, I get the following printed:

    OCaml Debugger version 4.03.0

(ocd) Loading program... Used modules: 
Std_exit Test Pervasives CamlinternalFormatBasics

And here it just hangs..., my program doesn't exit. Even when I do Ctrl-c/Ctrl-c in my terminal, there's an active ocamlrun process. The terminal however becomes responsive though.

I am missing something obvious here?

Can you show a more complete example, including how you read, and a better description of what the other process is actually doing, or, even better, a minimal source code for the other process? What is the "certain point?" Are you able to read with read_line_opt for a while, then it doesn't "work" anymore? In what sense? When you say the "end," are you referring to the end of one line, the end of the output from the other process, or something else? Does the other process actually exit or close its output stream? — antron
I updated the question with a complete example of what I'm trying to accomplish. — Seneca

ivg ivg · Accepted Answer · 2016-12-12T13:14:59

A call to Lwt.read_line_opt returns a deferred value, that will be determined in the future as Some data once the channel reads a newline-terminated string, or with None if the channel was closed. The channel will be closed if there was an end-of-file condition. For the regular files, the end-of-file condition occurs when the file pointer reaches the end of the file. For the pipes, that are used to communicate with the subprocess, the end-of-file condition occurs when the opposite side closes the file descriptor associated with the pipe.

The ocamldebug program doesn't close its inputs or outputs. It is an interactive program, that is ready to interact with a user for the infinite amount of time, or until a user closes the program, by either hitting Ctrl-D or using the quit command.

In your scenario, you wrote the info modules command into the channel's input. The process responded with the three lines (where each line is a piece of data terminated with the newline). Then the subprocess started to wait for the next input. You're not seeing the (ocd) prompt, because it is not terminated by the newline character. The program didn't hang-up. It is still waiting for the output from the subprocess, and the subprocess is waiting for the input from you (a dead lock).

If you really need to distinguish outputs from different commands, then you need to track the prompt in the subprocess output. Since the prompt is not terminated by the newline, you can't rely on the read_line* family of functions, since they are line buffered. You need to read all available characters and find the prompt in them manually.

On the other hand, if you do not really need to distinguish between the outputs of different commands, then you can ignore the prompt (actually, you may even filter it out, for the nicer output). In that case, you will have two concurrent subroutines - one would be responsible for feeding input, and another will read all the output, and dump it, without actually carrying about the contents of the data.

Ocaml lwt read stdout from other process

1 Answers