python - simple inter-process communication

19
votes

I'm looking for a simple way to pass messages from one process (Perl script, short-lived) to another (Python script, long-running) - both processes local to the same machine. I've done some research, but what I've found was either over my head or seemed unnecessarily complex - leaving me a bit lost and confused.

I imagine a minimal example roughly like the following:

# listener.py

class Listener:
    def __init__(self, port)
        self.port = port

    def on_message(self, msg):
        print "%s: %s" % (timestamp, msg)

recipient = Listener(1234)


# sender.pl

sub send_message {
    my ($msg, $port) = @_;
    # ...
}

send_message("hello world", 1234);

Any pointers on how to solve and/or where to read up on this would be greatly appreciated!

3
pythonperlipc
Why aren't you using the OS pipelines available directly in the shell? python somescript.py | perl otherscript.pl should work nicely. No sockets. – S.Lott
Via a POSIX-MQ style message send or can you solve with sockets? – Jé Queue
The Python script is a long-running process, and piping messages only works on a one-off basis (I think - you can't pipe into existing processes, can you?). To make up a silly example, imagine Listener was a desktop widget displaying incoming messages (kinda like tail -f mylog). – AnC
@S.Lott: I thought I did!? Anyway, I'm currently reading up on sockets and message queues, will accept an answer when I have more of an insight. – AnC
FWIW, I wasn't looking for the optimal solution to a specific issue, but trying to learn about the concepts in general - that's why I kept the OP fairly abstract/generic. The answers have provided me with enough data to get a basic idea of what's what, so I'm content with the outcome, and I'm confident it will prove useful to others in a similar situation. – AnC

3 Answers

3
votes

In general you are interested in sockets. A good place to get just the needed rough information is the documentation of IO::Socket::INET or more basic socket-stuff in perl from perldoc perlipc

20
votes

It turns out that interprocess communication is, while on the surface straightforward, actually fraught with complications. Whatever anyone tells you here in terms of a simplified answer, always keep in mind that there is probably a lot of caveats that are being left unsaid.

Now with that disclaimer out of the way, I claim that what you likely want are message queues. This is based on the fact that you did not include an ip address in your example api. If you need to go across machines, you will want sockets. However, I think you will find message queues to be simpler to understand if you can deal with the fact that this is only for communicating with processes on the same machine.

A good starting point for perl is:
http://perldoc.perl.org/IPC/Msg.html

for python, this seems to explain (ignore the other kinds of ipc like semaphores):
http://semanchuk.com/philip/sysv_ipc/

11
votes

And for powered-up communications in the same style as socket, consider looking at 0MQ. It can make use of different communication technologies depending where your two apps are located, and even for local processes it's very easy to use and solves the problems for you.

http://zeromq.org