Connected clients list in Python Tornado

Question

I have a Tornado WebSocket Server running in a separate process that is launched by a thread. This thread calls the publish method of my TornadoServer when it gets messages to send via websockets.

Running Tornado on a separate process was the only way I found to start the tornado loop without the thread blocking on this call.

In my thread, I start the tornado process by calling these methods on thread init method:

self.p = tornado_server.TornadoServer()
self.p.daemon = True
self.p.start()

In this thread, I have an infinite loop that tries to get messages from a Queue and if it gets messages, it calls the self.p.publish(client, message).

So far, so good.

On the Tornado process, I basically implemented a publish/subscribe system. When a user opens a webpage, the page sends a "subscription" message for a specific "client" let's say. On the "on_message" callback I append a tuple of the WebSocketHandler instance and the client that the user wants to subscribe to a global list.

Then, the publish method should search in the list for subscribed users to the message's target client and it should call the write_message on the WebSocket stored on that list.

The only thing that it isn't working is that my "clients" list have different scopes or something.

This is the code of my tornado_server file:

#!/usr/bin/python2
import tornado.web, tornado.websocket, tornado.ioloop, multiprocessing

clients = []

class TornadoServer(multiprocessing.Process):

    class WebSocketHandler(tornado.websocket.WebSocketHandler):
        def on_message(self, message):
            global clients
            print 'TORNADO - Received message:', str(message)
            channel, subtopic = message.split('/')
            print 'TORNADO - Subscribing:', str(subtopic)
            clients.append((self, subtopic))

        def on_close(self):
            global clients
            for websocket, client in clients:
                if self == websocket:
                    print 'TORNADO - Removed client'
                    to_remove = (self, client)
                    clients.remove(to_remove)

    def __init__(self):
        multiprocessing.Process.__init__(self)
        self.application = tornado.web.Application([(r"/tri-anim", WebSocketHandler)])
        self.application.listen(1339)

    def run(self):
        tornado.ioloop.IOLoop.current().start()

    def publish(self, client, message):
        global clients
        for websocket, websocketclient in clients:
            if websocketclient == client:
                websocket.write_message(str(message))

No matter what I do, clients have always different scopes. When publish is called, the "clients" is always empty. Is there any way to get this working?

Ben Darnell Ben Darnell · Accepted Answer · 2013-12-03T17:10:56

You're calling publish in the parent process, but the clients list is only updated in the child process. When using multiprocessing each process gets its own copy of all the variables. If you used threads instead the variables would be shared, but even then you'd need to use IOLoop.instance().add_callback to do a thread-safe handoff between the thread calling publish and the write_message function (which must be called on the IOLoop thread).

Connected clients list in Python Tornado

1 Answers