Using a BluetoothSocket in ZeroMQ/Tornado ioloop, or maybe in Twisted?

Question

I'm using PyBluez to connect to a device through bluetooth. It is working just fine. The device is using SPP and I connect using a BluetoothSocket (RFCOMM).

Basic interaction is: Send init command -> receive confirmation; send start command -> receive continuous data; send stop command -> ...

I need to be able to control the application from another app: starting, stopping ... I was thinking ZeroMQ or maybe Tornado and a HTTP/REST ifc.

I'm not very keen on making this a multi-threaded app, as I believe it to be a bit over-kill. I've worked a lot with threads and pools of them in C#/.NET, but I have a hunch it is not really necessary here. On the contrary, I think it will be quite messy.

However, I need to be able to process commands (e.g. "start", "stop") via ZeroMQ/REST/ ... while continosly receiving data and sending the occasional packet.

As I am very new to Python I am not sure how to implement this. I have a few ideas: Can the BluetoothSocket be hooked up to the ZeroMQ/Tornado IOLoop?

I assume I can accomplish almost anything using Twisted, but I don't really need everything that Twisted provides. If I need to pull in Twisted, I'll do it. I've actually found a Twisted implementation of BluetoothSocket. But again, do I need Twisted?

I've tried using the Tornado IOLoop. No exceptions are thrown, but on the other hand no data is being received or sent:

def eventhandler(s, events, error = None):
    if events & ioloop.IOLoop.READ:
        print 'Socket read: %r' % s.recv(1024)
    elif events & ioloop.IOLoop.ERROR:
        print 'Socket error!'

events = ioloop.IOLoop.READ | ioloop.IOLoop.ERROR
self._loop.add_handler(self._socket.fileno(), eventhandler, events)

I don't really know what I'm doing right now. I'll find a way eventually, but need a hint on which direction to go.

Extensive Google-fu hasn't turned up much useful information, so I'm asking here now.

Edit: I'm currently looking at "gevent". Seems simpler than Twisted at least.

Why do you think threads or Twisted are "overkill"? What is "overkill" in software? You're not going to grind your problem into a dust so fine that it blows away in the wind leaving you without a job. You're not going to use up all of the Twisteds or threads in the world by applying them to an "easy" problem. And why do you think this is an easy problem, anyway? What makes it any easier than any other concurrent I/O problem out there? :) Particularly since it sounds like you think you already know how to solve this with Twisted, why spend time trying to find a solution without it? — Jean-Paul Calderone
A fair point! I don't know how to do it in Twisted, but as I pointed out in my question I found an implementation. I'm trying to learn Python here, and I don't know how do this is an elegant way. For sure, I could fire up threads. I could even handle receives in a loop of my own design, polling ZeroMQ for commands from time to time. I don't want to automatically go with a huge library, to later discover that it could have been done with a few lines of code and an ioloop. In addition I'll probably be using ZeroMQ and/or Tornado anyway. Can it be handled there? So much better then. — Micke

Glyph Glyph · Accepted Answer · 2012-11-30T15:05:46

You should just use Twisted. Your real question seems to be about Twisted's size and resource utilization; whether you will pay an unreasonable cost in your application for using it, whether in terms of API complexity, memory, package size on disk, deployment hassles, or conflicts with other libraries you might want to use.

You won't.

API complexity: Twisted's API is straightforward. Contrary to certain FUD, you don't need to learn thousands of APIs to use it effectively: the core of Twisted has a quite lean API, which has several discrete layers and many explicitly documented formal interfaces between each layer. If you know what you want to know (and it seems like you do), with a little guidance you can easily pick it up. Also, as a mature project, there are plenty of people in the community who can help your users come up to speed.
Memory usage: Twisted has always been very careful to manage its run-time dependencies, and to only import what it needs. Loading the reactor won't load the IMAP implementation, so just import what you need and don't worry about it.
Package size: Twisted is two megabytes. Furthermore, if you're using any popular UNIX-y OS (Linux, OS X, FreeBSD), you've probably got an operating system package of Twisted already. On many distributions it's even installed already.
Deployment hassles: if you're going to worry about any problem, this is the one to think about, but the issues related to deployment are really very minor. "pip install Twisted" works fine (right now, as long as you have a C compiler). Twisted works with py2exe, py2app, and Debian packaging to the extent that anything in Python does. Really, the only thing anybody ever has any issues with is the plugin system, and for many applications you can just ignore that.
Library conflicts: You will be totally fine. Twisted goes out of its way to be compatible with every hare-brained idea that any other event loop library ever came up with, whether it's GUIs or event loops. You want to use ZMQ? Go for it. Tornado? Sure, whatever; and the integration goes both ways. Tornado itself supports such integration. Want to use a blocking library from Twisted, but you're on the main thread and you don't want to block? deferToThread has got you covered. Want to use a Twisted function from a blocking library, and you need to block? Okay, just make a blocking call from a thread.

I hope this clears up any misconceptions you might have had about Twisted being "overkill".

Using a BluetoothSocket in ZeroMQ/Tornado ioloop, or maybe in Twisted?

2 Answers