AWS IoT: Use MQTT on port 443

Question

I am trying to setup AWS IoT in Pi on port 443 using Paho MQTT .

As AWS document (https://docs.aws.amazon.com/iot/latest/developerguide/protocols.html) mentioned that

Clients wishing to connect using MQTT with X.509 Client Certificate authentication on port 443 must implement the Application Layer Protocol Negotiation (ALPN) TLS extension and pass x-amzn-mqtt-ca as the ProtocolName in the ProtocolNameList.

I actually don't know how to achieve it properly in Paho MQTT (https://github.com/eclipse/paho.mqtt.python)

What I tried to do (mqtt_apln.py)

import sys
import ssl
import time
import datetime
import logging, traceback
import paho.mqtt.client as mqtt


MQTT_TOPIC = "topictest"
MQTT_MSG = "hello MQTT"


IoT_protocol_name = "x-amzn-mqtt-ca"
aws_iot_endpoint = "xxxxxxx.iot.eu-west-1.amazonaws.com"
url = "https://{}".format(aws_iot_endpoint)

ca = ".xxxxx/rootCA.pem"
cert = ".xxxxx/xxxxx-certificate.pem.crt"
private = ".xxxxx/xxxxxx-private.pem.key"

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
handler = logging.StreamHandler(sys.stdout)
log_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(log_format)
logger.addHandler(handler)

# Define on connect event function
# We shall subscribe to our Topic in this function
def on_connect(mosq, obj, rc):
    mqttc.subscribe(MQTT_TOPIC, 0)

# Define on_message event function. 
# This function will be invoked every time,
# a new message arrives for the subscribed topic 
def on_message(mosq, obj, msg):
        print "Topic: " + str(msg.topic)
        print "QoS: " + str(msg.qos)
        print "Payload: " + str(msg.payload)

def on_subscribe(mosq, obj, mid, granted_qos):
    print("Subscribed to Topic: " +
        MQTT_MSG + " with QoS: " + str(granted_qos))

def ssl_alpn():
    try:
        #debug print opnessl version
        logger.info("open ssl version:{}".format(ssl.OPENSSL_VERSION))
        ssl_context = ssl.create_default_context()
        ssl_context.set_alpn_protocols([IoT_protocol_name])
        ssl_context.load_verify_locations(cafile=ca)
        ssl_context.load_cert_chain(certfile=cert, keyfile=private)

        return  ssl_context
    except Exception as e:
        print("exception ssl_alpn()")
        raise e


mqttc = mqtt.Client()

# Assign event callbacks
mqttc.on_message = on_message
mqttc.on_connect = on_connect
mqttc.on_subscribe = on_subscribe

ssl_context= ssl_alpn()
mqttc.tls_set_context(context=ssl_context)
logger.info("start connect")
mqttc.connect(aws_iot_endpoint, port=443)
logger.info("connect success")
mqttc.loop_start()

In Pi, I installed python 2.7.14 and paho-mqtt

But When I run python mqtt_apln.py, it shows error: ImportError: No module named paho.mqtt.client

Any suggestion is appreciated

gadams999 gadams999 · Accepted Answer · 2018-03-13T01:57:06

I think there are two things going on here. First, a pip install paho-mqtt should make the package active for the current referenced python. For instance, under a 3.6.2 virtualenv should return:

$ pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your 
pip.conf under the [list] section) to disable this warning.

paho-mqtt (1.3.1)
pip (9.0.1)
setuptools (28.8.0)

How did you install the paho-mqtt package, via an apt package or directly with pip? Personally I virtualenv everything or include the package within the application directory via pip install package_name -t . to reference the current working directory.

From there it's working with the ALPN configuration. I reduced your code to just publish to the test topic on my end-point and used the AWS IoT console-->Test to subscribe to the test topic. For both python 2.7.12 and 3.6.2 I successfully received messages.

The main changes were to remove the callbacks, place a mqttc.publish followed by a time.sleep(3) to give the thread time to publish, then closed the connection.

Here is the code paired down for the publish only:

import sys
import ssl
import time
import datetime
import logging, traceback
import paho.mqtt.client as mqtt

MQTT_TOPIC = "topictest"
MQTT_MSG = "hello MQTT"

IoT_protocol_name = "x-amzn-mqtt-ca"
aws_iot_endpoint = "xxxxxx.iot.us-east-1.amazonaws.com"

ca = "ca.pem"
cert = "xxxx-certificate.pem.crt"
private = "xxxx-private.pem.key"

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
handler = logging.StreamHandler(sys.stdout)
log_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(log_format)
logger.addHandler(handler)

def ssl_alpn():
    try:
        #debug print opnessl version
        logger.info("open ssl version:{}".format(ssl.OPENSSL_VERSION))
        ssl_context = ssl.create_default_context()
        ssl_context.set_alpn_protocols([IoT_protocol_name])
        ssl_context.load_verify_locations(cafile=ca)
        ssl_context.load_cert_chain(certfile=cert, keyfile=private)
        return  ssl_context
    except Exception as e:
        print("exception ssl_alpn()")
        raise e


mqttc = mqtt.Client()

ssl_context= ssl_alpn()
mqttc.tls_set_context(context=ssl_context)
logger.info("start connect")
mqttc.connect(aws_iot_endpoint, port=443)
logger.info("connect success")
mqttc.loop_start()

# After loop start publish and wait for message to be sent.
# Hard coded delay but would normally tie into event loop
# or on_publish() CB

# JSON payload because, pretty
mqttc.publish('test', '{"foo": "bar"}')
time.sleep(3)
mqttc.loop_stop()

Please let me know if that works for you? It's awesome that AWS now supports MQTT connections on port 443 without having to utilize websockets (and the need for SigV4 credentials).

AWS IoT: Use MQTT on port 443

2 Answers