Receive events from Azure Eventhub using AMQP

3
votes

GOAL: Receive messages from EventHub with python.

Setup:

  • Python 3.5
  • AMQP 1.0
  • EventHub created in new portal.
  • Ubuntu 16.04 and Mac OS X.
  • proton==0.8.8 and python-qpid-proton==0.17.0

I have been trying to get a connection with AMQP to Azure EventHub for a couple days now. I can do this fine with the Java Client (as a test to make sure the EventHub was working) but I need to be able to do this with Python.

After reading a couple examples online I have tried a lot of things over the past days. I am using the RootManageSharedAccessKey and the generated key which doesn't have any / for me. I have tried urllib.quote as well as normal.

Currently my string looks like:

amqps://RootManageSharedAccessKey:<primary key>@<namespace>.servicebus.windows.net/<internal hub name>

If I copy these same parameters into the Java client it works fine. If I use the python SDK to send it works fine as well (it sends it over HTTPS). I have tried using amqps and amqp if I turn on the debug parameter for AMQP I get the error:

condition=:"amqp:unauthorized-access",
description="Unauthorized access. 'Send' claim(s) are required to perform this operation.
Resource: 'sb://<namespace>.servicebus.windows.net/<internal event hub>

That sounds to be a permission issue but the same keys and everything work perfect in Java and in Python over HTTPS so it has to be something in my code vs the keys. Maybe I am not encoding them correctly (I have read you don't need to encode them)? Once I get it working I'll make a GIST and an update for anyone else.

My Current Code is as follows for send which should be easier:

from proton import Messenger, Message

messenger = Messenger()
message = Message()
key = "key"
message.address = "amqps://RootManageSharedAccessKey:"+key+"@<namespace>.servicebus.windows.net/<hubname>"

message.body = u"This is a text string"
messenger.put(message)
messenger.send()`

Things I have tried:

  1. Making policies for both send and receive separate on both the event hub and the namespace.
  2. Recreating the hub
  3. Secure vs Unsecure
  4. Adding the ports to the Firewall
  5. Python 2.7 proton vs 3.5
  6. Linux vs Mac
  7. Primary key vs Secondary key
  8. New namespaces
  9. New Keys with and without / in the key
  10. URL encoding portions and the whole string
  11. UTF-8 encoding
  12. JSON messages vs text messages.
1
python-3.xazureamqpazure-eventhub
I have a similar issue to you. I have an open support request with MS. When I work through my issue, I will update with an answer. – bearrito
How did you enable debug logging with Proton? – bearrito
I was able to get this working. I'd be happy to share what worked for me but I'm not sure I have anything to offer. The only thing I might have done different is to build proton from source. – bearrito
I created and env variable PN_TRACE_FRM=true; and it outputs the logs. of what is happening. I did not build proton from source maybe I should try that. Do you have a redacted gist of how your got yours working you would be willing to share? – JimYong
@bearrito Can you share some details on how to install proton from source or how you get this working. Thanks – Anurag Sharma

1 Answers

0
votes

This might be too late for your project, however there is a new AMQP 1.0 library for Python in development here:

http://github.com/Azure/azure-uamqp-python

Although it's still in early stages (not production ready), there are some samples in the repo for using with Azure Event Hubs:

https://github.com/Azure/azure-uamqp-python/tree/master/samples

Regarding the error you're seeing when using Proton - this looks like it may be attempting to use the wrong authentication mechanism. Have you tried using the Event Hubs Python SDK? They are using some settings here which may be of interest