How would you retrieve message and optional attributes from PubSubIO?

2
votes

I have a topic on PubSub, and I want to create a pipeline on dataflow that reads from the topic classifies them according to message data and further read the attribute key value pairs. Is there a way to differentiate between the message and attributes after they have transformed to PCollection using the PubSubIO's read transform?

1
google-cloud-dataflowgoogle-cloud-pubsub

1 Answers

1
votes

Unfortunately, Dataflow's PubsubIO API doesn't currently handle metadata and attributes of the Pubsub messages. I cannot think of an easy workaround.

This is something we are looking to improve rather soon. This work is tracked as BEAM-53. Please check there for the most recent status.