how to use opentelmetry with multiple microservices (python)

Question

in a single application i can easily create (nested) spans but i am trying to trace http requests throughout multiple services and nothing i try with context propagation is working. so maybe my setup is wrong.

can someone please explain the exact requirements for this in python?

so as far as i understand, in each microservice i have to setup my span exporter, the collector and trace providers.

exporter = JaegerSpanExporter(
    service_name="some-name",
    agent_host_name="localhost",
    agent_port=6831, ) span_processor = BatchExportSpanProcessor(exporter)
trace.set_tracer_provider(TracerProvider())
trace.get_tracer_provider().add_span_processor(span_processor) tracer
tracer = trace.get_tracer(__name__)

after that is done in the classes that i want to instrument, i would just navigate to the block of code in my service that i wanna trace:

def some_method()
    with tracer.start_as_current_span("my-span"):
        # set some attributes and some events ....
        start_some_request()

when i start my service the traces work perfectly and i can see them on my Jaeger UI but the spans are not aggregated under a single trace for that one individual http request.

I know there should be some context propagation in there to achieve this, but i can't find any way to get this done in python. I'm using the W3C ContextTrace but i can't get it to work and i'm not really sure if i should setup a trace provider in every service or what is wrong exactly.

I've read a lot of documentation but i still don't know how to get this to work.

charles-edouard gagnaire charles-edouard gagnaire · Accepted Answer · 2021-06-22T22:29:56

to do this you should use tracer.inject and tracer.extract.

For instance in the client:

with tracer.start_span('client') as span:
    headers = {}
    # inject your context in the headers, to pass it to downstream service
    tracer.inject(span, Format.HTTP_HEADERS, headers)
    ...
    your code
    ...

And in your server:

# extract context from upstream service
span_ctx = tracer.extract(Format.HTTP_HEADERS, self.headers)
span_tags = {tags.SPAN_KIND: tags.SPAN_KIND_RPC_SERVER}
# start a new span, child of the upstream span
with tracer.start_span('front', child_of=span_ctx, tags=span_tags) as span:
...
your code
...

Like this you should have a tree view in the Jaeger UI.

Please tell me if you need more informations.

how to use opentelmetry with multiple microservices (python)

1 Answers