Implement schema registry with a basic auth in confluent kafka

Question

I am new to confluent Kafka and become able to run Confluent kafka and its producer and consumer with Avro. I have register my new schema with these commands with the help of this guide.:

D:\ApachKafka\confluent>python register_Scehma.py http://localhost:8081 playerTopic messageFormat.avsc

Here is my python code for registry:

import os
import sys

import requests

schema_registry_url = sys.argv[1]
topic = sys.argv[2]
schema_file = sys.argv[3]

aboslute_path_to_schema = os.path.join(os.getcwd(), schema_file)

print("Schema Registry URL: " + schema_registry_url)
print("Topic: " + topic)
print("Schema file: " + schema_file)
print

with open(aboslute_path_to_schema, 'r') as content_file:
    schema = content_file.read()

payload = "{ \"schema\": \"" \
          + schema.replace("\"", "\\\"").replace("\t", "").replace("\n", "") \
          + "\" }"

url = schema_registry_url + "/subjects/" + topic + "-value/versions"
headers = {"Content-Type": "application/vnd.schemaregistry.v1+json"}

r = requests.post(url, headers=headers, data=payload)
if r.status_code == requests.codes.ok:
    print("Success")
else:
    r.raise_for_status()

All is working fine my producer sending avro data and i am receving it on my client using C#. But now I want to implement schema registry with basic auth credential. For this reason I googled and found this link which is very confusing for me. For example, it stated the use following settings configure Schema Registry to require authentication:

authentication.method=BASIC
authentication.roles=<user-role1>,<user-role2>,...
authentication.realm=<section-in-jaas_config.file>

where should i add these lines? The documentation is confusing so i am searching a way to implement schema registry with basic auth ?

Sergeant Sergeant · Accepted Answer · 2019-10-17T14:39:49

I think basic auth is only going to work when you put for example nginx in front of it and proxy the request towards the schema registry. Then use nginx to setup the basic auth.

The page you mention has an Authentication chapter but that's only from the schema registry towards the brokers.

To protect the schema registry endpoint Confluent offers a paid plugin:

Schema Registry Security Plugin
This is a commercial component of Confluent Platform.

I'm looking for a solution as well, hence landed on your question. You could also use a firewall to limit the IPs that can access the schema registry ports. In addition, your producers and consumers need to be able to handle the basic auth mechanism, all in all, not perfect. Not sure why the schema registry can't use the zookeeper SCRAM users for example, would be the easiest way to go.

EDIT: In addition, you could also use an url like this:

https://username:[email protected]/

Not ideal, but it works I guess...

Implement schema registry with a basic auth in confluent kafka

1 Answers