Storing Avro schema in schema registry

7
votes

I am using Confluent's JDBC connector to send data into Kafka in the Avro format. I need to store this schema in the schema registry, but I'm not sure what format it accepts. I've read the documentation here, but it doesn't mention much.

I have tried this (taking the Avro output and pasting it in - for one int and one string field):

curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json"     --data '{"type":"struct","fields":[{"type":"int64","optional":true,"field":"id"},{"type":"string","optional":true,"field":"serial"}],"optional":false,"name":"test"}' http://localhost:8081/subjects/view/versions

but I get the error: {"error_code":422,"message":"Unrecognized field: type"}

2
apache-kafkaavro

2 Answers

13
votes

The schema that you give as a JSON should start with a 'schema' key. The actual schema that you provide will be the value of the key schema.

So your request should look like this:

 curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json"     --data '{"schema" : "{\"type\":\"string\",\"fields\":[{\"type\":\"int64\",\"optional\":true,\"field\":\"id\"},{\"type\":\"string\",\"optional\":true,\"field\":\"serial\"}],\"optional\":false,\"name\":\"test\"}"}' http://localhost:8081/subjects/view/versions

I've made two other changes to the command:

  • I've escaped each double quote within the value of the schema key.
  • I've changed the struct data structure to string. I'm not sure why it isn't taking complex structures though.

Check out how they've modeled the schema here, for the first POST request described in the documentation.

0
votes

First, do you need to store the schema in advance? If you use the JDBC connector with the Avro converter (which is part of the schema registry package), the JDBC connector will figure out the schema of the table from the database and register it for you. You will need to specify the converter in your KafkaConnect config file. You can use this as an example: https://github.com/confluentinc/schema-registry/blob/master/config/connect-avro-standalone.properties

If you really want to register the schema yourself, there's some chance the issue is with the shell command - escaping JSON in shell is tricky. I installed Advanced Rest Client in Chrome and use that to work with the REST APIs of both schema registry and KafkaConnect.