KsqlStatementException: statement does not define the schema and the supplied format does not support schema inference

0
votes

I am using a ksql-server on kubernetes using the confluent helm charts.

https://github.com/confluentinc/cp-helm-charts/tree/master/charts/cp-ksql-server

I modified the queries.sql file for my own personal use case. https://github.com/confluentinc/cp-helm-charts/blob/master/charts/cp-ksql-server/queries.sql

this is my query:

CREATE STREAM ksql_test WITH (kafka_topic='orders-topic', value_format='DELIMITED', partitions='1', replicas='3');

Once i deploy this pod, i get this error:

ERROR Failed to start KSQL Server with query file: /etc/ksql/queries/queries.sql (io.confluent.ksql.rest.server.StandaloneExecutor:124)
io.confluent.ksql.util.KsqlStatementException: statement does not define the schema and the supplied format does not support schema inference
Statement: CREATE STREAM ksql_test WITH (kafka_topic='orders-topic', value_format='DELIMITED', partitions='1', replicas='3');

No matter if I change the format to JSON, the error remains the same and i dont have schema for this topic.

1
apache-kafkaconfluent-platformksqldb

1 Answers

2
votes

This is the problem:

statement does not define the schema

A KSQL stream is a Kafka topic plus a schema. No schema, no stream.

If your data is delimited, maybe it looks like this: 

1,FOO,0.1,400

That has a schema, perhaps something like:

CREATE STREAM example (COL1 INT, LABEL VARCHAR, WIBBLE DOUBLE, TARGET BIGINT)
WITH (KAFKA_TOPIC='example_topic',  VALUE_FORMAT='DELIMITED);

tl;dr you can't create a stream without a schema. If you're using Avro, you have a schema already (in the Schema Registry) and hence don't have to declare it. If you're using JSON or Delimited, you must declare the schema explicitly.