Cannot create new AWS Athena Data Catalog

Question

I am trying to create new Glue data catalogs in Athena. It always fails with INVALID_INPUT, no other error context.

Here is the API that I am using https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Athena/Client.html#create_data_catalog-instance_method.

It always fails with an invalid input when selecting type = GLUE.

require 'aws-sdk-athena'
Aws::Athena::Client.new.create_data_catalog(name: 'testing', type: "GLUE")
Aws::Athena::Errors::InvalidRequestException (INVALID_INPUT)
    from (irb):51:in `rescue in irb_binding'
    from (irb):51

Even using the AWS CLI it fails with the same cryptic error.

aws athena create-data-catalog --region us-west-1 --name testing --type GLUE                                                                                                                                                                               

An error occurred (InvalidRequestException) when calling the CreateDataCatalog operation: INVALID_INPUT

How do I create a new Glue Data Catalog for Athena? Or am I limited to the default, AwsDataCatalog?

I don't see region being a parameter of create-data-catalog according to cli documentation — Philipp Johannis
It is set via environment variable, AWS_REGION=us-west-1. Philipp. — allen

Theo Theo · Accepted Answer · 2020-09-25T20:16:09

TL; DR: you already have a Glue Data Catalog, you can only have one, and this is a very poor error.

My guess is that this API is poorly documented and implemented. When Athena was launched it had its own catalog implementation, because Glue didn't exist yet. When Glue was launched you had to opt in to using Glue's catalog if you had started using Athena, while new users got the Glue catalog by default. I don't remember if this was the API call you used to migrate to the Glue catalog, but it might have been.

Either way, there is only one Glue Data Catalog per account, and you already have one, and I suspect the error is really just that you can't create another one. This is roughly how useful and helpful Athena's error messages used to be back in the day. "Something went wrong, you figure it out!" When they added Federated Query and external Hive catalogs they could have improved this API call, added proper documentation about Glue, and meaningful error messages, but I guess they opted not to.

Cannot create new AWS Athena Data Catalog

1 Answers