4
votes

I have a data catalog managed by AWS Glue, and any update that my developers does in our S3 bucket with new tables or partitions we are using the crawlers to update that every day to keep the new partitions healthy.

But, we also need custom table properties. In our hive we have the data source of each table as a table property, and we added to the tables in the Data Catalog in glue but, every time we run the crawler it overwrites the custom table properties like Description.

Am I doing anything wrong? Or is this a bug from AWS Glue?

1
Hey, facing the same problem, did you resolve it?ronhash
Sorry, not solved yet :(Thiago Baldim
Same here, did you by chance figure it out? ThanksZack Bartel
Not yet Mate, We had a work around in the company structure.Thiago Baldim

1 Answers

0
votes

Have you checked Schema change policy in your crawler definition?

enter image description here