I created a glue crawler which crawls the data and creates the table in the glue data catalog. Suppose I have a CSV file (file1.csv) which has a schema like (id,name) and once the crawler job execution is completed it creates the Athena table (crawler_file) with 2 columns (id,name). Now there is a new file (file2.csv) which has a schema like (id,name,roll_no). Currently when the glue crawler is executing it is creating a new Athena table (crawler_file_111) with schema (id,name,roll_no). Can I configure the crawler in such a way that the crawler instead of creating the new table updates the existing schema of the table? In this scenario instead of creating a new Athena table (crawler_file_111) it should update the existing Athena table (crawler_file). Can I somehow achieve this scenario?
2
votes
