cloudformation glue job type python shell

2
votes

I am struggling to find how to create a python shell (not spark) glue job via cloudformation, i am unable to find what property defines that.

Only one that would make sence is jobcommand name, but its limited to glueetl as per https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-glue-job-jobcommand.html

3
amazon-cloudformationjobsaws-glue

3 Answers

3
votes

Even though cloudformation page states that the only accepted value is glueetl, glue documentation states that pythonshell is an option aswell. https://docs.aws.amazon.com/glue/latest/dg/add-job-python.html

0
votes

You have to use pythonshell for python and should not specify dpus.

0
votes

Use `pythonshell' in Name property of Command, as explained in docs.

PythonShellJob:
  Type: AWS::Glue::Job
  Properties:
    Command:
      Name: pythonshell
      ScriptLocation: "s3://my-glue-scripts//my-script-file.py"
    Role: !Ref MyJobRole