Terraform GCP Instance Metadata Startup Script Issue

Question

I've been working with Terraform, v0.15.4, for a few weeks now, and have gotten to grips with most of the lingo. I'm currently trying to create a cluster of RHEL 7 instances dynamically on GCP, and have, for the most part, got it to run okay.

I'm at the point of deploying an instance with certain metadata passed along to it for use in scripts built into the machine image for configuration thereafter. This metadata is typically just passed via an echo into a text file, which the scripts then pickup as required.

It's... very simple. Echo "STUFF" > file... Alas, I am hitting the same issue OVER AND OVER and it's driving me INSANE. I've Google'd around for ages, but all I can find is examples of the exact thing that I'm doing, the only difference is that theirs works, mine doesn't... So hopefully I can get some help here.

My 'makes it half-way' code is as follows:

resource "google_compute_instance" "GP_Master_Node" {
  ...
  metadata_startup_script = <<-EOF
  echo "hello
  you" > /test.txt
  echo "help
  me" > /test2.txt
  EOF

Now the instance with this does create successfully, although when I look onto the instance, I get one file called ' /test.txt? ' (or if I 'ls' the file, it shows as ' /test.txt^M ') and no second file.. I can run any command instead of echo, and whilst the first finishes, the second+ does not. Why?? What on earth is causing that??

The following code I found also, but it doesn't work for me at all, with the error, 'Blocks of type "metadata" are not expected here.'

resource "google_compute_instance" "GP_Master_Node" {
  ...
  metadata {
    startup-script = "echo test > /test.txt"
  }

Which operating system and editor are you using to develop your Terraform code? Windows uses CRLF for EOL. I think the shell script is failing due to EOL (end-of-line) issues. The compute engine serial console will show the metadata script executing so that you can see exactly what is happening. Tip: put your script into a file, make sure the EOL uses the Linux format and specify a file instead of text inside your .tf file. — John Hanley
For the first chunk of code, most probably not related to Terraform as even adding it in console does not work. Most probably related to how echo interacts with EOF. If the objective is to create 2 text files with the text supplied this script should work: #! /bin/bash echo "hello you" > /test.txt echo "help me" > /test1.txt — Fady
@Fady Good catch. His commands are missing the line continuation character \. — John Hanley
@JohnHanley Thanks, that's precisely the issue. Though in answer to your previous question, I am on Mac (against my will) using the Atom text editor. This morning I saved the file using VI just to see, and lo and behond, and it was [dos] and I don't know how that happened.. In any case, thanks for the heads up! — Vinny Pem

Vinny Pem Vinny Pem · Accepted Answer · 2021-06-02T08:15:11

Okaaaaay! Simple answer for a, in hindsight, silly question (sort of). The file was somehow formmated in DOS, meaning the script required a line continuation character to run correctly (specifically \ at the end of each individual command). Code as follows:

resource "google_compute_instance" "GP_Master_Node" {
  ...
  metadata_startup_script = <<-EOF
  echo "hello
  you" > /test.txt \
  echo "help
  me" > /test2.txt \
  echo "example1" > /test3.txt \
  echo "and so on..." > /final.txt
  EOF

However, what also fixed my issue was just 'refreshing' the file (probably a word for this, I don't know). I created a brand new file using touch, 'more'd the original file contents to screen, and then copy pasted them into the new one. On save, it is no longer DOS, as expected, and then when I run terraform the code runs as expected without requiring the line continuation characters at the end of commands.

Thank you to commentors for the help :)

Terraform GCP Instance Metadata Startup Script Issue

1 Answers