How to install stackdriver in LINUX(Ubuntu/Debian) GCE instances in GCP using Terraform script?

Question

I am creating a LINUX Debian instance using the below terraform script.

resource "template_dir" "config" {
  source_dir      = "${path.module}/config.d/"
  destination_dir = "/tmp/fluent-templates"

  vars = {
    instance-name = "${var.instance_name}"
  }
}

resource "google_compute_instance" "default" {
  name         = "${var.instance_name}"
  project      = "${var.project}"
  machine_type = "${var.machine_type}"
  zone         = "${var.zone}"

  boot_disk {
    initialize_params {
      image = "${var.boot_disk_image}"
    }
  }

  network_interface {
    network = "default"

    access_config {
      // Ephemeral IP
    }
  }


  #StackDriver must be installed before this command runs,
  #as it will create the "/etc/google-fluentd/config.d" directory,
  #which is supposed to be replaced by the below provisioner

  provisioner "file" {
    source      = "${template_dir.config.destination_dir}"
    destination = "/etc/google-fluentd/config.d"
  }
}

I want to install StackDriver Logging Agent on these Debian/Ubuntu using Terraform to avoid SSH manually and install it every time I spin up an instance.

I tried using remote-exec but it didn't work for me. Below is the code for remote-exec:

provisioner "remote-exec" {
    inline = [
      "curl -sSO https://dl.google.com/cloudagents/install-logging-agent.sh",
      "bash install-logging-agent.sh",
    ]
  }

Putting the above code inside the google_compute_instance resource in my terraform script didn't work and ended up failing to connect after about 5 minutes with the following error:

* google_compute_instance.default:
 timeout - last error: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

I am not sure how to connect to the server to use remote-exec.

Answer 1

Here's a link to create the Terraform script:

https://cloud.google.com/community/tutorials/getting-started-on-gcp-with-terraform

Add the following metadata in order to install the Stackdriver Logging agent:

metadata_startup_script = "curl -sSO https://dl.google.com/cloudagents/install-logging-agent.sh; sudo bash install-logging-agent.sh"

Finally SSH into the instance and check the status of the service:

$ sudo service google-fluentd status

Answer 2

Using remote-exec worked the best for me.

provisioner "remote-exec" {
    inline = [
      "curl -sSO https://dl.google.com/cloudagents/install-logging-agent.sh",
      "sudo bash install-logging-agent.sh",
      "rm install-logging-agent.sh",
    ]
    connection {
            type  = "ssh"
            user  = "${var.gce_ssh_user}"
            private_key = "${file(var.gce_ssh_private_key_file)}"
            timeout = "60s"
        }
  }

While spinning up the LINUX instance, use remote-exec to ssh into the instance and run the two commands mentioned one Installing the agent on Linux and Windows page.

I added rm install-logging-agent.sh to remove the script after installation is done.

Answer 3

It seems like you should be able to use the startup script field to specify the agent install:

https://www.terraform.io/docs/providers/google/r/compute_instance.html

metadata_startup_script = "echo hi > /test.txt"