How to generalize resources etc in terraform?

Question

I have been experimenting with terraform and AWS, doing something like:

...
provider "aws" {
  access_key = "${var.access_key}"
  secret_key = "${var.secret_key}"
  region = "${var.region}"
}

resource "aws_instance" "bastion" {
  ami = "${var.image}"
  instance_type = "${var.inst_type}"
  key_name = "My Keys"
  subnet_id = "${aws_subnet.my_public_subnet.id}"
  vpc_security_group_ids = [
    "${aws_security_group.my_vpc_security_group.id}"
  ]
  tags={
    Name="${var.inst_name}"
  }
}
...

But now I'd like to move one step away from AWS, and write something that isn't tied to AWS, but which would work for all cloud providers, ideally. Is there a way to achieve this? I would really appreciate a couple of pointers in the right direction.

===EDIT===

Maybe I need to explain a bit further: I am not after a full generalisation of everything in eg. the AWS API. However, since most, if not all, cloud environments let you create VMs with Linux, and also have some way of setting up some initial configuration via terraform, it ought to be possible to parametrize the provider specific parts, so that I can write the essential infrastructure structure code and specify whether I use AWS, Azure, Google etc as a set of parameters. My problem is that I haven't worked long enough with terraform to see how to do that.

Answer 1

You may be able to build something by using modules. You could try to build a cloud agnostic VM Module and decide on the cloud provider based on some switches.

However, It is not really what Terraform is designed to do, so It might get messy.

resource "azurerm_virtual_machine" "main" {
  name                  = "${var.prefix}-vm"
  count                  = "${var.deployToAzure ? 1 : 0}"
  ...
}

resource "aws_instance" "web" {
  ami           = "${data.aws_ami.ubuntu.id}"
  instance_type = "t2.micro"
  count         = "${var.deployToAws ? 1 : 0}"
}

As a more realistic first step, you can build your own modules around these VM resources. So you would have a Azure VM module and the same for AWS. If you strive to keep similar interfaces for these modules, your actual infrastructure code would be less dependent on the cloud provider.

Answer 2

In principle you can create multi-cloud abstractions in Terraform using Module Composition.

With that said, you can only practically abstract over concepts that are equivalent. An example shown in the Terraform guide is DNS records: that's a standard concept with many different implementations across different vendors, and so it's reasonable to define a set of conventions to allow one module to produce a set of requested DNS records that can then be passed to any one of a selection of vendor-specific DNS recordset modules. There are some examples of that (mostly experimental, at the time of writing) in the terraformdns namespace of the public Terraform registry.

To abstract over virtual machine services is a tougher proposition, because there isn't a clear and obvious common featureset across all vendors. To do so would require making some tradeoffs about how to generalize, and the decisions you make would then restrict your ability to use vendor-specific features.

Instead, we usually prefer to work at a higher level of abstraction when deploying similar infrastructure across multiple vendors. For example, if you are deploying a specific piece of software across multiple cloud systems then you can potentially write a module for each of your target cloud vendors that encapsulates the deployment of that software, and make the input variables and output values of those modules similar enough that they can substitute for one another in a reasonable way.

To pick a specific example, perhaps your unit of abstraction might be a Kubernetes cluster, and so you could write multiple modules that all result in a Kubernetes API hostname being exported as an output, but internally they can produce that using a number of approaches such as a vendor's managed Kubernetes service, a minikube deployment for development, etc. In this case all of these modules ultimately produce a functioning Kubernetes API, and you can choose for each configuration which of them to use.

Answer 3

This is not possible to generalize the solution. Different Cloud technologies have different offerings and different API's for them to connect. This is the reason terraform ask for provider information in start so it can load specific provider SDK to communicate with API.