Python Azure SDK: create multiple VMs in the same cloud service

Question

I am trying to create multiple VMs in the same cloud service (but in unique deployments within the cloud service). The below produces only one machine which will have the name mycluster-master-3 (given self.numMasters == 3). In other words, only the last machine is actually created and can be seen in Azure management portal. Note that there are no errors when I execute this code with self.numMasters == 3:

for node in range(self.numMasters):
            logger.info('Deploying virtual machine "%s"...' % (self.clusterName + '-master-' + str(node + 1)))
            logger.info('Creating linuxConfig...')
            linuxConfig = azure.servicemanagement.LinuxConfigurationSet(
                host_name=self.clusterName + '-master-' + str(node + 1),
                user_name='auser',
                user_password='Auser123!',
                disable_ssh_password_authentication=True
            )
            # Destination storage account container/blob where the VM disk
            # will be created
            media_link = 'https://xxxxxxx.blob.core.windows.net/xxxxxx/%s.vhd' % (self.clusterName + '-master-' + str(node + 1))
            os_hd = azure.servicemanagement.OSVirtualHardDisk(image_name, media_link)
            logger.info('Creating virtual machine deployment...')
            self.sms.create_virtual_machine_deployment(
                service_name=serviceName,
                deployment_name=self.clusterName + '-master-' + str(node + 1),
                label=self.clusterName + '-master-' + str(node + 1),
                role_name=self.clusterName + '-master-' + str(node + 1),
                system_config=linuxConfig,
                os_virtual_hard_disk=os_hd,
                role_size=self.instanceType,
                deployment_slot='production'
            )

I thought having all my VMs in one cloud service would allow them to talk to each other internally without being exposed to machines in my other cloud services. But apparently, a cloud service provides a single public IP to VM within it. Perhaps that's why only one VM is created, although, conceptually, I thought a single cloud service can have multiple VM instances. This confuses the hell out of me. So now I am thinking that I need to have one cloud service for each VM instance in my cluster. This way, all of the machines will have a public IP and can be accessed over the internet. I also need to put all the cloud services within a virtual network. That way, traffic between them will not be routed through internet. Is my assumption correct? Is there a better/other way to accomplish what I want (a cluster of machines talking to each other securely and fast, yet be SSH-able through internet.)

astaykov astaykov · Accepted Answer · 2015-07-02T07:33:56

First of all - if you are creating a cluster, do not put the VMs in separate Cloud Services. Please read thouroughly this article: Manage the availbility of Virtual Mchines.

Then to the main topic: multiple VMs can and should be deployed into same Cloud Service when they serve common purpose (building a cluster).

VMs in same Cloud service can talk to each other without going to Internet and without need for you to do something special, beside opening FireWalls on the VMs themselfs.

For how the name resolution works in Azure as a whole, you can read the following article: Name resolution for VM and role instances.

One VIP per Cloud service - this is so. But why should that be an issue? Please elaborate on the reason for you to require multiple VIP (Public IP Addresses) for the single Cloud Service (your cluster). The following article will give you an idea about different options for Load Balancing your clustered VMs: Load Balancing for Virtual Machines.

On how to configure SSH for per-VM access, you can create Endpoints for your VMs with different Public Ports.

Last, but not least, the reasony why you end up with only one VM created. You can only create one VM in Cloud Service at a time. You cannot create second VM in same cloud service while the first one is still being created. So, the creation of VM1 should be 100% complete, then you can create second VM in same Cloud Service.

Python Azure SDK: create multiple VMs in the same cloud service

1 Answers