Using PowerShell Remoting (not to be confused with RDP) looks to be perfectly possible for Azure VMs:
PowerShell Remoting needs the correct network configuration to allow
for remote connectivity. Since we will be accessing the VM over the
web we are required to use HTTPS to secure the communication channel.
Skipping over the details of Windows Remote Management which is used
by PowerShell Remoting, I will just say the default HTTPS port is
5986.
To allow connectivity to the VM this endpoint must be added to the
ServiceDefinition.csdef:
<endpoints>
<inputendpoint localport="5986" port="5986" protocol="tcp" name="WinRM" />
</endpoints>
With the proper port open it is just a matter of enabling PowerShell
Remoting. Two issues need to be resolved before enabling it.
The first is using the correct user account to setup Remoting. You
must use the account previously created to run the script to setup
Remoting[1]. With these commands the user you previously created will
execute the script e:\approot\StartRemotingListener.ps1:
schtasks /CREATE /TN "StartRemotingListener" /SC ONCE /SD 01/01/2020 /ST 00:00:00
/RL HIGHEST /RU <username> /RP <password>
/TR "powershell -ExecutionPolicy unrestricted
-Command e:\approot\StartRemotingListener.ps1 -Force" /F
schtasks /RUN /TN "StartRemotingListener"
The second issue is configuring the certificate needed for the HTTPS
connection. Acquiring a certificate might not be a problem for a
production environment. You might already have one for your service
and could reuse it. It can be a problem if your service doesn’t
require a certificate for normal operation or for non-production
environments.
From: http://blogs.msdn.com/b/mariok/archive/2011/08/08/command-line-access-to-azure-vms-powershell-remoting.aspx