sftp to aws lightsail python using pysftp key error

Question

I have aws ssh key as awskey.pem. I am trying to sftp to the host with pysftp.

import pysftp as sftp

keyfile='c:\\awskey.pem'
Cnopts=sftp.CnOpts()
Cnopts.hostkeys.load(keyfile)
srv=sftp.connection(host ='xx.xx.xx.xx',cnopts=Cnopts)
lpath='c:\\test.txt'
rpath='test.txt'
srv.put(lpath,rpath)
srv.close()

there seems to be a problem reading the pem file as i get the error
'pysftp.exceptions.HostKeysException: Host keys not found'
In searching other sites there is no discussion on the format of a key file.
Not sure if it is supposed to be a .pem format or I have to convert it to some other format.
After spending 2 days on this I am no further ahead.

keyfile='c:\\awskey.pem'     
srv=sftp.connection(host ='xx.xx.xx.xx',username='user',private_key=keyfile)

and now i get error
pysftp.exceptions.HostKeysException: No Host Keys Found
Must be missing something somewhere. This is on Windows 10 so no .ssh/known_hosts file.
Do I need to retrieve the host key ?
or switch back to pysftp 0.2.8 as suggested elsewhere?

Answer 1

finally got it to work.
Installed ubuntu on win10.
Logged in to aws with ssh -i "key.pem" uname@aws
which saved the known_hosts file.
Had to use ssh-keygen and key-scan from
https://unix.stackexchange.com/questions/6533
to correct the file.
finally copy known_hosts from /root/.ssh to c:/user/myname/.ssh and
not from /home/.ssh/
so code is now

awshost="www.xxxxxxx.com"
keyfile=r'C:\files\docs\1-AWS\LightsailDefaultKey-us-west-2.pem'
known=r'C:\Users\myname\.ssh\known_hosts'
cn = sftp.CnOpts(knownhosts=known)

srv = sftp.Connection(host=awshost,
                username='bitnami',
                private_key=keyfile,
                cnopts=cn)

and it connects with no errors

Answer 2

I had the same issue. Here are the steps I took to resolve:

1. Open a terminal and navigate to the folder containing the .pem file.
2. Run chmod 400 keyfile.pem
3. Use OpenSSH to ssh into your instance: ssh -i "keyfile.pem" USERNAME@IP_OF_INSTANCE

Now you can go back to running your python script.

Explanation:

pysftp searches for ssh key entries in 'known_hosts'. The first time you SSH into the instance, OpenSSH automatically creates an entry for it in 'known_hosts', which pysftp can now locate.

The reason for step 2 above is that AWS requires more restrictive permissions for the key file.

Note for Windows users:

If you try to use PuTTy to do this, pysftp won't be able to locate the SSH entries, as PuTTy stores them in the Windows registry. Instead, you can install and use OpenSSH on Windows, then in a bash terminal (such as Git BASH) you can follow the same 3 steps above.

Answer 3

Your problem is regarding the server host key. For that see Verify host key with pysftp.

Your problem has nothing to do with your private key, which you should use for authentication. The private key goes to private_key argument of Connection constructor.
See Connect to SFTP with key file using Python pysftp

---

See also my article about the keys involved in SSH.

Answer 4

Finally got it to work as Filezilla works with just the key file,user,hostip then setting hostkeys to None works fine after setting CnOpts call to the hostip.

import pysftp as sftp

hostip  = "xx.xx.xx.xx"
keyfile ='c:\\awskey.pem'
userb   = "bitnami"

Cnopts=sftp.CnOpts(hostip)
Cnopts.hostkeys = None

srv=sftp.connection(host = hostip,
                    username=userb,
                    cnopts=Cnopts)


lpath='c:\\test.txt'
rpath='apps/wordpress/htdocs/mytheme/test.txt'
srv.put(lpath,rpath)
srv.close()