How to fix pysftp Intermittent '[Errno 2] No such file'?

Question

I am uploading files to a remote sftp server using the pysftp module in Python. About 25% of the time a file won't upload and I receive the error: '[Errno 2] No such file.'

I am connecting to the remote sftp server with simply a username and password. No SSH key is being used. Upon establishing the connection I pass an instance of cnopts() with hostkeys set to None since no SSH keys are being used. I then loop through each file and and perform a put() to upload each file to the sftp server. The first couple files typically upload successfully, but then by the thrid or fourth file I typically receive the [Errno 2] error. If I rerun the script on the same file that just failed, it will upload just fine. Therefore, it doesn't seem to be an issue with the local or remote path since everything is the same the first run to the second.

Connection Code

elif self.ctype == 'sftp':
    if self.pkpath == None:
        #set pysftp to not check for ssh key, only use password
        cnopts = pysftp.CnOpts()
        cnopts.hostkeys = None
        #connecting to sftp server
        self.conn = pysftp.Connection(self.address, self.user, password=self.password,
                cnopts=cnopts)
    else:
        self.conn = pysftp.Connection(self.address, self.user, password=self.password,
                private_key=self.pkpath)

Put Code (uploadFile function)

def uploadFile(self,fileName,fdestname=None):
    ...
    if fdestname is None:
        fattr = self.conn.put(fileName, confirm=True, preserve_mtime=True)
    else:
        fattr = self.conn.put(fileName, fdestname, confirm=True, preserve_mtime=True)

Looping Code

elif inputdict['source'] == 'unprocessed': 
    Sftp.uploadFile(os.path.join(inputdict['unprocessedfolder'],vimsf), vimsfm)
    print Sftp.lst()

Since the local and remote file paths are correct and subsequent loops upload the previously failed files just fine, I'd expect that all files would upload successfully on the first try.

Matt Brauer Matt Brauer · Accepted Answer · 2019-03-08T22:31:39

In this particular case, the '[Errno 2] No such file.'error was showing up because the input on the put method was set to True. The files were being removed from the destination folder so quickly after the upload, the confirmation process could not complete. It would then return the '[Errno 2] No such file.' error. The issue was intermittent since about 50% of the time, the files would stick around long enough in the destination directory that the confirmation could complete successfully. Setting the confirm bool to False solved the issue.

How to fix pysftp Intermittent '[Errno 2] No such file'?

2 Answers