Using emacs tramp vs. rsync for remote development

Question

I have been doing some remote development using emacs tramp and found that it was quite slow. Every time I save a file, it takes about 10 seconds to complete the save. So, now I am using rsync to transfer the files remotely and it works much faster, it takes about a second plus the local saves from emacs are instant. Are there any configuration options within tramp to get it to run as fast as rsync does on the command line? Are there any advantages to using tramp instead of rsync even though I am seeing such poor performance?

Answer 1

While tramp can be configured to use a large number of transport method, I suppose you're using ssh to connect to the remote server.

I believe most of the time it takes to complete the operation comes from setting up a connection and authenticating. If you're using a new enough version of OpenSSH, this can be helped by using the connection sharing feature, see ControlMaster in ssh_config(5).

I suggest you try

ControlMaster auto

and see if that improves the situation.

Answer 2

If you're using tramp for ssh/scp functionality, you'll find opening a persistent SSH connection will make tramp operations a lot faster: they'll reuse the existing connection rather than creating a fresh one each time, cutting out a great deal of cryptographic overhead without affecting security. See this article on the SSH ControlMaster functionality.

Answer 3

A couple of answers have mentioned enabling ControlMaster to keep a persistent ssh connection open (which avoids the expensive connection set-up / hand-shaking otherwise needed for each Tramp operation over ssh), but you don't need to configure anything outside of Emacs to use this -- if you look at the tramp-methods variable you will see there are existing methods which enable ControlMaster.

Specifically, the rsyncc and scpc methods.

Both sides of the connection need to support ControlMaster, of course (for instance Cygwin cannot do so, which is a shame for Windows users*), but I would suggest setting one of those as your tramp-default-method.

Not having a persistent connection makes extended Tramp usage pretty painful, but with one it's incredibly usable (to the point where, given a reasonably fast connection, you can almost forget that it's happening).

In answer to the final part of your question, yes, there are very good reasons to use Tramp instead of rsyncing the files manually. The primary one is that shell commands can be executed directly on the remote server, and you don't even have to think about it, as Emacs takes care of the details.

For instance, from a remote-file buffer, M-x shell RET opens a shell on the remote server, and commands like M-x rgrep RET and M-x find-grep-dired RET will run the find + grep on the remote server. I believe that this applies in general to functions which invoke shell commands.

For me, that's more than enough reason to use Tramp (unless installing Emacs on the remote server and using your local display was an option in which case, for long-term usage, I would consider doing that instead).

(*) When using Windows I used to host a Linux VM locally and run Emacs inside that (with Cygwin providing the X display) for the sole reason of using ControlMaster for Tramp ( https://stackoverflow.com/a/3049375/324105 ).

Answer 4

Are you aware of the section in the tramp documentation on gnu.org that mentions using rsync? It says, in part:

5.3 External transfer methods

The external transfer methods operate through multiple channels, using the remote shell connection for many actions while delegating file transfers to an external transfer utility.

This saves the overhead of encoding and decoding that multiplexing the transfer through the one connection has with the inline methods.

...

rsync — ssh and rsync Using the ssh command to connect securely to the remote machine and the rsync command to transfer files is almost identical to the scp method.

While rsync performs much better than scp when transferring files that 
exist on both hosts, this advantage is lost if the file exists only on one side 
of the connection.

The rsync based method may be considerably faster than the rcp based
methods when writing to the remote system. Reading files to the local
machine is no faster than with a direct copy.

This method supports the ‘-p’ hack.

Answer 5

Since you said earlier that you have a problem on a particular server, check the sshd config file on the problem machine. It's probably something like /etc/ssh/sshd_config.

Look for a config option named "UseDNS". Set that to "No."

See if that doesn't speed things up for you. If so, then ssh is probably timing out waiting on a reverse dns lookup that you likely don't care about.

Answer 6

Recently I've come to embrace sshfs so that the remote files are locally editable. Works well for Linux, Mac and Unix systems and puts the ssh parts in one command rather than every access. And exists entirely in User space.

Answer 7

If you are using the ssh method, you can try the sftp method instead, which (I believe) opens a persistent sftp connection to the remote ssh server. Also, if you are using GNOME, you can add sftp to tramp-gvfs-methods to have TRAMP connect to sftp servers using GNOME's GVFS, which may or may not be faster/more convenient.