Yes. You can do it with RSA - to do a Diffie-Hellman-like exchange, because not only do the keys from 1 associated pair commute, but keys from different keypairs can commute as well.
alice -> bob: alice.pub
bob -> alice: bob.pub
alice: r = random.secret()
alice -> bob: ( r * (alice.priv * bob.pub) )
bob: r = ( (r * (alice.priv * bob.pub)) * (bob.priv * alice.pub) )
Notice that we did something odd here. We mixed RSA operations from different keypairs in one operation. The objects in parenthesis are effectively a new virtual RSA key, and neither one of these keys is public. Had we tried to create that RSA key directly, either alice or bob would know both keys of the pair. This keypair is effectively a secret key where you write to one end and only the other side can decrypt it, yet you cant decrypt what you wrote yourself, and nobody else can encrypt messages to the other side.
I have never seen anyone mix keypairs like this, but I tested this by writing the code. I had to do something unusual though because normally, applying the private key to the message is for 'signing'. But signing usually hashes the secret and applies the private key to a hash of it; something we do not want. So in my code, once I had the RSA components (D,E,N) extracted into arbitrary precision numbers... ie: decrypt,encrypt,modulus ... I just did:
wormholeSend(me,you,msg) =
(((me ^ {me_D}) \% me_N) ^ {you_E}) \% you_N
The thing that makes it a little tricky is that E (encrypt exponent) is actually a predictable value, but the modulus N is in the public key (E,N). D is private to each party. We need to be careful here, because you and I have a different modulus N.
I did this because I wanted a system where a program is authorized to encrypt keys that can be decrypted by users. Doing this, the user cannot encrypt keys, and the program cannot decrypt them.