There are two scenarios to compare files:
Scenario 1: Compare files at remote branches (both branches should exists in the remote repository)
Scenario 2: Compare local files (at the local working area copy) to the files at the remote repository.
The logic is simple. If you provide two branch names to diff, it will always compare the remote branches, and if you provide only one branch name, it will always compare your local working copy with the remote repository (the one you provided). You can use range to provide remote repositories.
E.g., check out a branch:
git checkout branch1
git diff branch2 [filename]
In this case, if you provide filename, it will compare your local copy of filename with the remote branch named "branch2".
git diff branch1 branch2 [filename]
In this case, it will compare filename from remote branches named "branch1" vs "branch2"
git diff ..branch2 [filename]
In this case also, it will compare filename from remote branches named "branch1" vs "branch2". So, it's the same as above. However, if you have just created a branch from another branch, say "master" and your current branch doesn't exists on the remote repository, it will compare remote "master" vs. remote "branch2".