I'm still a newbie but I'll try and help =].
It doesn't seem to work when you use a modifier key with two mouse buttons, so this is a way that kind of works:
+WheelUp::
KeyWait, RButton, D ; Waits for RButton to be pressed down.
MsgBox, This works!
Return
The problem is it clicks (or releases) the right mouse button once the hotkey has run. If you instead put it like so:
+RButton::
KeyWait, WheelUp, D
There will be another problem in that it will work fine for the first use of the hotkey, it will from then on work with only Shift + Right Mouse Button, because it's already waited for WheelUp to be pressed down (or rather scrolled up).
I mucked around for a little bit with GetKeyState and the like but still being new I can't find a way around it xD. These may be sufficient for what you need for now, otherwise better to wait for someone more knowledgeable to post.