You don't write on what OS you have problems with the artefacts after clearing.
At least with desktop composition activated it is a very bad idea to draw directly to the desktop and to do XOR painting (see "Drawing To and Reading From the Screen -- Baaaad!" in this blog post). Apart from the negative performance implications you can't be sure what other painting happens at the same time and what effects and animations alter the displayed content, so a simple XOR may not be enough to completely erase everything.
One possible way to implement it would be a transparent overlay window of desktop size, and to draw your rubber band selector over that. Invalidating the whole window if the size changes should be enough, no need to erase the old selection line. If the overlay is removed the line will be gone too. Desktop composition will make sure that no flicker occurs. However, switching applications while selecting an area will be problematic, you need to catch this and immediately cancel the selection.
Edit:
I just tested it with Delphi 2009, and with the following test app:
- a form with
FormStyle
set to fsStayOnTop
and with AlphaBlend
set to True
- with an overridden
CreateParams()
method to add the WS_EX_TRANSPARENT
extended style flag
I can pass all mouse clicks through to the underlying windows while being able to draw into a window on top of them. This should get you started.