PART 1: What Makes Resize Look Good or Bad?
There is so much ambiguity and unclarity in StackOverflow questions about smooth resize that we need to establish a common vocabulary to help people make their answers more clear.
That is what we will do in this section.
To keep things simple, we going to explain the problems of smooth resizing in the horizontal dimension only, but everything here applies to vertical resizing just the same.
Below we will refer to a window's
"non-client area:" the part of the window that Windows manages, including the title bar at the top and window borders around all edges, and
"client area:" the main part of the window that you are responsible for
Suppose you have an app with:
- a button or label L that is supposed to remain flush-left
- a button or label R that is supposed to remain flush-right
no matter how the window gets resized.
Your app might draw L/R itself (e.g. using GDI/OpenGL/DirectX inside the one window) or L/R might be some Microsoft control (which would have its own HWND separate from your main window HWND); doesn't matter.
Here is a simplified representation of the client area of your app window. As you can see, we have three-column-wide LLL at the far left of the client area, and three-column-wide RRR at the far right of the client area, with various other client area content represented by "-" in between (please ignore the grey background that StackOverflow insists on adding; L and R are at the far left and right edges of your client area):
LLL-----------RRR
Now imagine that you grab the left or right border of this window and drag it to make the window bigger or smaller.
1a. Easy Case: Drawing On Time
Imagine that your app is very fast at drawing so that it can always respond to the user's dragging action in 1 millisecond, and the OS lets your app draw that quickly without trying to draw anything else on the screen to "help" you.
As you drag the app border, the user sees the following on-screen (with each line of these figures representing one instant of time):
Dragging right border to the right (enlarging width):
(Figure 1a-1)
LLL-----------RRR (initially, when you click the mouse)
LLL------------RRR (as you drag the mouse)
LLL-------------RRR (as you drag the mouse)
LLL--------------RRR (when you release the mouse)
Dragging right border to the left (shrinking width):
(Figure 1a-2)
LLL-----------RRR
LLL----------RRR
LLL---------RRR
LLL--------RRR
Dragging left border to the left (enlarging width):
(Figure 1a-3)
LLL-----------RRR
LLL------------RRR
LLL-------------RRR
LLL--------------RRR
Dragging left border to the right (shrinking width):
(Figure 1a-4)
LLL-----------RRR
LLL----------RRR
LLL---------RRR
LLL--------RRR
These all look good and smooth:
- When adjusting the right border, R appears to move at a constant speed in one direction and L stays still as it should.
- When adjusting the left border, L appears to move at a constant speed in one direction and R stays still as it should.
So far so good.
1b. Hard Case: Drawing Falls Behind
Now, imagine that your app is so slow at drawing that the app cannot keep up with you as you drag with the mouse. Yes, eventually, your drawing will catch up, but we are talking about what happens during the time that you are dragging the mouse with your hand. Obviously the computer cannot reach out and grab your hand to slow your mouse movement down, so the key questions are:
- what should show on the screen during this period, and
- who decides what should show?
For example, when dragging the right border to the right (enlarging width):
(Figure 1b-1)
LLL-----------RRR
?????????????????? (what should show here?)
??????????????????? (what should show here?)
LLL--------------RRR (app catches up)
As another example, when dragging the left border to the left (shrinking width):
(Figure 1b-2)
LLL-----------RRR
???????????????? (what should show here?)
??????????????? (what should show here?)
LLL--------RRR (app catches up)
These turn out to be the key questions that determine whether the motion looks smooth or not, and they are the key questions around which this whole StackOverflow question revolves.
Different versions of Windows provide different answers to these questions in different contexts, meaning that the solution to getting smoother resize depends on which situation you are in.
1c. Temporary Solutions While Waiting for App to Draw
There are several choices of what to do in the period after the user has begun to drag the mouse to resize the window, but before your app has caught up by drawing the window at the new size.
1c1. Do Nothing
The screen could remain exactly as it is until the app catches up (neither your client pixels nor even the window border in the non-client area changes):
Example when dragging the right border to the right (enlarging width):
(Figure 1c1-1)
LLL-----------RRR
LLL-----------RRR
LLL-----------RRR
LLL--------------RRR (app catches up)
Example when dragging the left border to the left (shrinking width):
(Figure 1c1-2)
LLL-----------RRR
LLL-----------RRR
LLL-----------RRR
LLL--------RRR (app catches up)
The obvious disadvantage of this method is that during the period in question, the app appears to have "hung" and appears to be unresponsive to your mouse movements, because neither the R nor the '-' nor the L nor the the window border is moving.
Microsoft is often picked on for Windows being an unresponsive OS (and it's sometimes their fault and sometimes the fault of the app developer), so ever since Microsoft introduced live-resize (Windows XP?), Microsoft never uses the "do nothing" method by itself.
The "do nothing" method is annoying for the user and looks unprofessional, but it turns out (very non-obviously) that it's not always the worst choice. Read on...
1c2. Scale Content
Another possibility is that Windows could always make the window border follow your mouse movements instantly (because Windows itself has enough processing power to at least draw the non-client area in a timely manner), and while it is waiting for your app, Windows could take the old pixels of the client area and scale those pixels up or down just like when you zoom/blow up an image so that they "fit" in the smaller or bigger space.
This technique is generally worse than any other technique because it will result in a blurry image of your original content that is likely to be out of proportion. So nobody should ever do this in any case. Except, as we will see in PART 2, sometimes Microsoft does.
1c3. When Enlarging, Fill in Some Background Color
Another technique that could work when enlarging a window is the following: Windows could always make the window border follow your mouse movements instantly, and Windows could fill in new pixels of the now-larger client area with some temporary background color B:
For example, when dragging the right border to the right (enlarging width):
(Figure 1c3-1)
LLL-----------RRR
LLL-----------RRRB
LLL-----------RRRBB
LLL--------------RRR (app catches up)
This method has the advantage that during the period in question, at least your window border is moving, so the app feels responsive.
Another nice feature is that during the drag, L stays still, just like it should.
It's a little weird that the new space you are creating as you drag gets filled in with some random color, and even more weird that R doesn't actually move until later (notice that R jerks rightward by 3 columns at the last instant), but at least R only moves in the correct direction. It's a partial improvement.
A huge and important question is: what color should the newly filled-in background color B be? If B happens to be black and your app happens to have a mostly white background, or vice versa, it's going to be much uglier than if B matches your existing content's background color. As we will see in PART 2, Windows has deployed several different strategies to improve the choice of B.
Now consider the same idea, but instead apply it to the case where we are dragging the left border to the left (enlarging width).
The logical thing would be to fill in the new background color on the left side of the window:
(Figure 1c3-2)
LLL-----------RRR
BLLL-----------RRR
BBLLL-----------RRR
LLL--------------RRR (app catches up)
This would be logical because R would stay put, just like it should. L would have the same weirdness we described along with Figure 1c3-1 above (L would hold still and then jerk 3 columns leftward all of a sudden at the last instant), but at least L would only move in the correct direction.
However---and this is going to really come as a shock---in several important cases that you have to deal with, Windows does not do the logical thing.
Instead, Windows sometimes fills in background pixels B on the right even if you are dragging the left window border:
(Figure 1c3-3)
LLL-----------RRR
LLL-----------RRRB
LLL-----------RRRBB
LLL--------------RRR (app catches up)
Yes, this is insane.
Consider how this looks to the user:
L appears to move very smoothly at a constant speed in one direction, so that is actually good, but
Just look at what R is doing:
RRR
RRR
RRR
RRR (app catches up)
- R first moves to the left by two columns, which it should not do: R is supposed to stay flush-right at all times
- R then snaps back to the right again. Holy crap!
This looks horrible, terrible, abysmal, disgusting, ... there are not even words to describe how bad this looks.
The human eye is extremely sensitive to motion, even motion that happens over just a few frames of time. Our eye instantly picks up on this bizarre back-and-forth motion of R and we immediately know something is seriously wrong.
So here you can begin to get a sense of why some of these ugly resize problems only happen when you drag the left (or top) border and not the right (or bottom) border.
In reality, both cases (Figure 1c3-2 vs. Figure 1c3-3) do something weird. In Figure 1c3-2 we temporarily add some background pixels B that do not belong there. But this weird behavior is much less noticeable than the back-and-forth motion of Figure 1c3-3.
This back-and-forth motion is the jitter/flicker/jumping that so many StackOverflow questions are about.
So any solution to the problem of smooth resize has to:
at least prevent items in your client area from appearing to jump in one direction then back the other direction.
ideally also avoid the need to add background pixels B, if possible
1c4. When Shrinking, Cut Off Some Pixels
Section 1c3 dealt with expanding the window. If we look at shrinking the window, we will see there is an exactly analogous set of cases.
A technique that could work when shrinking a window is the following: Windows could always make the window border follow your mouse movements instantly, and Windows could simply chop off (crop) some pixels of your now-smaller client area.
For example, when dragging the right border to the left (shrinking width):
(Figure 1c4-1)
LLL-----------RRR
LLL-----------RR
LLL-----------R
LLL--------RRR (app catches up)
With this technique, L stays put as it should, but a weird thing happens on the right: R, which is supposed to stay flush-right no matter what the window size, appears to get its right edge incrementally sliced off by the right edge of the client area until R disappears, and then all of a sudden R reappears at its correct position when the app catches up. This is very weird, but keep in mind that at no point does R appear to be moving to the right. R's left edge appears to stay still, until the last moment when all of R jumps back 3 columns leftward. So, like we saw in Figure 1c3-1, R only moves in the correct direction.
Now consider what happens when we drag the left border to the right (shrinking width).
The logical thing to do would be to shave pixels off the left of the client area:
(Figure 1c4-2)
LLL-----------RRR
LL-----------RRR
L-----------RRR
LLL--------RRR (app catches up)
This would have the same weird properties as Figure 1c4-1, just with the roles of left and right reversed. L would appear to get incrementally shaved off from L's left edge but L's right edge would remain still until at the last instant L appears to jump to the right. So L only moves in the correct direction, albeit abruptly.
But---yes, get ready for total shock again---in several important cases that you have to deal with, Windows does not do the logical thing.
Instead, Windows sometimes chops pixels off of the right even if you are dragging the left window border:
(Figure 1c4-3)
LLL-----------RRR
LLL-----------RR
LLL-----------R
LLL--------RRR (app catches up)
Consider how this looks to the user:
L appears to move very smoothly at a constant speed in one direction, so that is actually good, but
Just look at what R is doing:
RRR
RR
R
RRR (app catches up)
- R first slides over to the right by two columns. R's left edge appears to move rightward along with the rest of R.
- R then snaps back to the left again.
As you should now be aware of after reading section 1c3, this back-and-forth motion looks absolutely horrible and is much worse than the admittedly weird behavior of Figure 1c4-1 and Figure 1c4-2.
1c5. Wait a Bit, Then Try One of Above
So far we have presented separate ideas for what to do when the user has begun to drag the window borders but the app hasn't redrawn yet.
These methods can actually be combined.
For a moment, try to think of this problem from Microsoft's point of view. At the moment that user starts dragging the mouse to resize your window, Microsoft has no way of knowing ahead of time how long it will take your app to draw. So Microsoft has to strike a balance:
if your app is going to respond quickly, then any changes Microsoft makes to the screen are going to make your app look worse than if Microsoft just lets you draw the real content (remember, all the tricks above are weird to varying degrees and will make your content appear strangely, so not using any of those tricks is definitely better).
but if Microsoft waits for you to draw for too long, your app (and Windows by extension) will look hangy and unresponsive as we explained in Section 1c1. This makes Microsoft lose face even if it's your fault.
So, another option is to first hold off on any screen changes and give the app a certain amount of time to draw, and if the app fails to meet the deadline, then employ one of the methods above to temporarily "fill in the gap."
Does this sound horrible and hacky to you? Guess what? That's what Windows does, in at least 2 different ways simultaneously with 2 different deadline times. PART 2 will dive into these cases...
