How to handle rotation of selection rectangle that can be resized?

Question

Im my application the user can select areas of an image. The user drags a rectangular area in a Canvas. It looks like the following:

The circle (Ellipse) on top of the selection area, is a handle to rotate the area. See picture below:

When the selection area is not rotated, and the user uses the square shaped handles to resize the area, it works. But when the user rotates the selection area using the ellipse, and then uses a square handle to resize the area, it resizes in both opposite directions (in context of the rotated coordinate system).

My element structure is as follows:

Canvas
+--- Image (covers whole canvas)
+--- Canvas (Background white with opacity as shown in images above, and contains the elements to draw the 
           selection area. I'll call this the 'area canvas')
    +--- Rectangle (left handle)
    +--- Rectangle (top handle)
    +--- Rectangle (right handle)
    +--- Rectangle (bottom handle)
    +--- Ellipse (rotation handle)

I do the rotation using a RotateTransform. The transform is applied the Canvas of the selection area. The RenderTransformOrigin of the canvas is (0.5, 0.5). So the Canvas will always be rotated around it's center.

Imagine the area is rotated for say 10 degrees clockwise. Then you resize it using the handle at the right. Now the area should only expand to the right side. In other words: only 2 corners of the area canvas should be moving, What happens how, is that the left side of the area also moves. So also the corners at the left side move (to the left). I think this is caused by the fact that the change of the width/height of the area canvas also changes the center point. But how to fix it? The behaviour of the selection area should be exactly the same as a selected element in programs like MS Word.

Link to Github repository containing the relevant code: https://github.com/websitetest/selection

Answer 1

The solution is to add a translation which cancels the move of the center of canvas after a rotation: so i have done some modification to you method UpdateResizeArea (mathematic solution)

    private void UpdateResizeArea(Point currentMousePoint)
    {
           :
           :
        switch (_activeResizeAreaSide)
        {
            case SelectionArea.ResizeSide.LEFT:
                {
                    _activeResizeArea.OffsetX = _activeResizeAreaXbeforeStart + deltaX;
                    _activeResizeArea.AreaWidth = _activeResizeAreaWidthbeforeStart - deltaX;
                    var tg = new TransformGroup();
                    var rad = _activeResizeArea.Rotation * Math.PI / 180;
                    var cx = _activeResizeArea.CalculateCenterPoint().X - _activeRotateAreaCenterPointBeforeStart.X;
                    var cy = _activeResizeArea.CalculateCenterPoint().Y - _activeRotateAreaCenterPointBeforeStart.Y;

                    var tx = cx * Math.Sin(rad / 2d) * Math.Sin(rad / 2d) * 2d + cy * Math.Sin(rad);
                    var ty = cx * Math.Sin(rad) - cy * Math.Sin(rad / 2d) * Math.Sin(rad / 2d) * 2d;

                    var t = new TranslateTransform(-tx, ty);

                    tg.Children.Add(_activeResizeArea.RotateTransform);
                    tg.Children.Add(t);

                    _activeResizeArea.AreaCanvas.RenderTransform = tg;
                }
                break;
            case SelectionArea.ResizeSide.TOP:
                {
                    _activeResizeArea.OffsetY = _activeResizeAreaYbeforeStart + deltaY;
                    _activeResizeArea.AreaHeight = _activeResizeAreaHeightbeforeStart - deltaY;

                    var tg = new TransformGroup();
                    var rad = _activeResizeArea.Rotation * Math.PI / 180;

                    var cx = _activeResizeArea.CalculateCenterPoint().X - _activeRotateAreaCenterPointBeforeStart.X;
                    var cy = _activeResizeArea.CalculateCenterPoint().Y - _activeRotateAreaCenterPointBeforeStart.Y;

                    var tx = cy * Math.Sin(rad) + cx * Math.Sin(rad / 2d) * Math.Sin(rad / 2d) * 2d;
                    var ty = cy * Math.Sin(rad / 2d) * Math.Sin(rad / 2d) * 2d - cx * Math.Sin(rad);

                    var t = new TranslateTransform(-tx, -ty);

                    tg.Children.Add(_activeResizeArea.RotateTransform);
                    tg.Children.Add(t);
                    _activeResizeArea.AreaCanvas.RenderTransform = tg;
                }
                break;
            case SelectionArea.ResizeSide.RIGHT:
                {
                    _activeResizeArea.AreaWidth = _activeResizeAreaWidthbeforeStart + deltaX;

                    var tg = new TransformGroup();
                    var rad = _activeResizeArea.Rotation * Math.PI / 180;
                    var cx = _activeResizeArea.CalculateCenterPoint().X - _activeRotateAreaCenterPointBeforeStart.X;
                    var cy = _activeResizeArea.CalculateCenterPoint().Y - _activeRotateAreaCenterPointBeforeStart.Y;

                    var tx = cx * Math.Sin(rad / 2d) * Math.Sin(rad / 2d) * 2d + cy * Math.Sin(rad);
                    var ty = cx * Math.Sin(rad) - cy * Math.Sin(rad / 2d) * Math.Sin(rad / 2d) * 2d;

                    var t = new TranslateTransform(-tx, ty);

                    tg.Children.Add(_activeResizeArea.RotateTransform);
                    tg.Children.Add(t);
                    _activeResizeArea.AreaCanvas.RenderTransform = tg;
                }
                break;
            case SelectionArea.ResizeSide.BOTTOM:
                {
                    _activeResizeArea.AreaHeight = _activeResizeAreaHeightbeforeStart + deltaY;

                    var tg = new TransformGroup();
                    var rad = _activeResizeArea.Rotation * Math.PI / 180;

                    var cx = _activeResizeArea.CalculateCenterPoint().X - _activeRotateAreaCenterPointBeforeStart.X;
                    var cy = _activeResizeArea.CalculateCenterPoint().Y - _activeRotateAreaCenterPointBeforeStart.Y;

                    var tx = cy * Math.Sin(rad) + cx * Math.Sin(rad / 2d) * Math.Sin(rad / 2d) * 2d;
                    var ty = cy * Math.Sin(rad / 2d) * Math.Sin(rad / 2d) * 2d - cx * Math.Sin(rad);

                    var t = new TranslateTransform(-tx, -ty);

                    tg.Children.Add(_activeResizeArea.RotateTransform);
                    tg.Children.Add(t);
                    _activeResizeArea.AreaCanvas.RenderTransform = tg;
                }
                break;
        }

you could adapt the solution to your project.. because i have just added some lines of coding without changing the logic of your program

__________________________________________________________________________________

Another solution (without math calculus) is to precalculate the difference betwwen the position of one point of canvas with the initial Center of rotation and the new position of same point with the new center:

        switch (_activeResizeAreaSide)
        {
            case SelectionArea.ResizeSide.LEFT:
                {
                    _activeResizeArea.OffsetX = _activeResizeAreaXbeforeStart + deltaX;
                    _activeResizeArea.AreaWidth = _activeResizeAreaWidthbeforeStart - deltaX;

                    var ir = new RotateTransform(_activeResizeArea.Rotation, _activeRotateAreaCenterPointBeforeStart.X, _activeRotateAreaCenterPointBeforeStart.Y);
                    var fr = new RotateTransform(_activeResizeArea.Rotation, _activeResizeArea.CalculateCenterPoint().X, _activeResizeArea.CalculateCenterPoint().Y);
                    var ip = ir.Transform(new Point(_activeResizeArea.OffsetX, _activeResizeArea.OffsetY));
                    var fp = fr.Transform(new Point(_activeResizeArea.OffsetX, _activeResizeArea.OffsetY));

                    var tg = new TransformGroup();

                    var txx = ip.X - fp.X;
                    var tyy = ip.Y - fp.Y;

                    var t = new TranslateTransform(txx, tyy);

                    tg.Children.Add(_activeResizeArea.RotateTransform);
                    tg.Children.Add(t);

                    _activeResizeArea.AreaCanvas.RenderTransform = tg;
                }
                break;
            case SelectionArea.ResizeSide.TOP:
                {
                    _activeResizeArea.OffsetY = _activeResizeAreaYbeforeStart + deltaY;
                    _activeResizeArea.AreaHeight = _activeResizeAreaHeightbeforeStart - deltaY;

                    var ir = new RotateTransform(_activeResizeArea.Rotation, _activeRotateAreaCenterPointBeforeStart.X, _activeRotateAreaCenterPointBeforeStart.Y);
                    var fr = new RotateTransform(_activeResizeArea.Rotation, _activeResizeArea.CalculateCenterPoint().X, _activeResizeArea.CalculateCenterPoint().Y);
                    var ip = ir.Transform(new Point(_activeResizeArea.OffsetX, _activeResizeArea.OffsetY));
                    var fp = fr.Transform(new Point(_activeResizeArea.OffsetX, _activeResizeArea.OffsetY));

                    var tg = new TransformGroup();

                    var txx = ip.X - fp.X;
                    var tyy = ip.Y - fp.Y;

                    var t = new TranslateTransform(txx, tyy);

                    tg.Children.Add(_activeResizeArea.RotateTransform);
                    tg.Children.Add(t);
                    _activeResizeArea.AreaCanvas.RenderTransform = tg;
                }
                break;
            case SelectionArea.ResizeSide.RIGHT:
                {
                    _activeResizeArea.AreaWidth = _activeResizeAreaWidthbeforeStart + deltaX;

                    var ir = new RotateTransform(_activeResizeArea.Rotation, _activeRotateAreaCenterPointBeforeStart.X, _activeRotateAreaCenterPointBeforeStart.Y);
                    var fr = new RotateTransform(_activeResizeArea.Rotation, _activeResizeArea.CalculateCenterPoint().X, _activeResizeArea.CalculateCenterPoint().Y);
                    var ip = ir.Transform(new Point(_activeResizeArea.OffsetX, _activeResizeArea.OffsetY));
                    var fp = fr.Transform(new Point(_activeResizeArea.OffsetX, _activeResizeArea.OffsetY));

                    var tg = new TransformGroup();

                    var txx = ip.X - fp.X;
                    var tyy = ip.Y - fp.Y;

                    var t = new TranslateTransform(txx, tyy);

                    tg.Children.Add(_activeResizeArea.RotateTransform);
                    tg.Children.Add(t);
                    _activeResizeArea.AreaCanvas.RenderTransform = tg;
                }
                break;
            case SelectionArea.ResizeSide.BOTTOM:
                {
                    _activeResizeArea.AreaHeight = _activeResizeAreaHeightbeforeStart + deltaY;

                    var ir = new RotateTransform(_activeResizeArea.Rotation, _activeRotateAreaCenterPointBeforeStart.X, _activeRotateAreaCenterPointBeforeStart.Y);
                    var fr = new RotateTransform(_activeResizeArea.Rotation, _activeResizeArea.CalculateCenterPoint().X, _activeResizeArea.CalculateCenterPoint().Y);
                    var ip = ir.Transform(new Point(_activeResizeArea.OffsetX, _activeResizeArea.OffsetY));
                    var fp = fr.Transform(new Point(_activeResizeArea.OffsetX, _activeResizeArea.OffsetY));

                    var tg = new TransformGroup();

                    var txx = ip.X - fp.X;
                    var tyy = ip.Y - fp.Y;

                    var t = new TranslateTransform(txx, tyy);

                    tg.Children.Add(_activeResizeArea.RotateTransform);
                    tg.Children.Add(t);
                    _activeResizeArea.AreaCanvas.RenderTransform = tg;
                }
                break;
        }

Result: