How to convert new Canvas ui image position to linerenderer position?

Question

I have an image UI in a canvas with Screen Space - Camera render mode. What I like to do is move my LineRenderer to the image vertical position by looping through all the LineRenderer positions and changing its y axis. My problem is I cant get the correct position of the image that the LineRenderer can understand. I've tried using ViewportToWorldPoint and ScreenToWorldPoint but its not the same position.

    Vector3 val = Camera.main.ViewportToWorldPoint(new Vector3(image.transform.position.x, image.transform.position.y, Camera.main.nearClipPlane));

    for (int i = 0; i < newListOfPoints.Count; i++)
    {
        line.SetPosition(i, new Vector3(newListOfPoints[i].x, val.y, newListOfPoints[i].z));
    }

Screenshot result using Vector3 val = Camera.main.ScreenToWorldPoint(new Vector3(image.transform.localPosition.x, image.transform.localPosition.y, -10));

The green LineRenderer is the result of changing the y position. It should be at the bottom of the square image.

Can you add a screenshot of how your current code renders? Matching things up between screenspace and worldspace are tricky things to do and it helps to see what's going on. — Draco18s no longer trusts SE
How about printing out the screen space position of your image? Maybe it is not at the position that you think it should be. Another thing to try could be to see what the (0, 0) or (100, 100) is transformed into. — YAC
@YAC The real problem is that overlay canvas objects use an entirely different coordinate system from worldspace...and viewport space. — Draco18s no longer trusts SE
@unice, if your problem is solved, please accept the answer so that it will be useful to others. — Programmer
@Programmer, sorry for the late reply. Been testing the script and it works perfectly in Canvas with render mode Screen Space - Overlay. But when I use Screen Space - Camera and drag my main camera to the render camera, the linerenderer's position is different. But this is enough to answer my question, I will stick to overlay for now. Thanks Draco18s for the time and effort to answer my question :) — unice

Draco18s no longer trusts SE Draco18s no longer trusts SE · Accepted Answer · 2018-01-07T05:38:44

Wow, this was annoying and complicated.

Here's the code I ended up with. The code in your question is the bottom half of the Update() function. The only thing I changed is what was passed into the ScreenToWorldPoint() method. That value is calculated in the upper half of the Update() function.

The RectTransformToScreenSpace() function was adapted from this Unity Answer post¹ about getting the screen space coordinates of a RectTransform (which is exactly what we want in order to convert from screen space coordinates back into world space!) The only difference is that I was getting inverse Y values, so I changed from Screen.height - transform.position.y to just transform.position.y which did the trick perfectly.

After that it was just a matter of grabbing that rectangle's lower left corner, making it a Vector3 instead of a Vector2, and passing it back into ScreenToWorldPoint(). The only trick there was because of the perspective camera, I needed to know how far away the line was from the camera originally in order to maintain that same distance (otherwise the line moves up and down the screen faster than the image). For an orthographic camera, this value can be anything.

void Update () {
    //the new bits:
    float dist = (Camera.main.transform.position - newListOfPoints[0]).magnitude;
    Rect r = RectTransformToScreenSpace((RectTransform)image.transform);
    Vector3 v3 = new Vector3(r.xMin, r.yMin, dist);

    //more or less original code:
    Vector3 val = Camera.main.ScreenToWorldPoint(v3);
    for(int i = 0; i < newListOfPoints.Count; i++) {
        line.SetPosition(i, new Vector3(newListOfPoints[i].x, val.y, newListOfPoints[i].z));
    }
}

//helper function:
public static Rect RectTransformToScreenSpace(RectTransform transform) {
    Vector2 size = Vector2.Scale(transform.rect.size, transform.lossyScale);
    Rect rect = new Rect(transform.position.x, transform.position.y, size.x, size.y);
    rect.x -= (transform.pivot.x * size.x);
    rect.y -= ((1.0f - transform.pivot.y) * size.y);
    return rect;
}

¹And finding that post from a generalized search on "how do I get the screen coordinates of a UI object" was not easy. A bunch of other posts came up and had some code, but none of it did what I wanted (including converting screen space coordinates back into world space coordinates of the UI object which was stupid easy and not reversibe, thanks RectTransformUtility!)

How to convert new Canvas ui image position to linerenderer position?

1 Answers