Winapi - How to achieve equivalent of LWA_COLORKEY for MS Edge browser

Question

My goal is to pick a color that will be transparent on the Microsoft Edge browser (i.e. so we can see the windows behind where that color occurs in Edge). I was able to achieve what I want on Mathematica windows in the past with the following lines of Python code:

win32gui.SetWindowLong(hwnd,win32con.GWL_EXSTYLE,
                       win32gui.GetWindowLong(hwnd,win32con.GWL_EXSTYLE) | win32con.WS_EX_LAYERED)
winxpgui.SetLayeredWindowAttributes(hwnd,win32api.RGB(*transparentColor),50,win32con.LWA_COLORKEY)

where hwnd is the window's handle and transparentColor is the RGB values for the desired color. Now I'm experimenting with Jupyter and wanted to do the same to Edge since Jupyter runs from a browser, but the above is pretty much ignored on Edge (depending on the color, some parts of the menu may become transparent, but not the body of the window). My research so far leads me to believe that this is because Edge (or any other modern browser) does not use GDI and ignores the LWA_COLORKEY request. I tried using LWA_ALPHA instead of LWA_COLORKEY and that does work (makes the whole window semi-transparent), but this is not the effect I'm looking for. I get the same results if I use win32gui.SetLayeredWindowAttributes.

Is it possible in any way for an external application to achieve the intent of the code above for the Edge browser application? From Rita Han's research, in her answer and comments, it is clearly possible, since it works for her. The question is why doesn't it work on my systems (I've tried on two computers).

Updated following a request for more information:

My version of Edge was 44.18362.449.0 but I have now updated it to 81.0.416.72 with the same results. The screen capture below shows two windows, the one on the left is MS Paint where I created an image called "almostBlack_010000.png" and the one on the right is Edge displaying the same image. I have set WS_EX_LAYERED and LWA_COLORKEY on both of these windows using the code above, using color (1,0,0) as the key (i.e. very dark red). As you can see, the Paint window shows my desktop through the "black" rectangle, whereas Edge does not.

A few additional notes:

I have also tried the with Internet Explorer and Chrome with the same negative results.
The transparent area on MS Paint is click through, i.e. clicking in that area is like clicking on the desktop or any window that happens to be behind.
When I use the RGB color (0,0,0) as the key, the whole window becomes click through. This happens with Edge as well, but black pixels still do not become transparent.
The code above would throw a Python traceback if there was an error. For example, if I provide a non-existing window handle in the call to SetLayeredWindowAttributes, I get a traceback with the following last line: error: (1400, 'SetLayeredWindowAttributes', 'Invalid window handle.')

I have tried with the c++ code below and I am getting the same results, i.e. I can get the selected color to be transparent in Paint and other applications, but not in Edge.

#include <iostream>
#include <Windows.h>

int main(int argc, char *argv[])
{
    HWND hwnd;
    COLORREF color;

    if (argc != 3)
    {
        printf("Usage: %s <HWND> <color>",argv[0]);
    }
    else
    {
        hwnd = (HWND)strtoul(argv[1],NULL,16);
        color = (COLORREF)strtoul(argv[2],NULL,16); // Note: order is reverse than usual. Use 0xBBGGRR
        printf("Setting window %x to %x", (unsigned int)hwnd, (unsigned int)color);

        if (!SetWindowLong(hwnd, GWL_EXSTYLE, GetWindowLong(hwnd, GWL_EXSTYLE) | WS_EX_LAYERED))
            printf("SetWindowLong error: %d\n", GetLastError());
        if (!SetLayeredWindowAttributes(hwnd,color,50,LWA_COLORKEY))
            printf("SetLayeredWindowsAttributes error: %d\n", GetLastError());
    }

}

I found out that LWA_COLORKEY and LWA_ALPHA can be ORed, in which case both the alpha and colorkey arguments of SetLayeredWindowAttributes can be used simultaneously. I can, therefore, see on the same Edge window that the overall alpha effect is visible, but that the transparency for the key color is not visible.

My Edge version is 81.0.416.72 and it doesn't ignore the LWA_COLORKEY request. Could you show your Edge version and some snapshots like what you expect and what you received? — Rita Han
Thanks Rita. I have updated my post with the information you requested. It's encouraging that it works for you. — user1595706
Thanks for your screen captures and reproduce steps. Can you check my answer to see if it works for you? — Rita Han

Rita Han Rita Han · Accepted Answer · 2020-05-13T07:02:29

I follow your steps, create a black image using MS Paint and run the following code for MS Paint and Edge, both work for me. Make sure you use the right target window handle and add error check for ensuring functions execute successfully.

HWND hwnd = (HWND)0x00070948; // The window handle I found via Spy++ for testing purpose.

if (!SetWindowLong(hwnd, GWL_EXSTYLE, GetWindowLong(hwnd, GWL_EXSTYLE) | WS_EX_LAYERED))
    printf("SetWindowLong error: %d\n", GetLastError());
if (!SetLayeredWindowAttributes(hwnd, RGB(0, 0, 0), 50, LWA_COLORKEY))
    printf("SetLayeredWindowAttributes error: %d\n", GetLastError());

The black image open with MS Paint:

The black image open with Edge and turns to transparent after executing SetLayeredWindowAttributes. From transparent area you can see my desktop (blue).

Winapi - How to achieve equivalent of LWA_COLORKEY for MS Edge browser

2 Answers