How to extract colors from NM_CUSTOMDRAW message in Win32?

Question

I have a hooked dll with WH_CALLWNDPROC from a x86 program (MSVS 2017). The code goes as follows:

extern "C" __declspec(dllexport) LRESULT SysMessageProc(int nCode, WPARAM wParam, LPARAM lParam)
{
if (nCode < 0) return CallNextHookEx(NULL, nCode, wParam, lParam);

    if (nCode == HC_ACTION) {

        CWPSTRUCT* pCWP = (CWPSTRUCT*)lParam;

        HWND hWnd = pCWP->hwnd;

        char wclass[256]; wclass[0] = 0;
        if (GetClassNameA(hWnd, wclass, 255) != 0) {
            
            if (pCWP->message == WM_NOTIFY && (strcmp(wclass, "SysHeader32") == 0 || strcmp(wclass, "SysListView32") == 0)) {

                // LPNMLISTVIEW pnm = (LPNMLISTVIEW)pCWP->lParam;
                // pnm->lParam is always 0

                NMHDR* hdr = (NMHDR*)pCWP->lParam;

                if ((int)hdr->code == NM_CUSTOMDRAW) {

                    char out[256]; out[0] = 0;
                    sprintf_s(out, 255, 
                        "class: '%s', hWnd: %08X, msg: %08X, ptr: %08X, HDR[%08X, %i, %i]\n",
                        wclass, (UINT)hWnd, pCWP->message, (UINT)pCWP->lParam,
                        (UINT)hdr->hwndFrom, hdr->idFrom, (int)hdr->code);

                    FILE *stream;
                    if (fopen_s(&stream, "F:\\tmp\\_dll\\out.txt", "a+") == 0) {
                        fprintf(stream, out);
                        fclose(stream);
                    }

                   // how to get
                   // LPNMLVCUSTOMDRAW lplvcd = ???;
                   // lplvcd->clrFace, lplvcd->clrText, lplvcd->clrTextBk ???
                   

               }

          }

     }

When I record messages I get the same results as in Spy++:

It seems strange that hdr->code is UINT and NM_CUSTOMDRAW = -12 so I have to convert to (int) but the result matches the output from Spy++.
Initially I thought that I can do this:

LPNMLISTVIEW pnm = (LPNMLISTVIEW)pCWP->lParam;
...
LPNMLVCUSTOMDRAW lplvcd = (LPNMLVCUSTOMDRAW)pnm->lParam;
...
print(lplvcd->clrFace, lplvcd->clrText, lplvcd->clrTextBk);

but pnm->lParam is always zero; pCWP->wParam is also zero.

How to I get to LPNMLISTVIEW and LPNMLVCUSTOMDRAW structures ?

EDIT:

CWPSTRUCT* pCWP = (CWPSTRUCT*)lParam;
HWND hWnd = pCWP->hwnd;
UINT msg = pCWP->message;

char wclass[256]; wclass[0] = 0;
if (RealGetWindowClassA(hWnd, wclass, 255) != 0) {
                
    if (msg == WM_NOTIFY && (strcmp(wclass, "SysHeader32") == 0 || strcmp(wclass, "SysListView32") == 0)) {

        NMHDR* hdr = (NMHDR*)pCWP->lParam;

        if (hdr->code == NM_CUSTOMDRAW) {
            
            if (strcmp(wclass, "SysListView32") == 0) {

                LPNMLVCUSTOMDRAW lplvcd = (LPNMLVCUSTOMDRAW)pCWP->lParam;
                
                if (lplvcd) {
                    RECT r = lplvcd->rcText;
                    char out[512]; out[0] = 0;
                    sprintf_s(out, 511, 
                    "class: '%s', hWnd: %08X, msg: %08X, ptr: %08X, \
draw: %08X, rect: (%i, %i)-(%i, %i), face: %08X, txt: %08X, txtBg: %08X\n",
                        wclass, (UINT)hWnd, msg, (UINT)lplvcd, 
                        lplvcd->nmcd.dwDrawStage, r.left, r.top, r.right, r.bottom, 
                        lplvcd->clrFace, lplvcd->clrText, lplvcd->clrTextBk);
                    
                    FILE *stream;
                    if (fopen_s(&stream, "F:\\tmp\\_dll\\out.txt", "a+") == 0) {
                        fprintf(stream, out);
                        fclose(stream);
                    }
                }
            }

Example output:

class: 'SysListView32', hWnd: 000A04F4, msg: 0000004E, ptr: 02E3F33C, draw: 00000001, rect: (0, 1)-(-1241442379, 48493508), face: FFB617B5, txt: 02E3F400, txtBg: 6F010726

FYI, GetClassName() may not return what you want for subclassed controls. Consider using RealGetWindowClass() instead. See What makes RealGetWindowClass so much more real than GetClassName?. — Remy Lebeau

Remy Lebeau Remy Lebeau · Accepted Answer · 2021-01-08T17:52:51

You are accessing NMLVCUSTOMDRAW from the wrong lParam.

In a ListView's NM_CUSTOMDRAW notification, the wParam is always 0, and the lParam is a NMLVCUSTOMDRAW*, but you are casting that to NMLISTVIEW* instead, which is wrong. There is no NMLISTVIEW provided in NM_CUSTOMDRAW. And even if there were, the NMLISTVIEW::lParam field contains the ListView item's user-defined data, as set by the application via LVM_SETITEM/ListView_SetItem(). So that is not what you want anyway.

You need to cast pCWP->lParam directly to NMLVCUSTOMDRAW* instead. And only for a SysListView32 control. A SysHeader32 control uses NMCUSTOMDRAW* instead, eg:

if (pCWP->message == WM_NOTIFY) {
    ...
    NMHDR* hdr = (NMHDR*)pCWP->lParam;
    ...
    if (hdr->code == NM_CUSTOMDRAW) {
        if (strcmp(wclass, "SysListView32") == 0) {
            LPNMLVCUSTOMDRAW lplvcd = (LPNMLVCUSTOMDRAW)pCWP->lParam;
            // use lplvcd as needed...
        }
        else if (strcmp(wclass, "SysHeader32") == 0) {
            LPNMCUSTOMDRAW lpcd = (LPNMCUSTOMDRAW)pCWP->lParam;
            // use lpcd as needed...
        }
        ...
    }
}

The documentation for the general-purpose NM_CUSTOMDRAW shows which struct is used by which type of custom-drawable control. There are 5 possibilities:

How to extract colors from NM_CUSTOMDRAW message in Win32?

1 Answers