How does TMemo eat an escape key, when TEdit doesn't?

Question

I'm trying to stop a TMemo (and also TRichEdit) control from eating Escape keys.

If the user is focused in a TEdit, pressing Escape will trigger the form to do what the form does when the user presses escape. If the user is focused in a TMemo, pressing escape is eaten by the TMemo.

Of course i could do the hack:

procedure TForm1.Memo1KeyPress(Sender: TObject; var Key: Char);
begin
    if Key = #27 then
    begin
       //figure out how to send a key to the form
    end;
end;

But that is not ideal (i have to handle the escape key, rather than letting the form handle it).

Of course i could do the hack:

Form1.KeyPreview := True;

procedure TForm1.FormKeyPress(Sender: TObject; var Key: Char);
begin
   if Key = #27 then
   begin
      //Figure out how to invoke what the form was going to do when the user presses escape
   end;
end;

But that is not ideal (i have to handle the escape key, rather than letting the form handle it).

So we'll answer the question rather than the problem

Instead we'll take this opportunity to learn something. How is it that a TMemo is even receiving a keyPress event associated with the escape key, when a TEdit doesn't:

procedure TForm1.Edit1KeyPress(Sender: TObject; var Key: Char);
begin
    if Key = #27 then
    begin
        //never happens
    end;
end;

The TEdit and TMemo are the same Windows EDIT common control.

Why does escape bypass the form's KeyPreview

If i turn on the form's KeyPreview, and the user presses Escape while focused in a TEdit box, and a button's Cancel property is set, the form closes and:

• the Edit1.KeyPress event is not triggered
• the Form1.KeyPress event is not triggered

If an Action is created, whose Shortcut is Esc, then no KeyPress event is raised, no matter what control the user is focused in.

tl;dr: Where is the TMemo.WantEscape property?

Answer 1

The behaviour you observe is controlled by the handling of the WM_GETDLGCODE message. For a memo that looks like this:

procedure TCustomMemo.WMGetDlgCode(var Message: TWMGetDlgCode);
begin
  inherited;
  if FWantTabs then Message.Result := Message.Result or DLGC_WANTTAB
  else Message.Result := Message.Result and not DLGC_WANTTAB;
  if not FWantReturns then
    Message.Result := Message.Result and not DLGC_WANTALLKEYS;
end;

For an edit control the VCL does not implement special handling for WM_GETDLGCODE and the underlying Windows edit control handles it.

In a standard Win32 app the Windows dialog manager sends the WM_GETDLGCODE messages. But Delphi is not built on top of the dialog manager, and so the VCL is in charge of sending WM_GETDLGCODE. It does so in the CN_KEYDOWN handler. The code looks like this:

Mask := 0;
case CharCode of
  VK_TAB:
    Mask := DLGC_WANTTAB;
  VK_LEFT, VK_RIGHT, VK_UP, VK_DOWN:
    Mask := DLGC_WANTARROWS;
  VK_RETURN, VK_EXECUTE, VK_ESCAPE, VK_CANCEL:
    Mask := DLGC_WANTALLKEYS;
end;
if (Mask <> 0) and
  (Perform(CM_WANTSPECIALKEY, CharCode, 0) = 0) and
  (Perform(WM_GETDLGCODE, 0, 0) and Mask = 0) and
  (GetParentForm(Self).Perform(CM_DIALOGKEY,
  CharCode, KeyData) <> 0) then Exit;

Notice that VK_RETURN, VK_EXECUTE, VK_ESCAPE and VK_CANCEL are all lumped together. This means that a VCL control has to decide whether or not to process these keys itself, or let the form handle them in its CM_DIALOGKEY handler.

As you can see from TCustomMemo.WMGetDlgCode you can influence that choice with the WantReturns property. So, you can persuade the VCL to let the form handle ESC by simply setting WantReturns on the memo to False. But that also stops the ENTER key reaching memo and makes it rather tricky for the user of the memo to enter new lines. They have to do it with CTRL + ENTER.

In fact WantReturns should really have been named WantReturnsAndEscapesAndExecutesAndCtrlBreaks. The VCL designers could have implemented a WantEscapes property but it's just not there.

So you are left handling it yourself one way or another. Personally, I do so with my own derived memo control. It overrides the KeyDown method and does this:

procedure TMyMemo.KeyDown(var Key: Word; Shift: TShiftState);
var
  Form: TCustomForm;
  Message: TCMDialogKey;
begin
  inherited;
  if (Key=VK_ESCAPE) and (Shift*[ssShift..ssCtrl])=[]) then begin
    Form := GetParentForm(Self);
    if Assigned(Form) then begin
      // we need to dispatch this key press to the form so that it can 'press' 
      // any buttons with Cancel=True
      Message.Msg := CM_DIALOGKEY;
      Message.CharCode := VK_ESCAPE;
      Message.KeyData := 0;
      Message.Result := 0;
      Form.Dispatch(Message);
    end;
  end;
end;

Another way to achieve this is to handle CM_WANTSPECIALKEY and WM_GETDLGCODE. Here's a crude interposer that illustrates the technique:

type
  TMemo = class(StdCtrls.TMemo)
  protected
    procedure CMWantSpecialKey(var Msg: TCMWantSpecialKey); message CM_WANTSPECIALKEY;
    procedure WMGetDlgCode(var Msg: TWMGetDlgCode); message WM_GETDLGCODE;
  end;

procedure TMemo.CMWantSpecialKey(var Msg: TCMWantSpecialKey);
begin
  case Msg.CharCode of
  VK_ESCAPE:
    Msg.Result := 0;
  VK_RETURN, VK_EXECUTE, VK_CANCEL:
    Msg.Result := 1;
  else
    inherited;
  end;
end;

procedure TMemo.WMGetDlgCode(var Msg: TWMGetDlgCode);
begin
  inherited;
  Msg.Result := Msg.Result and not DLGC_WANTALLKEYS;
end;