Create custom keyboard layout with AutoHotKey (or something else)

Question

I'm looking to create a custom keyboard layout, mainly for typing unicode math symbols. The symbol set I need is very large, and the scheme I came up with involves multiple layouts and special combinations.

I type ` (backtick) once and instead I get a special character that we'll symbolize by *. By typing additional keys, I get specific keyboard layouts relevant to a particular theme. (I want to replace the back tick with a special symbol so that I remember it's a control code of sorts. By typing it twice, I get a normal back tick)

Here are some example mappings:

*s -> Set theory layout:
    [ -> ∈ (element of)
    o -> ∅ (empty set)

*r -> General math:
    s -> ∫ (integral sign)
    S -> ∬ (double integral sign)

*e -> Misc operators:
    8 -> ∗ (convolution asterisk)
    * -> ⋆ (star operator)

*g -> Greek alphabet

After typing a character such as =, I can type a few other special combinations to modify that character. For example:

*x -> Negates the previous character:
    = -> ≠ (unequal)
    ≡ -> ≢ (negation of three line equality)

*w -> Expands the previous character:
    ∫ -> ∭ (triple integral)
    ∬ -> ⨌ (quad integral)

The mappings are mnemonic. I could conceivably cram all the symbols I want onto a single layout, but it would be unusable, so I want to try to stick with this scheme or something similar.

The keyboard is for a Windows environment, but there is no question of me writing keyboard DLLs for it myself. I looked into it and it's just way too complicated.

Right now, I'm looking into AHK for a solution. Can it be done in AHK (or something similar)? If so, could you show me some examples to get me started?

I'd also like to know if there are other ways to do this.

I'm aware of the Microsoft Keyboard Layout Creator, and have used it in the past, but it's not nearly powerful enough. I'm also aware of Keyman by Tavultesoft, and I know for a fact it can do what I want, but it's absurdly expensive, so it's not an option.

Answer 1

What you need can definitely be done with AutoHotkey.

The tricky part is: You need to assign your Hotkeys and Hotstrings dynamically. The former is possible with the Hotkey-command, the latter has not yet been implemented and probably will never be. You can either write them to a new .ahk-file and execute that script seperately, or use polyethene's really awesome dynamical regEx hotstring creator. autohotkey.net's repository has been down for years, but I found the script here. To get the following example script running, you'll need to download that script into Hotstrings.ahk and put it in the same directory where your main .ahk-file will be.

However, I haven't been able to assign the layout changer hotkey to

`

, so I set it to ä.

I know that this site is not a code-provider-network, but I have major interest in this issue as well.

#NoEnv
SendMode Input
SetWorkingDir %A_ScriptDir%
#include Hotstrings.ahk

lastKey := ""

; KEYS OF THE NORMAL LAYOUT WHICH SHOULD BE NEGATABLE:
hotstrings("=", "_equals")  ;

; CHOOSE LAYOUT OR NEGATE/EXPAND LAST CHARACTER:
:*?:ä::
; backSpacePressed:  ; wtf is this? sorry just saw this now. Does not belong here, does not belong anywhere
    sendRaw *
    tooltip, 
    (
    n   normal layout

    s   set theory
    r   general math

    x   negate previous
    w   expand previous

    {esc}   cancel
    ), %A_CaretX%, %A_CaretY%
    input, layout, L1, {Escape}, s,r,e,g,x,w
    send {Backspace}    ; remove the *
    if layout in n,s,r,e,g,x,w
    {
        tooltip, %layout%, %A_CaretX%, %A_CaretY%

        ; RESET
        if layout = n
        {
            reset_all_hotstrings()

            ; KEYS OF THE NORMAL LAYOUT WHICH SHOULD BE NEGATABLE:
            hotstrings("=", "_equals")
        }



        ; NEW LAYOUT
        else if layout = s
        {
            reset_all_hotstrings()

            ; SET THEORY SHORTCUTS
            hotstrings("o", "_emptySet")
            hotstrings("\[", "_elementOf")
        }
        else if layout = r
        {
            reset_all_hotstrings()

            ; MATH SHORTCUTS
            hotstrings("s", "_integral")
            hotstrings("S", "_doubleIntegral")
            hotstrings("=", "_identical")
        }
        ; and so on
        ; ...



        ; EDIT PREVIOUS CHARACTER
        else if layout = x
        {
            send {backSpace}

            if lastKey = identical
                sendUnicodeChar(0x2262)
            else if lastKey = equals
                sendUnicodeChar(0x2260)
        }
        ; EXPAND PREVIOUS CHARACTER
        else if layout = w
        {
            send {backSpace}

            if lastKey = integral
                sendUnicodeChar(0x222D)
            else if lastKey = doubleIntegral
                sendUnicodeChar(0x2A0C)
        }
    }
    else
    {
        tooltip, cancelled, %A_CaretX%, %A_CaretY%
    }
    sleep, 500
    tooltip
return

reset_all_hotstrings() {
    hotstrings("=")
    hotstrings("\[")
    hotstrings("o")
    hotstrings("s")
    hotstrings("S")
    ; and so on
}

; NORMAL LAYOUT SHORTCUTS:
_equals:
    sendUnicodeChar(0x003D)
    lastKey = equals
return



; SPECIAL LAYOUT SHORTCUTS:
_emptySet:
    ;sendUnicodeChar(0x00D8)
    altNumpad(0216)
        ; to find out numpad combination or unicode: press WIN+R, type in "charmap"
        ; or for unicode only, go to http://www.fileformat.info/info/unicode/category/index.htm
        ; (sendUnicodChar() needs 0x before the unicode string)
    ;altNumpad(0248)
    ;send Ø
    ;send ø
    ;   choose whatever works best for you
    lastKey = emptySet
return

_elementOf:
    sendUnicodeChar(0x2208)
    lastKey = elementOf
return

_integral:
    sendUnicodeChar(0x222B)
    lastKey = integral
return

_identical:
    sendUnicodeChar(0x2261)
    lastKey = identical
return

_doubleIntegral:
    sendUnicodeChar(0x222C)
    lastKey = doubleIntegral
return

; -------------------------------------------

altNumpad(numbers) {
    stringSplit, n, numbers
    setkeydelay, 100
    send {alt down}
    loop, %n0%
    {
        t := n%a_index%
        send {numpad%t%}
    }
    send {alt up}
}


SendUnicodeChar(charCode)
{
    VarSetCapacity(ki, 28 * 2, 0)
    EncodeInteger(&ki + 0, 1)
    EncodeInteger(&ki + 6, charCode)
    EncodeInteger(&ki + 8, 4)
    EncodeInteger(&ki +28, 1)
    EncodeInteger(&ki +34, charCode)
    EncodeInteger(&ki +36, 4|2)

    DllCall("SendInput", "UInt", 2, "UInt", &ki, "Int", 28)
}

EncodeInteger(ref, val)
{
    DllCall("ntdll\RtlFillMemoryUlong", "Uint", ref, "Uint", 4, "Uint", val)
}

^e::reload

There is much room for improvement, obviously