I solved the issue by creating my own layout and style for the QTVirtualKeyboard plugin.
Specifically I set the design height and width in the style.qml file and altered some of the key settings, to remove keys I didnt need for my layout.
I then rebuilt the QTVirtualkeyboard plugin and then linked my test application with the newly created dll.
This solved my issue.
I copied the default style.qml that comes with QT (normally found in QT\\Src\qtvirtualkeyboard\content\styles\default)
I then made the following modifications:
readonly property int defaultKeyFontSize: 28
readonly property real keyBackgroundMargin: Math.round(3 * scaleHint)
readonly property real keyContentMargin: Math.round(10 * scaleHint)
readonly property real keyIconScale: scaleHint * 0.2
keyboardDesignWidth: 480
keyboardDesignHeight: 360
keyboardRelativeLeftMargin: 8 / keyboardDesignWidth
keyboardRelativeRightMargin: 8 / keyboardDesignWidth
keyboardRelativeTopMargin: 8 / keyboardDesignHeight
keyboardRelativeBottomMargin: 8 / keyboardDesignHeight`
I set the pixelSize of any key font size value to defaultKeyFontSize;
After this, it will just be a case of styling your keyboard to your specific requirements.