I am creating an Arduino device that connects to a Windows computer and simulates a touchscreen on the computer. The screen does not belong to the Arduino, but rather the Arduino relays HID touch messages to the Windows 10 computer via USB.
The first thing I did was to relay AbsoluteMouse events instead of Digitizer events. This worked fine for me. However, as my device supports multiple touches, I would like to use multi-touch HID messages instead of AbsoluteMouse. As a proof of concept, I am starting with relaying only one finger's data. (I plan to add more fingers after I get this to work). I am taking the SerialReport approach, as it seems to be the easiest to implement:
- Sample Report Descriptor - Serial Reporting Mode (Windows 7)
- Selecting Packet Reporting Modes in Multi-touch Drivers (Windows 7)
I have found many examples of using AbsoluteMouse, so basically I didn't have to do much more than copy-paste the HID reports. However, with touchscreens it is very hard to come by code samples. (I have only found the synaptic touchpad sample which is not a touchscreen). As such I have tried to implement my own code to report touch messages. But this fails. I think I understand how to interpret the HID descriptor, but this is my first time, so I am not sure... Is there anything wrong with my message structure?
#define REPORTID_MTOUCH 1
static const uint8_t _hidSerialMultiTouchDescriptor[] PROGMEM = {
// https://docs.microsoft.com/en-us/windows-hardware/design/component-guidelines/sample-report-descriptor--serial-reporting-mode-
// Linked from: https://docs.microsoft.com/en-us/windows-hardware/design/component-guidelines/selecting-packet-reporting-modes-in-multitouch-drivers
0x05, 0x0d, // USAGE_PAGE (Digitizers)
0x09, 0x04, // USAGE (Touch Screen)
0xa1, 0x01, // COLLECTION (Application)
0x85, REPORTID_MTOUCH, // REPORT_ID (Touch)
0x09, 0x22, // USAGE (Finger)
0xa1, 0x00, // COLLECTION (Physical)
0x09, 0x42, // USAGE (Tip Switch)
0x15, 0x00, // LOGICAL_MINIMUM (0)
0x25, 0x01, // LOGICAL_MAXIMUM (1)
0x75, 0x01, // REPORT_SIZE (1)
0x95, 0x01, // REPORT_COUNT (1)
0x81, 0x02, // INPUT (Data,Var,Abs)
0x95, 0x03, // REPORT_COUNT (3)
0x81, 0x03, // INPUT (Cnst,Ary,Abs)
0x09, 0x32, // USAGE (In Range)
0x09, 0x47, // USAGE (Confidence)
0x95, 0x02, // REPORT_COUNT (2)
0x81, 0x02, // INPUT (Data,Var,Abs)
0x95, 0x0a, // REPORT_COUNT (10)
0x81, 0x03, // INPUT (Cnst,Ary,Abs)
0x05, 0x01, // USAGE_PAGE (Generic Desk..
0x26, 0xff, 0x7f, // LOGICAL_MAXIMUM (32767)
0x75, 0x10, // REPORT_SIZE (16)
0x95, 0x01, // REPORT_COUNT (1)
0x65, 0x00, // UNIT (None)
0x09, 0x30, // USAGE (X)
0x81, 0x02, // INPUT (Data,Var,Abs)
0x09, 0x31, // USAGE (Y)
0x46, 0x00, 0x00, // PHYSICAL_MAXIMUM (0)
0x81, 0x02, // INPUT (Data,Var,Abs)
0x05, 0x0d, // USAGE PAGE (Digitizers)
0x09, 0x48, // USAGE (Width)
0x09, 0x49, // USAGE (Height)
0x95, 0x01, // REPORT_COUNT (2)
0x81, 0x02, // INPUT (Data,Var,Abs)
0x81, 0x03, // INPUT (Cnst,Ary,Abs)
0x09, 0x51, // USAGE (Contact Identifier)
0x75, 0x10, // REPORT_SIZE (16)
0x95, 0x02, // REPORT_COUNT (1)
0x81, 0x02, // INPUT (Data,Var,Abs)
0x09, 0x55, // USAGE(Maximum Count)
0x15, 0x00, // LOGICAL_MINIMUM (0)
0x25, 0x08, // LOGICAL_MAXIMUM (255)
0x75, 0x08, // REPORT_SIZE (8)
0x95, 0x01, // REPORT_COUNT (1)
0xb1, 0x02, // FEATURE (Data,Var,Abs)
0xc0, // END_COLLECTION
0xc0 // END_COLLECTION
};
This is my implementation of said message:
typedef union {
struct{
uint8_t TipSwitchLsb4Bits_InRange_Confidence_Empty2Bits;
uint8_t Padding8_10Minus2;
int16_t xAxis;
int16_t yAxis;
int16_t width;
int16_t height;
int16_t ContactId;
uint8_t MaximumCount;
};
} HID_SerialTouchReport;
void reportTouch(unsigned int x, unsigned int y) {
HID_SerialTouchReport report;
report.TipSwitchLsb4Bits_InRange_Confidence_Empty2Bits = 0xff;
report.xAxis = x;
report.yAxis = y;
report.width = 4;
report.height = 4;
report.ContactId = 0;
report.MaximumCount = 1;
HID().SendReport(REPORTID_MTOUCH, &report, sizeof(report));
}
...
static HIDSubDescriptor node(_hidSerialMultiTouchDescriptor, sizeof(_hidSerialMultiTouchDescriptor));
HID().AppendDescriptor(&node);
...
//Move the pointer diagonally across the screen:
for(unsigned int x=0; x<32767; x+=1000) {
reportTouch(x, x);
delay(50);
}
The test code is supposed to draw a diagonal line from top left to bottom right on Microsoft Paint.
When I test this with an AbsoluteMouse descriptor/Message everything works. So I presume the problem is in my definition of HID_SerialTouchReport.
Some things that have grabbed my attention:
- Why does "PHYSICAL_MAXIMUM (0)" come while still defining "USAGE (Y)". Is it somehow relevant to Y and not X? And why is it "0"?
- What is the significance of "Ary" in "INPUT (Cnst, Ary, Abs)" for height and width?
- I am sort of "fudging" things with: "report.TipSwitchLsb4Bits_InRange_Confidence_Empty2Bits=0xff;". However, this seems alright to me, because I think all the non-padded bits need to be set to "1" anyway.
- I am not sure what I should set report.MaximumCount to be. I have tried experimenting with different Count & Id values, but all had no effect.
- My Arduino does show up as a "composite device" in Device Manager, so I figure not all is wrong.
- In the article I linked "Serial Packet" mode is for Legacy Windows 7, and I am using Windows 10. But I figure this should not be a problem.
- While I did copy the HID descriptor directly from the Microsoft site, I am not sure I properly constructed the message structure. (Did I forget a few bits?)
Or is it something entirely other which is causing my code to fail?
