By using the bold highlighting in your original question, the post formatting swallowed up key information in your post, so my original answer is not quite correct. I've tried to edit your post and restore your original intent, and have updated my answer appropriately. However, based on reviewing that function name in other DLLs I found in a brief search, I'm not sure you've quoted the source DLL correctly either, particularly whether the Baud variable is also a pointer (*).
Your first stop in reviewing correct mappings should be the JNA documentation .
For C pointer types, the JNA Pointer works, but requires you to do a lot more manual work allocating memory, reading the result, and then choosing the correct number of bytes to read from the pointer address. When the type of pointer is specified, as it is in this case, you should use the appropriate ByReference mapping which does the memory allocation work for you and better controls "strongly typed" access to the method.
For your unsigned char *ComAdr you would want to use a ByteByReference mapping. Similarly for int * FrmHandle you would use IntByReference. You have listed Baud as simply unsigned char type which would map directly to byte; however, other API's have that as a char * also, so it would be the same ByteByReference mapping.
Note that Java does not use unsigned variables so you'll have to take additional action on negative byte values once you extract them from the return values in order to make sure they correspond to the unsigned C value, e.g., using & 0xff.
bytein java could be mapped tounsigned char- user7859067*symbols that you have not included, and those are critical in answering your question. - Daniel Widdis