Windows Mobile != Windows CE 5.0, so first you need to verify what your hardware is actually running. Second, you have to verify what the "USB port" is. USB has two "modes" - Host and CLient - and they are different both physically and in software. If it's the port that you hoos the device to the PC through, then it's a client port and is not going to be able to accept a peripheral.
Now if it's a CE 5.0 device and if it's a USB Host port then we get down to the USB device type and the OS drivers. What kind of device is it? Does it act as a serial device already (e.g. it has an FTDI or Prolific chipset in it)? If so, just install the right CE driver and you're set. If it doesn't, then you need to write a host device driver to allow the USB system to communicate with this particular device. How that driver will work depends on the hardware.
AS to what the port name will be, that's up to the host driver's registry settings. Typically it will start at the first available port (i.e. if your hardware already has "COM1:", then it will start at "COM2:"), but that's easily overridden, so verify in the registry.
EDIT
For examples of the driver implementations see the following locations on your PC (this assumes you've got Platform Builder - I think any version since about 3.0 - installed):
%WINCEROOT%\PUBLIC\COMMON\OAK\DRIVERS\USB\CLASS\USBSER for a basic USB host serial class driver example
%WINCEROOT%\PUBLIC\COMMON\OAK\DRIVERS\USBFN\CLASS\SERIAL for a basic USB function serial driver example
