A receive buffer size of 100*1024*1024 is HUGE! I would seriously doubt you need this size at all, and certainly not for the actual serial port buffer.
It could be that you are overflowing the physical receive buffer of the receiver, so you may have to look into using flow control. This will effectively enable your Receiver to say to your Transmitter "Hang on, stop sending for a while, let me deal with what I have first."
Hardware flow-control is (commonly) utilised through the RTS (Request To Send) and CTS (Clear To Send) pins.
Take a look at this article, which explains a bit more about it.
I would personally recommend leaving the ReceivedBytesThreshold property of the serial port to its default value of 1, and then handling the DataReceived event. Who knows, maybe tomorrow you will need to read a message of 20 bytes, or perhaps 5 bytes. Maybe you will need to read messages of variable length in the future? Leaving the threshold at 1 will mean that you can handle any and all bytes that will be received, now and in the future.
This threshold will mean the event will fire when there is at least 1 byte in the buffer. There maybe more, and probably will be. Note that it does NOT necessarily mean it will fire for every single byte received. On each event, you need to check the BytesToRead property and read this many into your own buffer.
It is worth repeating that the event will NOT necessarily fire for every single byte received.
Also, it is best to do as little as possible within the DataReceived event handler. Read in any bytes received into your buffer, and perhaps add completed messages into a queue for further processing, but do little else.
Also note that the standard maximum spec of RS232 is a speed of 19200 baud and a length of cable of 50 feet. Anything above that is undefined. Higher speeds generally need better cable (lower capacitance) and shorter cable lengths. Make sure it is screened and not running near other 'noisy' items such as motors, motor inverters, live cables etc.
ReadByte
inside the while loop? How long does the processing take? – Ian Mercer