You have a major problem in your example.
With Java NIO, the thread doing the accept() must only be doing the accept(). Toy examples aside you are probably using Java NIO because of anticipated high number of connections. If you even think about doing the read in the same thread as the selects, the pending unaccepted selects will time out waiting for the connection to be established. By the time this one overwrought thread gets around to accepting the connection, the OS's on either side will have given up and the accept() will fail.
Only do the absolute minimum in the selection thread. Any more and you will just being rewriting the code until you do only the minimum.
[In response to comment]
Only in toy examples should the reading be handled on the main thread.
Try to handle:
- 300+ simultaneous connection attempts.
- Each connection once established sends 24K bytes to a single server - i.e. a small web page, a tiny .jpg.
- Slow down each connection slightly ( the connection is being established over a dialup, or the network is having a high-error/retry rate) - so the TCP/IP ACK takes longer than ideal (out of your control OS level thing)
- Have some of your test connections, send a single bytes every 1 milliseconds. (this simulates a client that is having its own high load condition, so is generating the data at a very slow rate.) The thread has to spend almost the same amount of effort processing a single bytes as it does 24K bytes.
- Have some connections be cut with no warning ( connection lost issues ).
As a practical matter, the connection needs to be established within 500ms -1500ms before the attempting machine drops the connection.
As a result of all these issues, a single thread will not be able to get all the connections set up fast enough before the machine on the other end gives up the connection attempt. The reads must be in a different thread. period.
[Key Point]
I forgot to really be clear about this. But the threads doing the reading will have their own Selector. The Selector used to establish the connection should not be used to listen for new data.
Addition (in response to Gnarly's contention that no I/O actually occurs during the java call to read the stream.
Each layer has a defined buffer size. Once that buffer is full, the IO is halted. For example, TCP/IP buffers have between 8K-64K buffers per connection. Once the TCP/IP buffer fills, the receiving computer tells the sending computer to stop. If receiving computer does not process the buffered bytes fast enough the sending computer will drop the connection.
If the receiving computer is processing the buffered bytes, the sender will continue to stream the bytes, while the java io read call is being made.
Furthermore, realize that the first byte to arrive triggers the "bytes available to be read" on the selector. There is no guarantee as to how many have arrived.
The buffer sizes defined in the java code have no relationship to the buffer size of the OS.