epoll and send lag

2
votes

I'm using Linux 64 bit Linux scv 3.2.0-39-generic #62-Ubuntu SMP Thu Feb 28 00:28:53 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux and have two processes using sockets which run on the same physical host.

One process (A) sends on a TCP/IP socket (would be a local socket given the host is the same) the following pieces of data:

  1. 276 bytes
  2. 16 bytes

This is done in 0.000023 seconds form process A. The data is being sent calling 2 times the send socket API.

Another process (B), receives the data via epoll using epoll_wait(efd, events, 10, 5). Data is received as follows (time is taken with clock_gettime(CLOCK_REALTIME, &cur_ts);, what matters is relative difference):

  1. Read data from socket buffer at 8051.177743 (276)
  2. Call to epoll 8051.177763 again
  3. Read data from socket buffer 8051.216250 (16)

Making the receiving process lag of 0.038507 seconds. Basically if the sending process A takes less than a ms, on the receiving side epoll to receive the data adds an additional lag of approximately 0.038 s.

Is this expected? What am I doing wrong?
Or how can I improve the situation?

Thanks

1
clinuxsocketsepoll
Point 3 is when second call to epoll returns. - Emanuele
40ms is standard time slice, looks like standard context switching on a single core. - Steve-o
Should I move away from epoll if I want to reduce this lag? What do you suggest? - Emanuele
I am currently seeing something equivalent - however, I'm getting a lag of 0.2 seconds before epoll_wait returns instead of 38 ms :(. I don't believe that this is "native" to epoll "because it only works with a large number of fds". Something is wrong and it must be possible to actually explain what that is. - Carlo Wood

1 Answers

-1
votes

Is this expected? ...

Yes. I would expect that. Here's why:

What am I doing wrong? ...

epoll was designed to be used in situations where large numbers of file descriptors need to be watched. That's what it's suitable for, and it seems to me that the situation you're using it for isn't that situation.

... how can I improve the situation?

If you want to improve the performance, use the right tool for the job. Don't use epoll for a single socket. Just use plain-old vanilla recv. If you're handling two or three sockets, consider using poll or select. If you're venturing into hundreds, then you might want to consider using epoll or kqueue.