PCIe bus latency when using ioctl vs read?

Question

I've got a hardware client¹ who's line of data acquisition cards I've written a Linux PCI kernel driver for.

The card can only communicate 1-4 bytes at a time depending on how the user specifies to utilize it, given this, I utilize ioctl for some of the functionality, but also make use of the file_operations structure to treat the card as a basic character device to give the user of the card the ability to just use read or write if they just want simple 1-byte communication with the card.

After discussing the driver with the client, one of their developers is in the understanding that treating the card as a character device by using open/read/write will introduce latency on the PCI bus, versus using open/ioctl.

Given that the driver makes no distinction how it's opened and the ioctl and read/write functions call the same code, is there any validity to this concern?

If so, how would I test the bus latency from my driver code? Are there kernel functions I can call to test this?

Lastly, wouldn't my test of the bus only be valid for my specific setup (kernel settings, platform, memory timing, CPU, etc.)?

^{1: they only have 2 other developers, neither of which have ever used Linux}

Your wording confuses me. I could imagine that some forms or patterns of access are more affected by PCIe bus latency than others, but I don't see how they could introduce bus latency. Isn't the latency you're talking about a hardware characteristic? — John Bollinger
@JohnBollinger .. that's basically my question. My code, as is, wouldn't introduce bus latency because the ioctl and read/write are the same code, so any bus latency is due to the card or PCI bus itself, not the driver (or necessarily even the kernel). I've tried to explain this to the client's hardware guys and the developer who thinks this, but the developer wants me to give proof, and I'm just the contractor .. so I'm trying to find out how I can test the timing of the bus through my driver based on the function I utilize, or provide other proofs to this fact. — txtechhelp
The part that really made my ears perk up was the contrast between "communicates 1-4 bytes at a time" and "1-byte communication". If the read and write implementations indeed operate only a single byte at a time, but the hardware supports four at a time, then they will indeed suffer from more latency than is necessary. Whether you provide an ioctl that could do better is unclear. — John Bollinger
Frankly, it's also a bit surprising to me that your driver exposes ioctls that can serve the same purpose as read and write. Of course, the ioctl interface is very generic and can support that, but it normally is used for providing access to device parameters, not for performing I/O. — John Bollinger
@JohnBollinger .. I edited my question to make that a little more clear; the card can communicate 1, 2 or 4 bytes at a time, depending on how the user specifies, so a 1-byte ioctl call is the same is a 1-byte read (at least that's the question) ... as an aside, the ioctl to communicate is something they wanted in the driver :/ — txtechhelp

David Schwartz David Schwartz · Accepted Answer · 2018-12-18T19:38:50

I suspect the client's developer is slightly confused. He's thinking that the distinction between using read or write versus ioctl corresponds to the type of operation performed on the bus. If you explain to him that this is just a software API difference and either option performs the exact same operation on the bus, that should satisfy them.

PCIe bus latency when using ioctl vs read?

1 Answers