DMA transfer to a slave PCI device in Linux

Question

I am a bit confused regarding DMA transfers with a PCIe device.

Say, for example, I have a slave PCIe device, and I want to transfer a block of data from the device to the RAM, using a DMA transaction. Note that the device is slave, and does not have a DMA "machine" on it.

I know I need to obtain a DMA-able buffer in RAM (either by allocating a coherent one, or by mapping a page) first.

But what's next? what's the API to start a DMA transfer of N bytes from address S to address D?

Can modern systems issue a DMA transfer to/from a slave pci device? if so, what is the Linux API for that?

As explained here:

[ISA] In the original IBM PC, there was only one Intel 8237 DMA controller [...] A PCI architecture has no central DMA controller, unlike ISA. Instead, any PCI component can request control of the bus ("become the bus master") and request to read from and write to system memory

The PCI bus does not have a "central" DMA controller - instead, each device can be a DMA "controller".

The answers here should help: stackoverflow.com/questions/2369411/… — Marc B
Thanks, but after already reading some material, I feel there is something basic that I'm missing - is there such an API at all? am I allowed to use the system DMA controller? — benzaita
I have no idea, actually. The last time I did any hardware level programming, 486's were king of the hill. Is DMA even still used for PCI/PCIe? My impression was that the device memory spaces simply get memmapped and then you do normal memcpy() type operations and the back-end hardware takes care of it all. — Marc B
If you're talking about DMA as it used to be on ISA bus systems, that doesn't really exist anymore in a modern system. The 8237 dma controllers had a 16meg addressing limit. — Marc B
kernel.org/doc/Documentation/DMA-API.txt , and old but still informative linuxjournal.com/article/7104 — nos

osgx osgx · Accepted Answer · 2012-06-08T22:18:45

First of all, there are no slaves and slave holders inside modern PC. There is south bridge (in PCI) or Root Complex (root of PCI-express device tree) and there are some other PCI/PCIe actors, like bridges, soldered chips, plugged cards, hardware debuggers etc. I'll assume that you are asking about plugged card or some other peripheral device, like soldered Sound Card or Ethernet chip.

According to this detailed description of "Transaction Layer Packet" (TLP, "PCIe’s uppermost layer"), there is "Bus Mastership (DMA)":

On PCIe, it’s significantly less exotic. ... anyone on the bus can send read and write TLPs on the bus, exactly like the Root Complex. This allows the peripheral to access the CPU’s memory directly (DMA) or exchange TLPs with peer peripherals (to the extent that the switching entities support that).

Also, there is some benefits of DMA capability from plugged devices: DMA attack. And PCIe is listed as capable of initiating DMA transfer:

Systems may be vulnerable to a DMA attack by an external device if they have a FireWire, ExpressCard, Thunderbolt, or other expansion port that, like PCI and PCI-Express in general, hooks up attached devices directly to the physical address space.

I think, there is no universal API for programming DMA transfers that are initiated from the peripheral device itself. This depends on the what the device is, when the DMA should be started and what will be sent.

DMA transfer to a slave PCI device in Linux

1 Answers