How to write CPU's DMA address to FPGA (PCIe Endpoint)?

Question

I'm trying to add DMA to my PCIe Linux driver using streaming DMA mappings. The FPGA (endpoint) has BAR4 configured for DMA and in my setup function I do (in order):

pci_set_master()
pci_enable_msi()
pci_set_dma_mask()
pci_set_consistent_dma_mask()
__get_free_pages()
dma_addr = pci_map_single(..., PCI_DMA_FROMDEVICE)

At this point I do not know how to tell the FPGA what my DMA address, dma_addr, is that was returned from pci_map_single(). Do I write dma_addr to BAR4 using pci_write_config_dword()? There has to be some way to tell the FPGA where it needs to write when using DMA or am I completely missing something here?

We need more information about your FPGA design. For exemple with a CycloneV, to use the PCIe the component "Avalon-MM for PCIe" component must be instanciated. And in this component we have some registers to configure to add the dma address. — FabienM
We are using a Cyclone V GT development board/kit with an iMX6. Both hardware and software are learning PCIe for the first time. What I'm interested in is where within BAR4 I write the DMA address the kernel gives me? — user2205930

FabienM FabienM · Accepted Answer · 2016-03-04T08:15:44

To read/write data in BARx of your PCIe, you have to map the BARx with function : void __iomem *pcim_iomap(struct pci_dev *pdev, int bar, unsigned long maxlen);

You can do as following for exemple:

/* declare the bar4 buffer */
static volatile u32 __iomem * bar4;

/* map the bar4 */
bar4 = pcim_iomap(&pdev->dev, 4, BAR4_SIZE);

bar4[DMA_VECTOR_REGISTER_ADDRESS] = dma_addr;

See more documentation about BARx/MMIO in kernel Documentation.

The address of DMA_VECTOR_REGISTER_ADDRESS depend of your FPGA architecture. As I can see in comments, you are using a CycloneV GT. You should look at the CRA (Configuration Registers Access) registers. On CycloneV GX PCIe Hard ip for Avalon-MM (page 83), the register address for DMA vector begin at 0x1000 (Avalon-MM-to-PCI Express Address Translation Table).

How to write CPU's DMA address to FPGA (PCIe Endpoint)?

1 Answers