DMA configuration for SPI not used in mainline

[Dennboy]

Dear mainline developers,

I'm writing a (mainline) SPI driver for de ads131 from TI which I would like to make use of DMA, on the orangePIzero H2, zero+ H3/H5 and nanopi NEO2+. On the 4.13.3-sunxi kernel (armbian 5.33), this works fine, the driver is automatically loaded when the associated devicetree overlay is loaded, and the DMA can be allocated. However, when I compile the same driver for newer kernels (4.14.17-sunxi, 4.14.39-sunxi, 4.14.67-sunxi armbian 5.59), the dma allocation no longer appears to work, while the SPI part of the devicetree is identical except for the phandle (checked with dtc -I fs /sys/firmware/devicetree/base on newest 4.14.67-sunxi kernel, since 4.14.65 headers were not available in newest 5.59 stretch mainline H3 image) and does specify the DMA addresses for the SPI controller:

spi@01c69000 {                                                                                                               
    compatible = "allwinner,sun8i-h3-spi";                                                                               
    clocks = <0x6 0x1f 0x6 0x53>;                                                                                        
    resets = <0x6 0x10>;                                                                                                 
    clock-names = "ahb", "mod";                                                                                          
    status = "okay";                                                                                                     
    #address-cells = <0x1>;                                                                                              
    interrupts = <0x0 0x42 0x4>;                                                                                         
    #size-cells = <0x0>;                                                                                                 
    dma-names = "rx", "tx";                                                                                              
    phandle = <0x5e>;                                                                                                    
    reg = <0x1c69000 0x1000>;                                                                                            
    pinctrl-0 = <0x21>;                                                                                                  
    dmas = <0x1f 0x18 0x1f 0x18>;                                                                                        
    linux,phandle = <0x5e>;                                                                                              
    pinctrl-names = "default";                                                                                           

    ads131a04@0 {                                                                                                        
      compatible = "ti,ads131a04";                                                                                 
      status = "okay";                                                                                             
      reg = <0x0>;                                                                                                 
      spi-max-frequency = <0x1e84800>;                                                                             
    };
};

I use the following DMA calls that appear to fail in newer kernels, and the driver reverts to non-DMA spi transfers:

static int ads131_probe(struct spi_device *spi)
{
    const struct spi_device_id *id = spi_get_device_id(spi);
    int ret;

    ret=spi_setup(spi);
    if (ret!=0) {
        printk("ads131: Problem setting up spi-device, ret=%d",ret);
        return ret;
    }
    printk("ads131: spi->controller->can_dma=0x%lx",(unsigned long)spi->controller->can_dma);
    ...  
    if (!dma_set_mask_and_coherent(&spi->dev, DMA_BIT_MASK(64))) {
        printk("ads131: dma 64 bit");
    } else if (!dma_set_mask_and_coherent(&spi->dev, DMA_BIT_MASK(32))) {
        printk("ads131: dma 32 bit");
    } else if (dma_set_mask_and_coherent(&spi->dev, DMA_BIT_MASK(24))) {
        dev_warn(&spi->dev, "ads131: No suitable DMA available\n");
        goto dma_ignore;
    } else {
        printk("ads131: dma 24 bit");
    }
    ...

I get the following output from my driver in kernel 4.14.67-sunxi:

dennis@orangepizeroplus2:~$ dmesg |grep -i dma
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.091893] DMA: preallocated 2048 KiB pool for atomic coherent allocations
[    5.254126] ads131: spi->controller->can_dma=0x0
[    5.255134] ads131a04 spi0.0: ads131: No suitable DMA available

I get the following with the older 4.13.3-sunxi kernel (armbian 5.33):

[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool                                                   
[   19.848567] ads131: spi->controller->can_dma=0x0                                                                                          
[   19.849538] ads131: dma 64 bit                                                                                                            
[   19.849544] ads131: dma_mask=18446744073709551615                                                                                         
[   19.849564] xfer dma memory allocated with 16 buffers  

 

Do I need to specify additional information in the devicetree for DMA to work in SPI drivers with newer kernels, or do I need to add something to my driver before trying to use DMA with newer kernels?

Kind regards,

Dennis

[Ohms]

From your description it seems that the function dma_set_mask_and_coherent() which used to work for kernel version 4.13.3 and no longer for newer kernel versions?

That's a shame, Have you checked the release notes of newer kernel versions with sunxi modifications or maybe some of the experts on this forum can react? 

[Dennboy]

Dear Ohms,

 

Many thanks for your feedback. Indeed, the dma_set_mask_and_coherent call does no longer work with the same driver code and device tree. Regarding the sunxi modifications, I cannot easily find release notes of the modifications between kernel 4.13.3 and newer. The change could also be in the mainline kernel updates, e.g. stricter checks in the DMA or SPI driver.

The strange thing is that independent of kernel version spi->controller->can_dma is 0x0, which is supposed to tell whether the controller supports DMA according to spi.h.

• @can_dma: determine whether this controller supports DMA

Could it be that the DMA part of the SPI controller is not completely configured from the information in the device tree?

 

Furthermore, I don't notice any performance gain when I have DMA enabled in the older 4.13.3 kernel and allocate coherent memory for rx_dma and tx_dma in the spi_transfer struct (using dmam_alloc_coherent).

[Dennboy]

I digged further into the kernel code and had a look at spi.c and spi-sun6i.c. It appears that can_dma is a function pointer that is checked while setting up DMA for a transaction. This function pointer is not assigned in spi-sun6i.c (and spi-sun4i.c for that matter). This would explain why I don't see performance improvements when I try to use DMA for with kernel 4.13.3, since the DMA ignored because can_dma=0x0.

 

I compared the behavior with the spi-stm32.c in the 4.14.67 kernel, and this driver reads the controller DMA addresses and sets the can_dma function pointer when they are set:

	spi->dma_tx = dma_request_slave_channel(spi->dev, "tx");
	if (!spi->dma_tx)
		dev_warn(&pdev->dev, "failed to request tx dma channel\n");
	else
		master->dma_tx = spi->dma_tx;

	spi->dma_rx = dma_request_slave_channel(spi->dev, "rx");
	if (!spi->dma_rx)
		dev_warn(&pdev->dev, "failed to request rx dma channel\n");
	else
		master->dma_rx = spi->dma_rx;

	if (spi->dma_tx || spi->dma_rx)
		master->can_dma = stm32_spi_can_dma;

The assigned function stm32_spi_can_dma mainly checks if it makes sense to do dma, i.e. is the transfer size bigger than the fifo (I believe the fifo is 64 bits for the H3 SPI Controller, so that may pay off for most SPI transactions):

return (transfer->len > spi->fifo_size);

Bottom line, the SPI controller does not yet take the configured DMA configuration into account, and it could be fixed using a similar approach to the spi-stm32.c. The