xDMA enqueuing (scatter-gather) support

[br]

Summary

Typical operation example:

Network device driver call flow:

/* Get xDMA controller */
sc->xdma_tx = xdma_ofw_get(sc->dev, "tx");

/* Allocate virtual channel */
sc->xchan_tx = xdma_channel_alloc(sc->xdma_tx, caps);

int
atse_xdma_tx_intr(void *arg, xdma_transfer_status_t *status)
{
    for (;;) {
            err = xdma_dequeue_mbuf(sc->xchan_tx, &m, &st);
            if (err != 0) {
                    break;
            }
    }
}

/* Setup interrupt handler. */
xdma_setup_intr(sc->xchan_tx, atse_xdma_tx_intr, sc, &sc->ih_tx);

/* Prepare SG transfer */
xdma_prep_sg(sc->xchan_tx, TX_QUEUE_SIZE);

/* Enqueue (multiple) mbufs to receive */
m = m_getcl();
xdma_enqueue_mbuf(sc->xchan_rx, &m, 0, 4, 4, XDMA_DEV_TO_MEM);
m = m_getcl();
xdma_enqueue_mbuf(sc->xchan_rx, &m, 0, 4, 4, XDMA_DEV_TO_MEM);
...
xdma_queue_submit(sc->xchan_rx);

/* Enqueue mbuf to transmit */
xdma_enqueue_mbuf(sc->xchan_tx, &m, 0, 4, 4, XDMA_MEM_TO_DEV);
xdma_queue_submit(sc->xchan_tx);

DMA engine driver functions:

/* Prepare a channel for SG transfer */
softdma_channel_prep_sg(device_t dev, struct xdma_channel *xchan)
{
...
}

/* Submit sg list */
softdma_channel_submit_sg(device_t dev, struct xdma_channel *xchan,
    struct xdma_sglist *sg, uint32_t sg_n) 
{
...
}

/* Provide the amount of free entries for requests. */
softdma_channel_capacity(device_t dev, xdma_channel_t *xchan,
    uint32_t *capacity)
{
...
}

Example usage:
xDMA support for atse(4) D9618
mSGDMA driver D9619
SoftDMA® driver D9620
ARM PrimeCell® PL330 driver D10201
Cadence Quad SPI Flash driver D10245

Test Plan

tested with:

• atse(4) driver, Altera SoftDMA® and mSGDMA engines.
• Cadence QSPI driver and ARM PrimeCell® DMA engine.

[br]

change comment

[br]

don't use busdma for SoftDMA®

[br]

request caps on channel allocation

[kan]

sys/dev/xdma/xdma.h

138

Can you use little more consistent naming please? xchan_buf is turning into xdma_buf magically here.

[br]

rename struct

[kan]

This kinda somewhat works for mbufs, but not much more.

sys/dev/xdma/xdma.c

311

Why? How xchan knows boundaries handed by the underlying DMA device? How does it even know if bus_get_dma_tag(xdma->dev) is even a right tag to be used for descriptors? Descriptors often have much stricter requirements on alignment and locations of descriptors and just 'uint32_t desc_size, uint32_t align' cannot possibly describe them. The operation does not belong in generic xchan layer at all and is better left to an actual pdma implementation.

340

Error handling, in particular, freeing already allocated memory and descriptors and unmapping mapped descriptors is considered futile?

360

Same thing. This seems to be very mbuf specific and ridden with arbitrary limitations.

364

The name of the function and later use of MCLBYTES do correspond to each other rather weakly. It if an mbuf-specific functionality, name better should reflect that.

364

BUS_DMA_COHERENT???

383

Again, WHY???

581

You did not protect XCHAN_DESC_ALLOCATED with ()'s above, why now?

761

Same here? Why ()'?

829

()'s again

sys/dev/xdma/xdma.h

203

Why is this a function? Asserts are supposed to maintain the call site information in form of file:line and functions are good at obscuring that.

[kan]

sys/dev/xdma/xdma.c

256–257

This is a xdma_desc_dmamap_cb in , the name is too short and nondescript. I do not think descriptor allocation belongs in xdma at all, but this is wrong nonetheless.

753

This is unconditional? You did not call map_create for !XCHAN_CAP_BUSDMA case.

758

You did not create tag either.

816

By using circular buffer here you encode the requirement that dma requests should be completed by the controller in the order they were submitted, but that is not necessarily the case. I guess same applies by sglists.

888

Is there a direct relationship of number of mbuf fragments and number of physical addresses it can result in?

905

What happens if segment in the middle completes with the error?
Also, what is st_trasferred and in what units it is specified? You invoke callback in per-desc (WHY? DMA driver is not allowed to do desc chaining?), but then only report status on the last desc up the consumer chain.

[br]

move descriptors allocation out of xDMA

[br]

remove any evidence of descriptors existence
add xchan_capacity method

[br]

remove xdma_assert_locked function. Using macro is better way

[br]

sys/dev/xdma/xdma.c

311

Sounds right, thanks. I moved all the descriptors allocation/handling to DMA engine driver.

sys/dev/xdma/xdma.h

203

Agree. We have to use macro, not a function here. So removed

[br]

Add support for enqueuing bio: xdma_enqueue_bio()

[br]

specify port width for bio transfers

[br]

cleanup

[br]

sys/dev/xdma/xdma.c

256–257

I moved descriptors allocation to DMA engine drivers. Thanks

360

tag parameters now specified by transfer preparation functions, e.g. xdma_prep_sg

[br]

sys/dev/xdma/xdma.c

753

Fixed. thanks!

[br]

use linked-lists of requests, instead of ring queue

[br]

cleanup

[br]

remove xchan_buf_t as not commonly used

[br]

sys/dev/xdma/xdma.c

816

I switched back to linked lists. Works a bit slower due to locking, but easier to improve /add features in future.
thanks!

[br]

• add xdma_bank_put() method.
• BIO queues locking

[br]

dehardcode maxnsegs: add it as a parameter to xdma_prep_sg()

[br]

ensure sglist is not full

[br]

• add xchan_bank_init()
• create busdma tag in busdma case only

[br]

• add xchan_bank_free()
• release resources on xdma_prep_sg() errors

[br]

cleanup, ensure kproc_create succeeded