iflib(4): Replace admin taskqueue group with per-interface taskqueues

[krzysztof.galazka_intel.com]

Summary

Using one taskqueue group with single thread to execute all admin
tasks may lead to unexpected timeouts when long running task (e.g.
handling a reset after FW update) for one interface prevents
tasks from other interfaces being executed. Taskqueue group API
doesn't let to dynamically add threads, and pre-allocating thread
for each CPU as it's done for traffic queues would be a waste
of resources on systems with small number of interfaces. Replace
global taskqueue group for admin tasks with taskqueue allocated
for each interface to allow independent execution.

Signed-off-by: Krzysztof Galazka <[e-mail]>

[krzysztof.galazka_intel.com]

Rebased on top of master.
Free taskqueue after releasing interrupts per jhb comment on Github.

[jhb]

This looks ok to me, though I'm not intimately familiar with all of iflib.

[kbowling]

Seems reasonable to me. If there is any work that depends or interacts with these changes it would be nice to see them linked/referenced so we can look at that too but I can't foresee any problems.

sys/net/iflib.c

6342

Can this line be removed entirely?

6404–6405

Can you explain how this works? Looks like we used to have vflr_task but this will not be needed anymore with the per interface task which I suppose could do the same work?

[krzysztof.galazka_intel.com]

In D46062#1076736, @kbowling wrote:

Seems reasonable to me. If there is any work that depends or interacts with these changes it would be nice to see them linked/referenced so we can look at that too but I can't foresee any problems.

There is no dependent work.

sys/net/iflib.c

6404–6405

Uh, I'm sorry, I don't know what I was thinking. I changed iflib_iov_intr_deferred to enqueue a task to admin taskqueue but this task is never initialized. I need to fix that. Thanks for pointing this out.

[krzysztof.galazka_intel.com]

Add forgotten task init for IFLIB_INTR_IOV soft irq type.

[kbowling]

sys/net/iflib.c

5295

Prefix name with if_

6404–6405

Does this need cleanup of the taskqueue?

[krzysztof.galazka_intel.com]

Prefix taskqueue name with 'if_'.

[krzysztof.galazka_intel.com]

sys/net/iflib.c

6342

gtask is used below:

info->ifi_task = gtask;

but I guess I can add initialization to NULL in the declaration instead. Is that what you mean?

6404–6405

Do you mean something more elaborate than taskqueue_free in iflib_device_deregister?

[krzysztof.galazka_intel.com]

@kbowling Do you want me to make any additional changes in this patch?