Page MenuHomeFreeBSD

Improve pfctl configuration load time for large numbers of queues
ClosedPublic

Authored by jps_rgnets.com on Wed, Jul 27, 2:01 PM.

Details

Summary

In the current implementation of altq_hfsc.c, whne new queues are being added (by pfctl), each queue is added to the tail of the siblings linked list under the parent queue.

On a system with many queues (50,000+) this leads to very long load times at the insertion process must scan the entire list for every new queue,

Since this list is unordered, this changes merely adds the new queue to the head of the list rather than the tail.

Test Plan

Step one:
Test a variety of pf.conf files, both valid and invalid.

Step two:
Test a variety of queue configurations. Validate that the throughput for each client matches the specifications in the configuration.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

i think this is somewhat suspicious. i think it would be safer to keep the track of the tail. this would still give you O(1) insertion without changing ordering

In D35964#816428, @mjg wrote:

i think this is somewhat suspicious. i think it would be safer to keep the track of the tail. this would still give you O(1) insertion without changing ordering

cl_siblings is only ever used in alto_hfsc.c, and the order never matters, so it seems fine to me to do this. We could keep the order by also tracking the tail as you say, but it doesn't buy us anything and costs (an infinitesimal amount of) memory.

This revision was not accepted when it landed; it landed in state Needs Review.Thu, Jul 28, 8:11 PM
This revision was automatically updated to reflect the committed changes.

Yes, I also considered this. I spent some time reassuring myself that the order was not important, but of course I was interested in your feedback in case I missed something. I'm certainly willing to add a tail if the consensus indicates that is a good idea.