Page MenuHomeFreeBSD

Improve pfctl configuration load time for large numbers of queues
ClosedPublic

Authored by jps_rgnets.com on Jul 27 2022, 2:01 PM.
Tags
None
Referenced Files
F82060081: D35964.diff
Thu, Apr 25, 3:31 AM
Unknown Object (File)
Jan 14 2024, 10:31 AM
Unknown Object (File)
Dec 23 2023, 12:42 AM
Unknown Object (File)
Sep 5 2023, 6:47 AM
Unknown Object (File)
Aug 31 2023, 2:47 AM
Unknown Object (File)
Aug 31 2023, 2:46 AM
Unknown Object (File)
Aug 31 2023, 2:35 AM
Unknown Object (File)
Aug 15 2023, 12:17 AM

Details

Summary

In the current implementation of altq_hfsc.c, whne new queues are being added (by pfctl), each queue is added to the tail of the siblings linked list under the parent queue.

On a system with many queues (50,000+) this leads to very long load times at the insertion process must scan the entire list for every new queue,

Since this list is unordered, this changes merely adds the new queue to the head of the list rather than the tail.

Test Plan

Step one:
Test a variety of pf.conf files, both valid and invalid.

Step two:
Test a variety of queue configurations. Validate that the throughput for each client matches the specifications in the configuration.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

i think this is somewhat suspicious. i think it would be safer to keep the track of the tail. this would still give you O(1) insertion without changing ordering

In D35964#816428, @mjg wrote:

i think this is somewhat suspicious. i think it would be safer to keep the track of the tail. this would still give you O(1) insertion without changing ordering

cl_siblings is only ever used in alto_hfsc.c, and the order never matters, so it seems fine to me to do this. We could keep the order by also tracking the tail as you say, but it doesn't buy us anything and costs (an infinitesimal amount of) memory.

This revision was not accepted when it landed; it landed in state Needs Review.Jul 28 2022, 8:11 PM
This revision was automatically updated to reflect the committed changes.

Yes, I also considered this. I spent some time reassuring myself that the order was not important, but of course I was interested in your feedback in case I missed something. I'm certainly willing to add a tail if the consensus indicates that is a good idea.