Change Details

Allow consumers to get one state at a time, rather than having to retrieve them all at once before we begin processing. Especially for very large state tables this can significantly reduce memory use. With this change time -l reports "3280 maximum resident set size" retrieving ~5M states. Without it it reported "2209008 maximum resident set size" for 2M states. Sponsored by: Rubicon Communications, LLC ("Netgate")

libpfctl: introduce state iterator Allow consumers to start processing states as the kernel supplies them, rather than having to build a full list and only then start processing. Especially for very large state tables this can significantly reduce memory use. With this change when retrieving 1M states time -l reports: real 3.55 user 1.95 sys 1.05 318832 maximum resident set size 194 average shared memory size 15 average unshared data size 127 average unshared stack size 79041 page reclaims 0 page faults 0 swaps 0 block input operations 0 block output operations 15096 messages sent 250001 messages received 0 signals received 22 voluntary context switches 34 involuntary context switches Without it it reported: real 3.32 user 1.88 sys 0.86 3220 maximum resident set size 195 average shared memory size 11 average unshared data size 128 average unshared stack size 260 page reclaims 0 page faults 0 swaps 0 block input operations 0 block output operations 15096 messages sent 250001 messages received 0 signals received 21 voluntary context switches 31 involuntary context switches Sponsored by: Rubicon Communications, LLC ("Netgate")

libpfctl: introduce state iterator Allow consumers to get onestart processing state at a time, rather than having tos as the kernel supplies them, retrieve them all at once before we begin processing. Especially forrather than having to build a full list and only then start processing. Especially for very large state tables this can significantly reduce memory use. With this change time -l reports "3280 maximum resident set size"when retrieving 1M states time -l reports: real 3.55 retrieving ~5M states. Without it it reported "2209008 maximum resident user 1.95 set size" for 2M states. sys 1.05 318832 maximum resident set size 194 average shared memory size 15 average unshared data size 127 average unshared stack size 79041 page reclaims 0 page faults 0 swaps 0 block input operations 0 block output operations 15096 messages sent 250001 messages received 0 signals received 22 voluntary context switches 34 involuntary context switches Sponsored by: Without it it reported: real 3.32 user 1.88 sys 0.86 3220 maximum resident set size 195 average shared memory size 11 average unshared data size 128 average unshared stack size 260 page reclaims 0 page faults 0 swaps 0 block input operations 0 block output operations 15096 messages sent 250001 messages received 0 signals received 21 voluntary context switches 31 involuntary context switches Sponsored by: Rubicon Communications, LLC ("Netgate")