libpfctl: introduce state iterator

[kp]

Summary

libpfctl: introduce state iterator

Allow consumers to start processing states as the kernel supplies them,
rather than having to build a full list and only then start processing.
Especially for very large state tables this can significantly reduce
memory use.

Without this change when retrieving 1M states time -l reports:

real 3.55
user 1.95
sys 1.05
    318832  maximum resident set size
       194  average shared memory size
        15  average unshared data size
       127  average unshared stack size
     79041  page reclaims
         0  page faults
         0  swaps
         0  block input operations
         0  block output operations
     15096  messages sent
    250001  messages received
         0  signals received
        22  voluntary context switches
        34  involuntary context switches

With it it reported:

real 3.32
user 1.88
sys 0.86
      3220  maximum resident set size
       195  average shared memory size
        11  average unshared data size
       128  average unshared stack size
       260  page reclaims
         0  page faults
         0  swaps
         0  block input operations
         0  block output operations
     15096  messages sent
    250001  messages received
         0  signals received
        21  voluntary context switches
        31  involuntary context switches

Sponsored by: Rubicon Communications, LLC ("Netgate")

[mjg]

this description is rather misleading, suggesting states get fetched one at a time. instead they are already all fetched, except there is a per-state callback

you should report include user/sys/real reports as well as far a time(1) goes

[kp]

In D42091#960178, @mjg wrote:

this description is rather misleading, suggesting states get fetched one at a time. instead they are already all fetched, except there is a per-state callback

It does actually start processing before we get the complete list. That's one of the reasons we don't use the same stupid amounts of memory as we did before.

you should report include user/sys/real reports as well as far a time(1) goes

Done.