Details

Reviewers

Summary

We have created a preliminary test suite for kernel pf tests and
integrated it under FreeBSD's src/tests using kyua/ATF. The tests
reside under src/tests/sys/netpfil/pf, and can get installed
under /usr/tests.

The tests consist mostly of connecting to the test target machine
via SSH, activating pf and possibly forwarding, and then running
tests. We use scapy (runnning on Python) for some of the tests.

The host machine is connected to the test machine via several
interfaces. Usually we have one interface for SSH, and two or
three for the tests. The reason for that is so that we can have
different pf rules on the test machine for the different
interfaces, which helps verify the validity of the tests by
looking at the difference in behavior of the different
interfaces.

Currently the tests need the user to manually change
configuration parameters for all addresses and names of
interfaces used.

The tests are put together using FreeBSD's recommended test suite
which uses kyua. The tests are hierarchically organized under
src/tests using a structure of Kyuafile files.

To run the tests, first run 'make'. Then, run either 'kyua test'
as root or 'make install' as root followed by 'kyua test' as root
under /usr/tests.

Test Plan

Since this is a test tool, there is no separate test plan. Running
the tool and proving its usefulness can provide adequate testing.

However, a simple way to test that the tests are working is to
run the simple tests pf_test:remote_block_return and
pf_test:remote_block_drop.

Diff Detail

Repository

rS FreeBSD src repository - subversion

Lint

Lint Skipped

Unit

Tests Skipped

Event Timeline

paggas1_yandex.com created this revision.Jun 29 2017, 1:25 AM

Herald added a reviewer: kp. · View Herald TranscriptJun 29 2017, 1:25 AM

Herald added subscribers: Contributor Reviews (src), imp. · View Herald Transcript

We're intermingling two sets of changes here.

Can you update D11322 with this version of the pfctl tests? I think those are getting close to ready for inclusion. Let's keep this review for the netpfil tests, without the pfctl test files.
(Make sure to include the pfctl kyuafile, because I think it's missing here.)

Will do!

There is no Kyuafile anymore, it gets generated with 'make' now. Same as
with other tests in the suite, I'm calling FreeBSD's standard makefiles
now :)

Den 2017-07-01 kl. 12:55, skrev kristof (Kristof Provost):

kristof added a comment.

We're intermingling two sets of changes here.

Can you update https://reviews.freebsd.org/D11322 with this version of the pfctl tests? I think those are getting close to ready for inclusion. Let's keep this review for the netpfil tests, without the pfctl test files.
(Make sure to include the pfctl kyuafile, because I think it's missing here.)

REPOSITORY

rS FreeBSD src repository

REVISION DETAIL

https://reviews.freebsd.org/D11401

EMAIL PREFERENCES

https://reviews.freebsd.org/settings/panel/emailpreferences/

To: paggas1_yandex.com, kristof
Cc: imp, Contributor Reviews (src)

Only the diff pertaining to kernel pf tests is now included.

paggas1_yandex.com retitled this revision from Preliminary kernel pf tests to Kernel pf tests.Aug 28 2017, 10:55 PM

paggas1_yandex.com edited the summary of this revision. (Show Details)

paggas1_yandex.com edited the test plan for this revision. (Show Details)

New version.

More work.

Work on make_baseimg.sh.

More work.

Fix mtree and makefiles.
Split tests into separate files.
Some documentation.
Improve error handling and reporting.
Some style fixing.

New version. A summary of changes since the last one, pulled from git, is shown below:

Made VMs boot faster via autoboot_delay in loader.conf.
Fixed chflags problem in make_baseimg.sh.
Fail instead of skipping if VM not start.
make_baseimg.sh check for root.
Fixed coding style.
Not exporting PF_TEST_DIR.
scrub_forward6 complete
Renamed scrub6.py to scrub_forward6.py.
Removed conf.py.
Load nmdm when testing.
Updated documentation of pf_test_scrub_forward6.sh.
Renamed test scrub_forward6 to scrub6.
Added comment in scrub6.py.
Added new idea in "Future work" section of README.
Added address prefix variable in test scrub_forward.

kp added inline comments.Sep 8 2017, 11:48 AM

tests/sys/netpfil/pf/files/make_baseimg.sh
101	I'm seeing these errors building the base image: pkg: Error fetching http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/Latest/pkg.txz: No address record A pre-built version of pkg could not be found for your system. Consider changing PACKAGESITE or installing it from ports: 'ports-mgmt/pkg'. Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest, please wait... make_baseimg.sh: Cannot install packages into image file, is there an active internet connection? I don't immediately see anything configuring the name server in the chroot, so perhaps that's the trigger. Am I missing something?

While looking at this I also noticed that this file is (mostly) intended with spaces. FreeBSD style is to use tabs.

tests/sys/netpfil/pf/files/make_baseimg.sh

101

This seems to help:

diff --git a/tests/sys/netpfil/pf/files/make_baseimg.sh b/tests/sys/netpfil/pf/files/make_baseimg.sh
index 6c303a552f7..187c60b0dcc 100755
--- a/tests/sys/netpfil/pf/files/make_baseimg.sh
+++ b/tests/sys/netpfil/pf/files/make_baseimg.sh
@@ -97,6 +99,10 @@ image file malformed?"
             error "Cannot edit loader.conf on image file!"
             return 1
         }
+        cp /etc/resolv.conf "${mountdir}/etc/resolv.conf" || {
+            error "Cannot set resolv.conf in the image file"
+            return 1
+        }
         chroot "${mountdir}" \
                env ASSUME_ALWAYS_YES="yes" \
                pkg install ${packages} || {

I've made a change which I'm testing currently, running pkg with the -r
flag instead of chrooting. Will commit if it works.

Den 2017-09-08 kl. 11:48, skrev kristof (Kristof Provost):

kristof added inline comments.

INLINE COMMENTS

make_baseimg.sh:100
+ }
+ chroot "${mountdir}" \
+ env ASSUME_ALWAYS_YES="yes" \

I'm seeing these errors building the base image:
pkg: Error fetching http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/Latest/pkg.txz: No address record
A pre-built version of pkg could not be found for your system.
Consider changing PACKAGESITE or installing it from ports: 'ports-mgmt/pkg'.
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest, please wait...
make_baseimg.sh: Cannot install packages into image file, is there an active internet connection?
I don't immediately see anything configuring the name server in the chroot, so perhaps that's the trigger. Am I missing something?

REPOSITORY
rS FreeBSD src repository
REVISION DETAIL
https://reviews.freebsd.org/D11401
EMAIL PREFERENCES
https://reviews.freebsd.org/settings/panel/emailpreferences/
To: paggas1_yandex.com, kristof
Cc: imp, Contributor Reviews (src)

I had already changed the coding style, mostly regarding error checking,
to use if blocks when suitable. I didn't change the indentation
however, as it needed some rethinking regarding error checking and
cleanup code. I will change it to use tabs as soon as possible.

Den 2017-09-08 kl. 13:33, skrev kristof (Kristof Provost):

kristof added a comment.

While looking at this I also noticed that this file is (mostly) intended with spaces. FreeBSD style is to use tabs.

INLINE COMMENTS

kristof wrote in make_baseimg.sh:100
I'm seeing these errors building the base image:
pkg: Error fetching http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/Latest/pkg.txz: No address record
A pre-built version of pkg could not be found for your system.
Consider changing PACKAGESITE or installing it from ports: 'ports-mgmt/pkg'.
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest, please wait...
make_baseimg.sh: Cannot install packages into image file, is there an active internet connection?
I don't immediately see anything configuring the name server in the chroot, so perhaps that's the trigger. Am I missing something?

This seems to help:

diff --git a/tests/sys/netpfil/pf/files/make_baseimg.sh b/tests/sys/netpfil/pf/files/make_baseimg.sh
index 6c303a552f7..187c60b0dcc 100755
--- a/tests/sys/netpfil/pf/files/make_baseimg.sh
+++ b/tests/sys/netpfil/pf/files/make_baseimg.sh
@@ -97,6 +99,10 @@ image file malformed?"
             error "Cannot edit loader.conf on image file!"
             return 1
         }
+        cp /etc/resolv.conf "${mountdir}/etc/resolv.conf" || {
+            error "Cannot set resolv.conf in the image file"
+            return 1
+        }
         chroot "${mountdir}" \
                env ASSUME_ALWAYS_YES="yes" \
                pkg install ${packages} || {

REPOSITORY

rS FreeBSD src repository

REVISION DETAIL

https://reviews.freebsd.org/D11401

EMAIL PREFERENCES

https://reviews.freebsd.org/settings/panel/emailpreferences/

To: paggas1_yandex.com, kristof
Cc: imp, Contributor Reviews (src)

In D11401#254776, @paggas1_yandex.com wrote:

I've made a change which I'm testing currently, running pkg with the -r
flag instead of chrooting. Will commit if it works.

Yes, that's even better.

New version. A summary of changes since the last one, pulled from git, is shown below:

No chroot in make_baseimg.sh.
Removed scrub_pflog.py.

New version. A summary of changes since the last one, pulled from git, is shown below:

tests: pf: Typo.
tests: pf: Added address prefix variables in all tests.
tests: pf: Fixed scrub_forward by changing implementation.
tests: pf: Renamed test scrub_forward to scrub.
tests: pf: Updated documentation of pf_test_scrub.sh.
tests: pf: Changed idea in "Future work" section of README.
tests: pf: Reduced boilerplate code.
tests: pf: Reduced boilerplate code, IPv6 functions.
tests: pf: Changed test names.
tests: pf: Improved network interface creation code.
tests: pf: Function calling examples.
tests: pf: Added init_test function.

kp commandeered this revision.Oct 28 2018, 5:57 AM

kp edited reviewers, added: paggas1_yandex.com; removed: kp.

Herald added a subscriber: farrokhi. · View Herald TranscriptOct 28 2018, 5:57 AM

An alternative approach (VIMAGE based) was committed instead.

Kernel pf tests
AbandonedPublic
Actions

Details

Diff Detail

Event Timeline

Revision Contents
Changeset List

Diff 32470

etc/mtree/BSD.tests.dist

sbin/pfctl/tests/files/pfctl_test_descr.sh

targets/pseudo/tests/Makefile.depend

tests/sys/Makefile

tests/sys/netpfil/Kyuafile

tests/sys/netpfil/Makefile

tests/sys/netpfil/pf/Makefile

tests/sys/netpfil/pf/README

tests/sys/netpfil/pf/files/Makefile

tests/sys/netpfil/pf/files/conf.py

tests/sys/netpfil/pf/files/make_baseimg.sh

tests/sys/netpfil/pf/files/pf_test_conf.sh

tests/sys/netpfil/pf/files/pf_test_util.sh

tests/sys/netpfil/pf/files/scrub6.py

tests/sys/netpfil/pf/files/scrub_forward.py

tests/sys/netpfil/pf/files/scrub_pflog.py

tests/sys/netpfil/pf/files/util.py

tests/sys/netpfil/pf/files/vmctl.sh

tests/sys/netpfil/pf/files/vmctl.sh.zvol

tests/sys/netpfil/pf/pf_test.sh

Kernel pf testsAbandonedPublicActions

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 32470

etc/mtree/BSD.tests.dist

sbin/pfctl/tests/files/pfctl_test_descr.sh

targets/pseudo/tests/Makefile.depend

tests/sys/Makefile

tests/sys/netpfil/Kyuafile

tests/sys/netpfil/Makefile

tests/sys/netpfil/pf/Makefile

tests/sys/netpfil/pf/README

tests/sys/netpfil/pf/files/Makefile

tests/sys/netpfil/pf/files/conf.py

tests/sys/netpfil/pf/files/make_baseimg.sh

tests/sys/netpfil/pf/files/pf_test_conf.sh

tests/sys/netpfil/pf/files/pf_test_util.sh

tests/sys/netpfil/pf/files/scrub6.py

tests/sys/netpfil/pf/files/scrub_forward.py

tests/sys/netpfil/pf/files/scrub_pflog.py

tests/sys/netpfil/pf/files/util.py

tests/sys/netpfil/pf/files/vmctl.sh

tests/sys/netpfil/pf/files/vmctl.sh.zvol

tests/sys/netpfil/pf/pf_test.sh

Kernel pf tests
AbandonedPublic
Actions

Revision Contents
Changeset List