4. In the meantime the kernel delayed object reclamation model issue is fixed, maybe adapt 80fc25025ffcb0d369fc0b6d4d272ad6fd3f53c3 ?

Committed in 20ea7f26e413 .

3. An atf_get_srcdir-equivalent to eventually read files relative to the source directory, however I think the idea is to disassociate from ATF later on.

This one slipped through the cracks, thanks for reminding! Will add in a day or two.

I thought of passing it via wrapper initially, but ended up with a bit more natural way - getting it from pytest itself.
I committed an example here.

In D38053#867487, @ngie wrote:

In D38053#864580, @asomers wrote:

This looks like a good start. I'm not a fan of the methodize stuff, though. I think it makes the tests harder to read and harder to search. Is there a way to parameterize the tests without using methodize, even if it becomes slightly more verbose?

If you’re referring to “pytest.marks.parametrize”, it supports an optional id= parameter which better describes tests for humans.

In D38053#864580, @asomers wrote:

This looks like a good start. I'm not a fan of the methodize stuff, though. I think it makes the tests harder to read and harder to search. Is there a way to parameterize the tests without using methodize, even if it becomes slightly more verbose?

In D38053#866669, @melifaro wrote:

In D38053#865570, @jlduran_gmail.com wrote:

Thank you! I'll wait for @asomers comments to update the revision.

I'll document here my small - not really important - wishlist:

1. Optional randomization of the test order, maybe something similar to pytest-random-order.

Nice one, can probably work out-of-the box, but the scope will be a single file, due to the way of ATF<>pytest interaction is implemented.

2. When debugging atf-python tests, the output could be a little bit less busy, maybe pytest_terminal_summary et al. instead of print() could be one answer.

I get the intent and I agree. Maybe you could come up with an example (or even a diff) on how do you see it?

Sure! Once all ping tests are added, I should be more acquainted with the testing framework and propose a revision.

3. An atf_get_srcdir-equivalent to eventually read files relative to the source directory, however I think the idea is to disassociate from ATF later on.

This one slipped through the cracks, thanks for reminding! Will add in a day or two.

4. In the meantime the kernel delayed object reclamation model issue is fixed, maybe adapt 80fc25025ffcb0d369fc0b6d4d272ad6fd3f53c3 ?

Yep, been on my list for quite some time, I'll do the change in a day or two.

Great!

In D38053#865570, @jlduran_gmail.com wrote:

Thank you! I'll wait for @asomers comments to update the revision.

I'll document here my small - not really important - wishlist:

1. Optional randomization of the test order, maybe something similar to pytest-random-order.

Nice one, can probably work out-of-the box, but the scope will be a single file, due to the way of ATF<>pytest interaction is implemented.

2. When debugging atf-python tests, the output could be a little bit less busy, maybe pytest_terminal_summary et al. instead of print() could be one answer.

I get the intent and I agree. Maybe you could come up with an example (or even a diff) on how do you see it?

3. An atf_get_srcdir-equivalent to eventually read files relative to the source directory, however I think the idea is to disassociate from ATF later on.

This one slipped through the cracks, thanks for reminding! Will add in a day or two.

4. In the meantime the kernel delayed object reclamation model issue is fixed, maybe adapt 80fc25025ffcb0d369fc0b6d4d272ad6fd3f53c3 ?

Yep, been on my list for quite some time, I'll do the change in a day or two.

Thank you! I'll wait for @asomers comments to update the revision.

So generally, it looks good to me, and I'm fine with committing the change. @asomers: what do you think?

In D38053#865288, @melifaro wrote:

In D38053#865132, @jlduran_gmail.com wrote:

Here are the last iteration changes:

• Use a dictionary for expectations — this keeps both ping and pinger tests inline. Comparing the expected with the actual subprocess.CompletedProcess was also not feasible. It also was suggested initially by @melifaro.

One thing I noticed is that atf-python tests are slower than atf-sh, this was somewhat expected, but when all tests run, the total time builds up. I believe the gains are really from the development perspective.

Q: what's the runtime? e.g. median time reported by kyua test?

In D38053#865132, @jlduran_gmail.com wrote:

Here are the last iteration changes:

• Use a dictionary for expectations — this keeps both ping and pinger tests inline. Comparing the expected with the actual subprocess.CompletedProcess was also not feasible. It also was suggested initially by @melifaro.

One thing I noticed is that atf-python tests are slower than atf-sh, this was somewhat expected, but when all tests run, the total time builds up. I believe the gains are really from the development perspective.

Q: what's the runtime? e.g. median time reported by kyua test?
There are some tests in net/routing, doing similar SingleVnet-isolated testing, written both in python and in C.
C version takes ~120ms per test, python version takes ~330ms. My take is that the runtime under 0.5 second is fine. It may also be not easy to decrease those 330ms, as pytest does a lot of preparation work, but the runner calls it for a single test.
I suspect that importing scapy.all may contribute to the delay here. Note that kyua needs to first run list for the tests (to determine the isolation details) and then actual test and cleanup procedures.
Importing scapy.all in the top of the file will cause all 3 cases to wait till scapy init. I'd probably try to check if it's possible to use scapy subset and/or try to load it only when needed.

Here are the last iteration changes:

Updates:

• Simple one-line ping parameter expectations are subprocess.CompletedProcesses. No need to have a custom class (ExpectedProcess) that does essentially the same thing.
• Incorporate the code from D37876. The pinger utility (now a function) returns a subprocess.CompletedProcess as well. Parametrization and cleanup are next.

In D38053#864674, @melifaro wrote:

Thank you for addressing the comments!
I’d still prefer to have the ids embedded explicitly via pytest.param, but I don’t insist on doing it.

Generally there may be somewhat conflicting ideas on what’s the “best” approach here, but personally I don’t think it’s a big deal here. This is a test code, it already is reasonably simple and allow for easy extension or debugging - that’s “good enough” and that’s what matters most (to me).

Address more suggestions:

• Embed the ids explicitly via pytest.param().

Thank you for addressing the comments!
I’d still prefer to have the ids embedded explicitly via pytest.param, but I don’t insist on doing it.

Address some of the suggestions:

In D38053#864580, @asomers wrote:

This looks like a good start. I'm not a fan of the methodize stuff, though. I think it makes the tests harder to read and harder to search. Is there a way to parameterize the tests without using methodize, even if it becomes slightly more verbose?

Conceptually LGTM, please see some comments on the structure

This looks like a good start. I'm not a fan of the methodize stuff, though. I think it makes the tests harder to read and harder to search. Is there a way to parameterize the tests without using methodize, even if it becomes slightly more verbose?

Update to 8.0.0.18

• Update WWW: to Makefile

• Change versioning to handbook: Example 19. Using USE_GITHUB to Access a Commit Between Two Versions
• Update WWW: to Makefile

@dereks_lifeofadishwasher.com Can we the/an existing version format that upstream uses (vX.Y.Z), optionally suffixed appropriately with relevent git tag if its 'in between versions' ?

Move to a GH_TAGNAME from the released 8.0.0 to include some IMAP fixes
All tests pass.

• Use USES=pytest

portfmt sort order

Yes, that works well. Thanks for the recommendation.

• update for USES=pytest

LGTM mate, nice work!

er sorry I misread that. Ok added comment.

• Add comment stating why USE_GITHUB vs sdist

As expected. I do USES -> USE_* -> *_FOO, but wont block review on that.

They do disagree. 15.8 porters handbook recommends keeping USE_* vars together (using USE_GITHUB GH_* as example).

Upstream updated https://github.com/sethmlarson/rfc6555/pull/8 and prefer not to add test to sdist.

All review comments have been addressed. LGTM.

Tested on py37:

• Added patch to fix idle folder with spaces (forget this from the init. review).
• Capped offlineimap3 to 3.7-3.9 since imaplib2 is limited to 3.9. In turn only testing offlineimap3 for py37/8/9
• Patched docs sphinx-build command to use the correct conncurrent version. Will need to fix upstream.
• Patched out tests that require a network connection and user auth. Maybe have upstream move these out of unittest?

Run unittests with python37, 38. 39. 310, and 311

Sorry in the wrong branch. Updating from correct one.

• Add comments for build depends and pypi packaging
• include CONFLICTS_INSTALL just in case someone is still using the removed py-offlineimap package

Added tests and moved to github until tests are added

• Add TEST_DEPENDS and (do-)test target
• Confirm QA (Added to TEST PLAN section)

• Add TEST_DEPENDS and (do-)test target for QA
• Need confirmation of QA in TEST PLAN

missing file to remove

Delete old patches from wxPython30

rebase

In D21915#522759, @lbartoletti_tuxfamily.org wrote:

My biggest fear with svn :)
How to be sure? I see that x11-toolkits/py-wxPython40 is copied from x11-toolkits/py-wxPython30. OK, but I deleted useless patches [1] and are not marked as deleted

[1] files/patch-src_gtk_propgridwrap.cpp, files/patch-src_gtk_gdi__wrap.cpp, files/patch-setup.py

In D21915#522637, @tcberner wrote:

Looks good to me. Make sure to include

Exp-run by: antoine

Looks good to me. Make sure to include

exp-run looks fine https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=241893#c13

Fix my mail adress

Bump portversion for graphics/djvusmooth

Bump portversion for all ports which require wxPython40

Replace LIB_DEPENDS to USE_WX for graphics/djvusmooth

Update comms/congruity and graphics/py-mayavi to USE_WX=3.0+

Bump portversion emulators/playonbsd

bump portversion for cad/kicad and cad/kicad-devel

Fix cad/kicad-devel

Bump portrevision for audio/py-karaoke

Fix build for audio/py-karaoke. Use wxgtk-2.8

• Update Mk/bsd.wx.mk to use x11-toolkits/py-wxPython40 for wxWidgets 3.0
• While I'm here fix stagging for emulators/playonbsd
• Fix Makefile for cad/kicad (explicitly use wxPyton40 aka Phoenix)

Fix stripcmd. Thanks koobs@

gentle ping koobs@

New diff:

If this is a copy of an existing port, create the new port using svn copy (as this is what committers will have to do). Also have the benefits of that copy being tracked and displayed in the phabricator differential, and showing the diffs to the original source (so we know whats been updated/changed, rather than everything being new)

Fix pkg-descr