Page MenuHomeFreeBSD

Upgrade devel/py-mock from 2.0.0 to 3.0.5 and fix "make test" target
ClosedPublic

Authored by ngie on Sep 14 2019, 12:15 AM.
Tags
None
Referenced Files
Unknown Object (File)
Feb 25 2024, 2:39 AM
Unknown Object (File)
Feb 23 2024, 1:46 PM
Unknown Object (File)
Jan 31 2024, 8:38 AM
Unknown Object (File)
Jan 31 2024, 8:38 AM
Unknown Object (File)
Jan 31 2024, 8:38 AM
Unknown Object (File)
Jan 31 2024, 8:38 AM
Unknown Object (File)
Jan 31 2024, 8:38 AM
Unknown Object (File)
Jan 31 2024, 8:38 AM

Details

Summary

Version 3.0.5 includes multiple bugfixes and such for the backport library
that will aid in writing tests where the environment doesn't contain
unittest.mock.

In order to ensure that the code actually works, I needed to run the tests.
This required working around https://github.com/testing-cabal/mock/issues/469
by pulling the sources from the GitHub project that correspond with the
release tag, as the source tarball tagged for the release doesn't contain the
tests (prior to this change it would run 0 tests with make test, because
there weren't any).

Remove an unnecessary build dependency while here: pbr is no longer needed to
build the package.

Add py-pytest to TEST_DEPENDS, as running the tests as currently done via
distutils doesn't work due to how the tests are enumerated.

Conversely, remove the devel/py-tox dependency: it was never installed before
due to a typo in the variable name and it adds a lot of unnecessary
dependencies that aren't required when testing the port with make test. This
helps keep the footprint down, which helps others upgrade the devel/py-mock
port by itself.

Address portlint issues while here.

The changelog for the project can be found here on GitHub.

Test Plan

portlint/make test

1+ sudo make clean
2===> Cleaning for py36-mock-3.0.5
3===> Cleaning for py27-mock-3.0.5
4+ DEVELOPER=yes
5+ portlint -Ca
6looks fine.
7+ sudo make test
8===> License BSD2CLAUSE accepted by the user
9===> py36-mock-3.0.5 depends on file: /usr/local/sbin/pkg - found
10===> Fetching all distfiles required by py36-mock-3.0.5 for building
11===> Extracting for py36-mock-3.0.5
12=> SHA256 Checksum OK for testing-cabal-mock-3.0.5-e0180b98d0e07e895a3f699b7e9afcac4716fc03_GH0.tar.gz.
13===> Patching for py36-mock-3.0.5
14===> py36-mock-3.0.5 depends on package: py36-six>=1.9 - found
15===> py36-mock-3.0.5 depends on package: py36-setuptools>0 - found
16===> py36-mock-3.0.5 depends on file: /usr/local/bin/python3.6 - found
17===> Configuring for py36-mock-3.0.5
18running config
19===> Building for py36-mock-3.0.5
20running build
21running build_py
22creating build
23creating build/lib
24creating build/lib/mock
25copying mock/__init__.py -> build/lib/mock
26copying mock/mock.py -> build/lib/mock
27===> Staging for py36-mock-3.0.5
28===> py36-mock-3.0.5 depends on package: py36-setuptools>0 - found
29===> py36-mock-3.0.5 depends on file: /usr/local/bin/python3.6 - found
30===> Generating temporary packing list
31running install
32running build
33running build_py
34running install_lib
35creating /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6
36creating /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6/site-packages
37creating /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6/site-packages/mock
38copying build/lib/mock/__init__.py -> /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6/site-packages/mock
39copying build/lib/mock/mock.py -> /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6/site-packages/mock
40byte-compiling /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6/site-packages/mock/__init__.py to __init__.cpython-36.pyc
41byte-compiling /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6/site-packages/mock/mock.py to mock.cpython-36.pyc
42writing byte-compilation script '/tmp/tmp_5m51khe.py'
43/usr/local/bin/python3.6 /tmp/tmp_5m51khe.py
44removing /tmp/tmp_5m51khe.py
45running install_egg_info
46running egg_info
47creating mock.egg-info
48writing mock.egg-info/PKG-INFO
49writing dependency_links to mock.egg-info/dependency_links.txt
50writing requirements to mock.egg-info/requires.txt
51writing top-level names to mock.egg-info/top_level.txt
52writing manifest file 'mock.egg-info/SOURCES.txt'
53reading manifest file 'mock.egg-info/SOURCES.txt'
54reading manifest template 'MANIFEST.in'
55writing manifest file 'mock.egg-info/SOURCES.txt'
56Copying mock.egg-info to /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6/site-packages/mock-3.0.5-py3.6.egg-info
57running install_scripts
58writing list of installed files to '/usr/ports/devel/py-mock/work-py36/.PLIST.pymodtmp'
59===> Creating unique files: Move MAN files needing SUFFIX
60===> Creating unique files: Move files needing SUFFIX
61====> Compressing man pages (compress-man)
62===> Testing for py36-mock-3.0.5
63===> py36-mock-3.0.5 depends on package: py36-pytest>0 - found
64===> py36-mock-3.0.5 depends on package: py36-tox>0 - found
65===> py36-mock-3.0.5 depends on file: /usr/local/bin/python3.6 - found
66============================= test session starts ==============================
67platform freebsd11 -- Python 3.6.9, pytest-4.5.0, py-1.8.0, pluggy-0.12.0
68rootdir: /usr/ports/devel/py-mock/work-py36/mock-e0180b98d0e07e895a3f699b7e9afcac4716fc03, inifile: setup.cfg
69collected 406 items
70
71mock/tests/testcallable.py ........... [ 2%]
72mock/tests/testhelpers.py .............................................. [ 14%]
73..ss....s....s.........s........ [ 21%]
74mock/tests/testhelpers_py3.py . [ 22%]
75mock/tests/testmagicmethods.py ............................s.....s [ 30%]
76mock/tests/testmock.py ................................................. [ 42%]
77................................................................ss...... [ 60%]
78 [ 60%]
79mock/tests/testpatch.py ................................................ [ 72%]
80............................................................ [ 87%]
81mock/tests/testsealable.py .................. [ 91%]
82mock/tests/testsentinel.py ...... [ 93%]
83mock/tests/testsupport.py .. [ 93%]
84mock/tests/testwith.py .......................... [100%]
85
86==================== 397 passed, 9 skipped in 1.39 seconds =====================

Sanity check

$ python3
Python 3.6.9 (default, Jul 11 2019, 01:10:57) 
[GCC 4.2.1 Compatible FreeBSD Clang 6.0.0 (tags/RELEASE_600/final 326565)] on freebsd11
Type "help", "copyright", "credits" or "license" for more information.
>>> import mock
>>> mock.__version__
'3.0.5'
>>> mock.__file__
'/usr/local/lib/python3.6/site-packages/mock/__init__.py'

Running poudriere:

$ make index
...
$ egrep "py(27|36)-mock" INDEX-11 | awk -F\| '$8 ~ /py(27|36)-mock/ { print $2 }' | sort -u
/usr/ports/devel/py-buildbot-www
/usr/ports/devel/py-oslotest
/usr/ports/devel/py-pycadf
/usr/ports/devel/py-pyee
/usr/ports/graphics/qgis
/usr/ports/graphics/qgis-ltr
/usr/ports/security/py-keystone
/usr/ports/www/py-textile
$ PORTS=$(egrep "py(27|36)-mock" INDEX-11 | awk -F\| '$8 ~ /py(27|36)-mock/ { print $2 }' | sort -u | sed -e 's,^/usr/ports/,,')
 sudo poudriere bulk -j 11-amd64 $PORTS
[00:00:00] Creating the reference jail... done
[00:00:03] Mounting system devices for 11-amd64-default
[00:00:03] Mounting ports/packages/distfiles
[00:00:03] Stashing existing package repository
[00:00:04] Mounting packages from: /poudriere/data/packages/11-amd64-default
/etc/resolv.conf -> /poudriere/data/.m/11-amd64-default/ref/etc/resolv.conf
[00:00:04] Starting jail 11-amd64-default
[00:00:04] Logs: /poudriere/data/logs/bulk/11-amd64-default/2019-09-26_11h54m44s
[00:00:04] Loading MOVED for /poudriere/data/.m/11-amd64-default/ref/usr/ports
[00:00:05] Ports supports: FLAVORS SELECTED_OPTIONS
[00:00:05] Gathering ports metadata
...

The rest of the output can be found in the following paste after having rebuilt the ports and their dependencies several times:

1Script started on Fri Sep 27 18:16:52 2019
2Command: sudo poudriere bulk -j 11-amd64 devel/py-buildbot-www devel/py-oslotest devel/py-pycadf devel/py-pyee graphics/qgis graphics/qgis-ltr security/py-keystone www/py-textile
3[00:00:00] Creating the reference jail... done
4[00:00:01] Mounting system devices for 11-amd64-default
5[00:00:01] Mounting ports/packages/distfiles
6[00:00:01] Using packages from previously failed build: /poudriere/data/packages/11-amd64-default/.building
7[00:00:01] Mounting packages from: /poudriere/data/packages/11-amd64-default
8/etc/resolv.conf -> /poudriere/data/.m/11-amd64-default/ref/etc/resolv.conf
9[00:00:01] Starting jail 11-amd64-default
10[00:00:01] Logs: /poudriere/data/logs/bulk/11-amd64-default/2019-09-27_18h16m52s
11[00:00:01] Loading MOVED for /poudriere/data/.m/11-amd64-default/ref/usr/ports
12[00:00:02] Ports supports: FLAVORS SELECTED_OPTIONS
13[00:00:02] Gathering ports metadata
14[00:00:07] Calculating ports order and dependencies
15[00:00:08] Sanity checking the repository
16[00:00:08] Checking packages for incremental rebuild needs
17[00:00:11] Deleting stale symlinks... done
18[00:00:11] Deleting empty directories... done
19[00:00:11] Cleaning the build queue
20[00:00:11] Sanity checking build queue
21[00:00:11] Processing PRIORITY_BOOST
22[00:00:11] Balancing pool
23[00:00:11] Recording filesystem state for prepkg... done
24[00:00:11] Building 3 packages using 3 builders
25[00:00:11] Starting/Cloning builders
26[00:00:12] Hit CTRL+t at any time to see build progress and stats
27[00:00:12] [01] [00:00:00] Building databases/grass7 | grass7-7.6.1_6
28[00:00:12] [01] [00:00:00] Finished databases/grass7 | grass7-7.6.1_6: Ignored: is marked as broken: fails to package
29[00:00:12] [01] [00:00:00] Skipping graphics/qgis | qgis-3.8.2: Dependent port databases/grass7 | grass7-7.6.1_6 ignored
30[00:00:12] [01] [00:00:00] Skipping graphics/qgis-ltr | qgis-ltr-3.4.11_1: Dependent port databases/grass7 | grass7-7.6.1_6 ignored
31[00:00:12] Stopping 3 builders
32[00:00:30] No package built, no need to update the repository
33[00:00:30] Committing packages to repository: /poudriere/data/packages/11-amd64-default/.real_1569633442 via .latest symlink
34[00:00:30] Removing old packages
35[00:00:30] Skipped ports: graphics/qgis graphics/qgis-ltr
36[00:00:30] Ignored ports: databases/grass7
37[11-amd64-default] [2019-09-27_18h16m52s] [committing:] Queued: 3 Built: 0 Failed: 0 Skipped: 2 Ignored: 1  Tobuild: 0 Time: 00:00:29
38[00:00:30] Logs: /poudriere/data/logs/bulk/11-amd64-default/2019-09-27_18h16m52s
39[00:00:30] Cleaning up
40[00:00:30] Unmounting file systems
41
42Command exit status: 2
43Script done on Fri Sep 27 18:17:22 2019
.

I can't build qgis and qgis-ltr though, because one of their dependencies, database/grass, is marked BROKEN:

$ grep BROK databases/grass7/Makefile 
BROKEN=         fails to package
BROKEN_sparc64= Does not configure on sparc64

Diff Detail

Repository
rP FreeBSD ports repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

bdrewery added inline comments.
devel/py-mock/Makefile
16–17 ↗(On Diff #62078)

Are these versions still accurate?

$ DEVELOPER=yes portlint -Ca
WARN: Makefile: [28]: use a tab (not space) after a variable name
0 fatal errors and 1 warning found.
ngie edited the test plan for this revision. (Show Details)
ngie marked an inline comment as done.
ngie added inline comments.
devel/py-mock/Makefile
16–17 ↗(On Diff #62078)

pbr's no longer needed, but six is still needed. The underlying code doesn't specify a specific six package version -- it just requires one to be installed.

ngie marked an inline comment as done.

Update the diff after feedback received and to fix the test target
by using github to pull down the sources and pytest to run the tests,
like tox.ini does.

Remove devel/py-tox dependency

It creates a lot of additional external dependencies and isn't strictly
required to run the tests.

ngie retitled this revision from Upgrade devel/py-mock from 2.0.0 to 3.0.5 to Upgrade devel/py-mock from 2.0.0 to 3.0.5 and fix "make test" target.Sep 18 2019, 2:12 PM
ngie edited the summary of this revision. (Show Details)

Hi ngie,

Can you please complete build testing ("poudriere bulk -j 12amd64 -t devel/py-buildbot-www devel/py-oslotest ...") for the following ports which depend on this port?

As shown by https://www.freshports.org/devel/py-mock:

This port is required by:

for Build

        devel/py-buildbot-www
        devel/py-oslotest
        graphics/qgis
        graphics/qgis-ltr
        security/py-keystone

for Run

        audio/py-gmusicapi
        devel/py-oslotest
        devel/py-pytest-shutil
        devel/py-responses
        devel/py-sure
        devel/py-vsts-cd-manager
        graphics/qgis

        graphics/qgis-ltr
        science/py-MDAnalysis
        science/py-MDAnalysisTests
        security/py-certbot-apache
        security/py-certbot-dns-cloudflare
        security/py-certbot-dns-cloudxns
        security/py-certbot-dns-digitalocean
        security/py-certbot-dns-dnsimple
        security/py-certbot-dns-dnsmadeeasy
        security/py-certbot-dns-gehirn
        security/py-certbot-dns-google
        security/py-certbot-dns-linode
        security/py-certbot-dns-luadns
        security/py-certbot-dns-nsone
        security/py-certbot-dns-ovh
        security/py-certbot-dns-rfc2136
        security/py-certbot-dns-route53
        security/py-certbot-dns-sakuracloud
        security/py-certbot-nginx
        sysutils/py-azure-cli
        sysutils/py-azure-cli-batchai
        textproc/py-mkdocs
        www/py-kallithea
        www/py-rhodecode

Hi ngie,

Can you please complete build testing ("poudriere bulk -j 12amd64 -t devel/py-buildbot-www devel/py-oslotest ...") for the following ports which depend on this port?

As shown by https://www.freshports.org/devel/py-mock:

This port is required by:

for Build

        devel/py-buildbot-www
        devel/py-oslotest
        graphics/qgis
        graphics/qgis-ltr
        security/py-keystone

for Run

        audio/py-gmusicapi
        devel/py-oslotest
        devel/py-pytest-shutil
        devel/py-responses
        devel/py-sure
        devel/py-vsts-cd-manager
        graphics/qgis

        graphics/qgis-ltr
        science/py-MDAnalysis
        science/py-MDAnalysisTests
        security/py-certbot-apache
        security/py-certbot-dns-cloudflare
        security/py-certbot-dns-cloudxns
        security/py-certbot-dns-digitalocean
        security/py-certbot-dns-dnsimple
        security/py-certbot-dns-dnsmadeeasy
        security/py-certbot-dns-gehirn
        security/py-certbot-dns-google
        security/py-certbot-dns-linode
        security/py-certbot-dns-luadns
        security/py-certbot-dns-nsone
        security/py-certbot-dns-ovh
        security/py-certbot-dns-rfc2136
        security/py-certbot-dns-route53
        security/py-certbot-dns-sakuracloud
        security/py-certbot-nginx
        sysutils/py-azure-cli
        sysutils/py-azure-cli-batchai
        textproc/py-mkdocs
        www/py-kallithea
        www/py-rhodecode

Will do. Thank you for the pointer.

How did you determine this list BTW: using INDEX, etc?

devel/py-mock/Makefile
18 ↗(On Diff #62269)

I think I was a bit hasty removing this line. Yes, six is needed for runtime use of the mock library.

In D21654#473469, @ngie wrote:

How did you determine this list BTW: using INDEX, etc?

Just using that FreshPorts link I included in my previous comment :)

Restore the previous RUN_DEPENDS logic

In D21654#482501, @ngie wrote:

The poudriere test completeness is unclear to me. It may seem like a big ask but I'm sure you understand the bound version dependency problem between python ports. I think you did run it but your paste was cut off or something. Just confirm you ran as Ben requested and I'll +1 this.

devel/py-mock/Makefile
27 ↗(On Diff #62293)

Is this line necessary? IMHO it only complicates future updating, slightly. What do I miss?

devel/py-mock/Makefile
27 ↗(On Diff #62293)

Neither GH_PROJECT now GH_TAGNAME should be needed here. I see a 3.0.5 tag. I think it should just work.

Use the PORTVERSION for GH_TAGNAME instead of a full hash

ngie marked 2 inline comments as done.Nov 5 2019, 5:44 PM
In D21654#482501, @ngie wrote:

The poudriere test completeness is unclear to me. It may seem like a big ask but I'm sure you understand the bound version dependency problem between python ports. I think you did run it but your paste was cut off or something. Just confirm you ran as Ben requested and I'll +1 this.

I ran it the best I could, as @woodsb02 requested.

Use CHEESESHOP again.

This appears to be working now -- not sure why it wasn't when I tried earlier.

Oh wait... GitHub was required because the release tarball doesn't contain the tests.

Urhhhhhghhhhhhh... I wish this would just get merged already.

devel/py-mock/Makefile
27 ↗(On Diff #62293)

@bdrewery: could you please confirm that the above change is what you requested?

devel/py-mock/Makefile
27 ↗(On Diff #62293)

Not exactly, I believe. You can completely delete those two lines. You assign the same values which they get by default.

I tested your patch without those lines for py36 flavor: 'port test', portmaster, poudriere, 'make test' - all ran OK.

ngie marked an inline comment as done.

Remove unnecessary GitHub boilerplate noted by @bdrewery and @Vladimir.Chukharev_gmail.com .

devel/py-mock/Makefile
27 ↗(On Diff #62293)

@Vladimir.Chukharev_gmail.com: I appreciate the additional explanation (the previous reason wasn't gelling with me for some reason). I just updated the diff with your and @bdrewery's recommended change.

I am ok with this change, and provide my approval to proceed as a ports committer.
@ngie - would you like to commit this, or would you like me to? Please note the maintainer timeout discussion below.

TESTING
I completed build testing of the ports which depend on devel/py-mock using the commands below. This was successful (no regressions, all build successfully).

# poudriere bulk -j 12amd64 -t devel/py-buildbot-www devel/py-oslotest graphics/qgis graphics/qgis-ltr security/py-keystone audio/py-gmusicapi devel/py-oslotest devel/py-pytest-shutil devel/py-responses devel/py-sure devel/py-vsts-cd-manager graphics/qgis graphics/qgis-ltr science/py-MDAnalysis science/py-MDAnalysisTests security/py-certbot-apache security/py-certbot-dns-cloudflare security/py-certbot-dns-cloudxns security/py-certbot-dns-digitalocean security/py-certbot-dns-dnsimple security/py-certbot-dns-dnsmadeeasy security/py-certbot-dns-gehirn security/py-certbot-dns-google security/py-certbot-dns-linode security/py-certbot-dns-luadns security/py-certbot-dns-nsone security/py-certbot-dns-ovh security/py-certbot-dns-rfc2136 security/py-certbot-dns-route53 security/py-certbot-dns-sakuracloud security/py-certbot-nginx sysutils/py-azure-cli textproc/py-mkdocs www/py-kallithea www/py-rhodecode

MAINTAINER TIMEOUT
I noticed the maintainer has not been engaged for this update, so I have requested maintainer feedback here:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=241780

Note that there is no requirement to wait 2 weeks as per the normal process, since the last 3 updates all suffered from maintainer timeout, and the maintainer hasn't submitted any changes to this port since 2012.
https://www.freebsd.org/doc/en/books/porters-handbook/makefile-maintainer.html

I suggest waiting a couple of days to give Jesse a chance to see this request, but if no feedback progress with committing this change and reset the maintainer (either to ports@FreeBSD.org or take maintainership).

This revision is now accepted and ready to land.Nov 7 2019, 11:17 AM