Page MenuHomeFreeBSD

Upgrade devel/py-mock from 2.0.0 to 3.0.5 and fix "make test" target
Needs ReviewPublic

Authored by ngie on Sep 14 2019, 12:15 AM.

Details

Summary

Version 3.0.5 includes multiple bugfixes and such for the backport library
that will aid in writing tests where the environment doesn't contain
unittest.mock.

In order to ensure that the code actually works, I needed to run the tests.
This required working around https://github.com/testing-cabal/mock/issues/469
by pulling the sources from the GitHub project that correspond with the
release tag, as the source tarball tagged for the release doesn't contain the
tests (prior to this change it would run 0 tests with make test, because
there weren't any).

Remove an unnecessary build dependency while here: pbr is no longer needed to
build the package.

Add py-pytest to TEST_DEPENDS, as running the tests as currently done via
distutils doesn't work due to how the tests are enumerated.

Conversely, remove the devel/py-tox dependency: it was never installed before
due to a typo in the variable name and it adds a lot of unnecessary
dependencies that aren't required when testing the port with make test. This
helps keep the footprint down, which helps others upgrade the devel/py-mock
port by itself.

Address portlint issues while here.

The changelog for the project can be found here on GitHub.

Test Plan

portlint/make test

1+ sudo make clean
2===> Cleaning for py36-mock-3.0.5
3===> Cleaning for py27-mock-3.0.5
4+ DEVELOPER=yes
5+ portlint -Ca
6looks fine.
7+ sudo make test
8===> License BSD2CLAUSE accepted by the user
9===> py36-mock-3.0.5 depends on file: /usr/local/sbin/pkg - found
10===> Fetching all distfiles required by py36-mock-3.0.5 for building
11===> Extracting for py36-mock-3.0.5
12=> SHA256 Checksum OK for testing-cabal-mock-3.0.5-e0180b98d0e07e895a3f699b7e9afcac4716fc03_GH0.tar.gz.
13===> Patching for py36-mock-3.0.5
14===> py36-mock-3.0.5 depends on package: py36-six>=1.9 - found
15===> py36-mock-3.0.5 depends on package: py36-setuptools>0 - found
16===> py36-mock-3.0.5 depends on file: /usr/local/bin/python3.6 - found
17===> Configuring for py36-mock-3.0.5
18running config
19===> Building for py36-mock-3.0.5
20running build
21running build_py
22creating build
23creating build/lib
24creating build/lib/mock
25copying mock/__init__.py -> build/lib/mock
26copying mock/mock.py -> build/lib/mock
27===> Staging for py36-mock-3.0.5
28===> py36-mock-3.0.5 depends on package: py36-setuptools>0 - found
29===> py36-mock-3.0.5 depends on file: /usr/local/bin/python3.6 - found
30===> Generating temporary packing list
31running install
32running build
33running build_py
34running install_lib
35creating /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6
36creating /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6/site-packages
37creating /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6/site-packages/mock
38copying build/lib/mock/__init__.py -> /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6/site-packages/mock
39copying build/lib/mock/mock.py -> /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6/site-packages/mock
40byte-compiling /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6/site-packages/mock/__init__.py to __init__.cpython-36.pyc
41byte-compiling /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6/site-packages/mock/mock.py to mock.cpython-36.pyc
42writing byte-compilation script '/tmp/tmp_5m51khe.py'
43/usr/local/bin/python3.6 /tmp/tmp_5m51khe.py
44removing /tmp/tmp_5m51khe.py
45running install_egg_info
46running egg_info
47creating mock.egg-info
48writing mock.egg-info/PKG-INFO
49writing dependency_links to mock.egg-info/dependency_links.txt
50writing requirements to mock.egg-info/requires.txt
51writing top-level names to mock.egg-info/top_level.txt
52writing manifest file 'mock.egg-info/SOURCES.txt'
53reading manifest file 'mock.egg-info/SOURCES.txt'
54reading manifest template 'MANIFEST.in'
55writing manifest file 'mock.egg-info/SOURCES.txt'
56Copying mock.egg-info to /usr/ports/devel/py-mock/work-py36/stage/usr/local/lib/python3.6/site-packages/mock-3.0.5-py3.6.egg-info
57running install_scripts
58writing list of installed files to '/usr/ports/devel/py-mock/work-py36/.PLIST.pymodtmp'
59===> Creating unique files: Move MAN files needing SUFFIX
60===> Creating unique files: Move files needing SUFFIX
61====> Compressing man pages (compress-man)
62===> Testing for py36-mock-3.0.5
63===> py36-mock-3.0.5 depends on package: py36-pytest>0 - found
64===> py36-mock-3.0.5 depends on package: py36-tox>0 - found
65===> py36-mock-3.0.5 depends on file: /usr/local/bin/python3.6 - found
66============================= test session starts ==============================
67platform freebsd11 -- Python 3.6.9, pytest-4.5.0, py-1.8.0, pluggy-0.12.0
68rootdir: /usr/ports/devel/py-mock/work-py36/mock-e0180b98d0e07e895a3f699b7e9afcac4716fc03, inifile: setup.cfg
69collected 406 items
70
71mock/tests/testcallable.py ........... [ 2%]
72mock/tests/testhelpers.py .............................................. [ 14%]
73..ss....s....s.........s........ [ 21%]
74mock/tests/testhelpers_py3.py . [ 22%]
75mock/tests/testmagicmethods.py ............................s.....s [ 30%]
76mock/tests/testmock.py ................................................. [ 42%]
77................................................................ss...... [ 60%]
78 [ 60%]
79mock/tests/testpatch.py ................................................ [ 72%]
80............................................................ [ 87%]
81mock/tests/testsealable.py .................. [ 91%]
82mock/tests/testsentinel.py ...... [ 93%]
83mock/tests/testsupport.py .. [ 93%]
84mock/tests/testwith.py .......................... [100%]
85
86==================== 397 passed, 9 skipped in 1.39 seconds =====================

Sanity check

$ python3
Python 3.6.9 (default, Jul 11 2019, 01:10:57) 
[GCC 4.2.1 Compatible FreeBSD Clang 6.0.0 (tags/RELEASE_600/final 326565)] on freebsd11
Type "help", "copyright", "credits" or "license" for more information.
>>> import mock
>>> mock.__version__
'3.0.5'
>>> mock.__file__
'/usr/local/lib/python3.6/site-packages/mock/__init__.py'

Running poudriere:

$ make index
...
$ egrep "py(27|36)-mock" INDEX-11 | awk -F\| '$8 ~ /py(27|36)-mock/ { print $2 }' | sort -u
/usr/ports/devel/py-buildbot-www
/usr/ports/devel/py-oslotest
/usr/ports/devel/py-pycadf
/usr/ports/devel/py-pyee
/usr/ports/graphics/qgis
/usr/ports/graphics/qgis-ltr
/usr/ports/security/py-keystone
/usr/ports/www/py-textile
$ PORTS=$(egrep "py(27|36)-mock" INDEX-11 | awk -F\| '$8 ~ /py(27|36)-mock/ { print $2 }' | sort -u | sed -e 's,^/usr/ports/,,')
 sudo poudriere bulk -j 11-amd64 $PORTS
[00:00:00] Creating the reference jail... done
[00:00:03] Mounting system devices for 11-amd64-default
[00:00:03] Mounting ports/packages/distfiles
[00:00:03] Stashing existing package repository
[00:00:04] Mounting packages from: /poudriere/data/packages/11-amd64-default
/etc/resolv.conf -> /poudriere/data/.m/11-amd64-default/ref/etc/resolv.conf
[00:00:04] Starting jail 11-amd64-default
[00:00:04] Logs: /poudriere/data/logs/bulk/11-amd64-default/2019-09-26_11h54m44s
[00:00:04] Loading MOVED for /poudriere/data/.m/11-amd64-default/ref/usr/ports
[00:00:05] Ports supports: FLAVORS SELECTED_OPTIONS
[00:00:05] Gathering ports metadata
...

The rest of the output can be found in the following paste after having rebuilt the ports and their dependencies several times:

1Script started on Fri Sep 27 18:16:52 2019
2Command: sudo poudriere bulk -j 11-amd64 devel/py-buildbot-www devel/py-oslotest devel/py-pycadf devel/py-pyee graphics/qgis graphics/qgis-ltr security/py-keystone www/py-textile
3[00:00:00] Creating the reference jail... done
4[00:00:01] Mounting system devices for 11-amd64-default
5[00:00:01] Mounting ports/packages/distfiles
6[00:00:01] Using packages from previously failed build: /poudriere/data/packages/11-amd64-default/.building
7[00:00:01] Mounting packages from: /poudriere/data/packages/11-amd64-default
8/etc/resolv.conf -> /poudriere/data/.m/11-amd64-default/ref/etc/resolv.conf
9[00:00:01] Starting jail 11-amd64-default
10[00:00:01] Logs: /poudriere/data/logs/bulk/11-amd64-default/2019-09-27_18h16m52s
11[00:00:01] Loading MOVED for /poudriere/data/.m/11-amd64-default/ref/usr/ports
12[00:00:02] Ports supports: FLAVORS SELECTED_OPTIONS
13[00:00:02] Gathering ports metadata
14[00:00:07] Calculating ports order and dependencies
15[00:00:08] Sanity checking the repository
16[00:00:08] Checking packages for incremental rebuild needs
17[00:00:11] Deleting stale symlinks... done
18[00:00:11] Deleting empty directories... done
19[00:00:11] Cleaning the build queue
20[00:00:11] Sanity checking build queue
21[00:00:11] Processing PRIORITY_BOOST
22[00:00:11] Balancing pool
23[00:00:11] Recording filesystem state for prepkg... done
24[00:00:11] Building 3 packages using 3 builders
25[00:00:11] Starting/Cloning builders
26[00:00:12] Hit CTRL+t at any time to see build progress and stats
27[00:00:12] [01] [00:00:00] Building databases/grass7 | grass7-7.6.1_6
28[00:00:12] [01] [00:00:00] Finished databases/grass7 | grass7-7.6.1_6: Ignored: is marked as broken: fails to package
29[00:00:12] [01] [00:00:00] Skipping graphics/qgis | qgis-3.8.2: Dependent port databases/grass7 | grass7-7.6.1_6 ignored
30[00:00:12] [01] [00:00:00] Skipping graphics/qgis-ltr | qgis-ltr-3.4.11_1: Dependent port databases/grass7 | grass7-7.6.1_6 ignored
31[00:00:12] Stopping 3 builders
32[00:00:30] No package built, no need to update the repository
33[00:00:30] Committing packages to repository: /poudriere/data/packages/11-amd64-default/.real_1569633442 via .latest symlink
34[00:00:30] Removing old packages
35[00:00:30] Skipped ports: graphics/qgis graphics/qgis-ltr
36[00:00:30] Ignored ports: databases/grass7
37[11-amd64-default] [2019-09-27_18h16m52s] [committing:] Queued: 3 Built: 0 Failed: 0 Skipped: 2 Ignored: 1  Tobuild: 0 Time: 00:00:29
38[00:00:30] Logs: /poudriere/data/logs/bulk/11-amd64-default/2019-09-27_18h16m52s
39[00:00:30] Cleaning up
40[00:00:30] Unmounting file systems
41
42Command exit status: 2
43Script done on Fri Sep 27 18:17:22 2019
.

I can't build qgis and qgis-ltr though, because one of their dependencies, database/grass, is marked BROKEN:

$ grep BROK databases/grass7/Makefile 
BROKEN=         fails to package
BROKEN_sparc64= Does not configure on sparc64

Diff Detail

Repository
rP FreeBSD ports repository
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 26561
Build 24949: arc lint + arc unit

Event Timeline

ngie created this revision.Sep 14 2019, 12:15 AM
ngie edited the summary of this revision. (Show Details)Sep 14 2019, 12:17 AM
bdrewery added inline comments.
devel/py-mock/Makefile
14–15

Are these versions still accurate?

$ DEVELOPER=yes portlint -Ca
WARN: Makefile: [28]: use a tab (not space) after a variable name
0 fatal errors and 1 warning found.
ngie edited the summary of this revision. (Show Details)Wed, Sep 18, 2:02 PM
ngie edited the test plan for this revision. (Show Details)
ngie edited the summary of this revision. (Show Details)Wed, Sep 18, 2:06 PM
ngie marked an inline comment as done.
ngie added inline comments.
devel/py-mock/Makefile
14–15

pbr's no longer needed, but six is still needed. The underlying code doesn't specify a specific six package version -- it just requires one to be installed.

ngie updated this revision to Diff 62268.Wed, Sep 18, 2:09 PM
ngie marked an inline comment as done.

Update the diff after feedback received and to fix the test target
by using github to pull down the sources and pytest to run the tests,
like tox.ini does.

ngie updated this revision to Diff 62269.Wed, Sep 18, 2:10 PM

Remove devel/py-tox dependency

It creates a lot of additional external dependencies and isn't strictly
required to run the tests.

ngie retitled this revision from Upgrade devel/py-mock from 2.0.0 to 3.0.5 to Upgrade devel/py-mock from 2.0.0 to 3.0.5 and fix "make test" target.Wed, Sep 18, 2:12 PM
ngie edited the summary of this revision. (Show Details)

Hi ngie,

Can you please complete build testing ("poudriere bulk -j 12amd64 -t devel/py-buildbot-www devel/py-oslotest ...") for the following ports which depend on this port?

As shown by https://www.freshports.org/devel/py-mock:

This port is required by:

for Build
        devel/py-buildbot-www
        devel/py-oslotest
        graphics/qgis
        graphics/qgis-ltr
        security/py-keystone
for Run
        audio/py-gmusicapi
        devel/py-oslotest
        devel/py-pytest-shutil
        devel/py-responses
        devel/py-sure
        devel/py-vsts-cd-manager
        graphics/qgis
        graphics/qgis-ltr
        science/py-MDAnalysis
        science/py-MDAnalysisTests
        security/py-certbot-apache
        security/py-certbot-dns-cloudflare
        security/py-certbot-dns-cloudxns
        security/py-certbot-dns-digitalocean
        security/py-certbot-dns-dnsimple
        security/py-certbot-dns-dnsmadeeasy
        security/py-certbot-dns-gehirn
        security/py-certbot-dns-google
        security/py-certbot-dns-linode
        security/py-certbot-dns-luadns
        security/py-certbot-dns-nsone
        security/py-certbot-dns-ovh
        security/py-certbot-dns-rfc2136
        security/py-certbot-dns-route53
        security/py-certbot-dns-sakuracloud
        security/py-certbot-nginx
        sysutils/py-azure-cli
        sysutils/py-azure-cli-batchai
        textproc/py-mkdocs
        www/py-kallithea
        www/py-rhodecode
ngie added a comment.Wed, Sep 18, 2:36 PM

Hi ngie,
Can you please complete build testing ("poudriere bulk -j 12amd64 -t devel/py-buildbot-www devel/py-oslotest ...") for the following ports which depend on this port?
As shown by https://www.freshports.org/devel/py-mock:

This port is required by:

for Build
        devel/py-buildbot-www
        devel/py-oslotest
        graphics/qgis
        graphics/qgis-ltr
        security/py-keystone
for Run
        audio/py-gmusicapi
        devel/py-oslotest
        devel/py-pytest-shutil
        devel/py-responses
        devel/py-sure
        devel/py-vsts-cd-manager
        graphics/qgis
        graphics/qgis-ltr
        science/py-MDAnalysis
        science/py-MDAnalysisTests
        security/py-certbot-apache
        security/py-certbot-dns-cloudflare
        security/py-certbot-dns-cloudxns
        security/py-certbot-dns-digitalocean
        security/py-certbot-dns-dnsimple
        security/py-certbot-dns-dnsmadeeasy
        security/py-certbot-dns-gehirn
        security/py-certbot-dns-google
        security/py-certbot-dns-linode
        security/py-certbot-dns-luadns
        security/py-certbot-dns-nsone
        security/py-certbot-dns-ovh
        security/py-certbot-dns-rfc2136
        security/py-certbot-dns-route53
        security/py-certbot-dns-sakuracloud
        security/py-certbot-nginx
        sysutils/py-azure-cli
        sysutils/py-azure-cli-batchai
        textproc/py-mkdocs
        www/py-kallithea
        www/py-rhodecode

Will do. Thank you for the pointer.

How did you determine this list BTW: using INDEX, etc?

devel/py-mock/Makefile
18

I think I was a bit hasty removing this line. Yes, six is needed for runtime use of the mock library.

In D21654#473469, @ngie wrote:

How did you determine this list BTW: using INDEX, etc?

Just using that FreshPorts link I included in my previous comment :)

ngie updated this revision to Diff 62293.Thu, Sep 19, 4:27 AM

Restore the previous RUN_DEPENDS logic

ngie edited the test plan for this revision. (Show Details)Thu, Sep 26, 11:50 PM
ngie edited the test plan for this revision. (Show Details)Sat, Sep 28, 1:22 AM