Page MenuHomeFreeBSD

Simplify testing for some py-* ports
Needs RevisionPublic

Authored by jbeich on Oct 3 2015, 6:55 PM.

Details

Reviewers
sunpoet
AMDmi3
koobs
sbz
Group Reviewers
portmgr
Python
Summary

A lot of ports use setuptools, some have test command defined. In order to avoid redundancy and minor differences let's streamline testing. For details see individual commit messages.

Test Plan

Only very few ports tested manually on a single architecture/release combination e.g.,

# pkg delete -ayq
pkg: Cannot delete pkg itself without force flag
# make test -C devel/py-apscheduler
===>  License MIT accepted by the user
===>   py27-apscheduler-3.0.3 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by py27-apscheduler-3.0.3 for building
===>  Extracting for py27-apscheduler-3.0.3
=> SHA256 Checksum OK for APScheduler-3.0.3.tar.gz.
===>  Patching for py27-apscheduler-3.0.3
===>   py27-apscheduler-3.0.3 depends on package: py27-setuptools27>0 - not found
===>   Installing existing package /packages/All/py27-setuptools27-17.0.txz
[93i386-default] Installing py27-setuptools27-17.0...
[93i386-default] `-- Installing python27-2.7.10...
[93i386-default] |   `-- Installing libffi-3.2.1...
[93i386-default] |   | `-- Installing indexinfo-0.2.3...
[93i386-default] |   | `-- Extracting indexinfo-0.2.3: 100%
[93i386-default] |   `-- Extracting libffi-3.2.1: 100%
[93i386-default] |   `-- Installing gettext-runtime-0.19.5.1...
[93i386-default] |   | `-- Installing libiconv-1.14_8...
[93i386-default] |   | `-- Extracting libiconv-1.14_8: 100%
[93i386-default] |   `-- Extracting gettext-runtime-0.19.5.1: 100%
[93i386-default] `-- Extracting python27-2.7.10: 100%
[93i386-default] Extracting py27-setuptools27-17.0: 100%
Message from python27-2.7.10:
=====================================================================

Note that some standard Python modules are provided as separate ports
as they require additional dependencies. They are available as:

bsddb           databases/py-bsddb
gdbm            databases/py-gdbm
sqlite3         databases/py-sqlite3
tkinter         x11-toolkits/py-tkinter

=====================================================================
===>   py27-apscheduler-3.0.3 depends on package: py27-setuptools27>0 - found
===>   Returning to build of py27-apscheduler-3.0.3
===>   py27-apscheduler-3.0.3 depends on file: /usr/local/bin/python2.7 - found
===>  Configuring for py27-apscheduler-3.0.3
running config
===>  Building for py27-apscheduler-3.0.3
running build
running build_py
creating build
creating build/lib
creating build/lib/apscheduler
copying apscheduler/util.py -> build/lib/apscheduler
copying apscheduler/__init__.py -> build/lib/apscheduler
copying apscheduler/events.py -> build/lib/apscheduler
copying apscheduler/job.py -> build/lib/apscheduler
creating build/lib/apscheduler/schedulers
copying apscheduler/schedulers/tornado.py -> build/lib/apscheduler/schedulers
copying apscheduler/schedulers/__init__.py -> build/lib/apscheduler/schedulers
copying apscheduler/schedulers/background.py -> build/lib/apscheduler/schedulers
copying apscheduler/schedulers/twisted.py -> build/lib/apscheduler/schedulers
copying apscheduler/schedulers/blocking.py -> build/lib/apscheduler/schedulers
copying apscheduler/schedulers/base.py -> build/lib/apscheduler/schedulers
copying apscheduler/schedulers/qt.py -> build/lib/apscheduler/schedulers
copying apscheduler/schedulers/gevent.py -> build/lib/apscheduler/schedulers
copying apscheduler/schedulers/asyncio.py -> build/lib/apscheduler/schedulers
creating build/lib/apscheduler/jobstores
copying apscheduler/jobstores/memory.py -> build/lib/apscheduler/jobstores
copying apscheduler/jobstores/base.py -> build/lib/apscheduler/jobstores
copying apscheduler/jobstores/__init__.py -> build/lib/apscheduler/jobstores
copying apscheduler/jobstores/redis.py -> build/lib/apscheduler/jobstores
copying apscheduler/jobstores/mongodb.py -> build/lib/apscheduler/jobstores
copying apscheduler/jobstores/sqlalchemy.py -> build/lib/apscheduler/jobstores
creating build/lib/apscheduler/triggers
copying apscheduler/triggers/__init__.py -> build/lib/apscheduler/triggers
copying apscheduler/triggers/base.py -> build/lib/apscheduler/triggers
copying apscheduler/triggers/date.py -> build/lib/apscheduler/triggers
copying apscheduler/triggers/interval.py -> build/lib/apscheduler/triggers
creating build/lib/apscheduler/executors
copying apscheduler/executors/__init__.py -> build/lib/apscheduler/executors
copying apscheduler/executors/pool.py -> build/lib/apscheduler/executors
copying apscheduler/executors/twisted.py -> build/lib/apscheduler/executors
copying apscheduler/executors/asyncio.py -> build/lib/apscheduler/executors
copying apscheduler/executors/debug.py -> build/lib/apscheduler/executors
copying apscheduler/executors/gevent.py -> build/lib/apscheduler/executors
copying apscheduler/executors/base.py -> build/lib/apscheduler/executors
creating build/lib/apscheduler/triggers/cron
copying apscheduler/triggers/cron/fields.py -> build/lib/apscheduler/triggers/cron
copying apscheduler/triggers/cron/expressions.py -> build/lib/apscheduler/triggers/cron
copying apscheduler/triggers/cron/__init__.py -> build/lib/apscheduler/triggers/cron
===>  Staging for py27-apscheduler-3.0.3
===>   py27-apscheduler-3.0.3 depends on package: py27-pytz>0 - not found
===>   Installing existing package /packages/All/py27-pytz-2015.4,1.txz
[93i386-default] Installing py27-pytz-2015.4,1...
[93i386-default] Extracting py27-pytz-2015.4,1: 100%
===>   py27-apscheduler-3.0.3 depends on package: py27-pytz>0 - found
===>   Returning to build of py27-apscheduler-3.0.3
===>   py27-apscheduler-3.0.3 depends on package: py27-six>=1.4.0 - not found
===>   Installing existing package /packages/All/py27-six-1.9.0.txz
[93i386-default] Installing py27-six-1.9.0...
[93i386-default] Extracting py27-six-1.9.0: 100%
===>   py27-apscheduler-3.0.3 depends on package: py27-six>=1.4.0 - found
===>   Returning to build of py27-apscheduler-3.0.3
===>   py27-apscheduler-3.0.3 depends on package: py27-tzlocal>0 - not found
===>   Installing existing package /packages/All/py27-tzlocal-1.1.1.txz
[93i386-default] Installing py27-tzlocal-1.1.1...
[93i386-default] Extracting py27-tzlocal-1.1.1: 100%
===>   py27-apscheduler-3.0.3 depends on package: py27-tzlocal>0 - found
===>   Returning to build of py27-apscheduler-3.0.3
===>   py27-apscheduler-3.0.3 depends on package: py27-setuptools27>0 - found
===>   py27-apscheduler-3.0.3 depends on file: /usr/local/bin/python2.7 - found
===>   py27-apscheduler-3.0.3 depends on package: py27-futures>0 - not found
===>   Installing existing package /packages/All/py27-futures-3.0.3.txz
[93i386-default] Installing py27-futures-3.0.3...
[93i386-default] `-- Installing python2-2_3...
[93i386-default] `-- Extracting python2-2_3: 100%
[93i386-default] Extracting py27-futures-3.0.3: 100%
===>   py27-apscheduler-3.0.3 depends on package: py27-futures>0 - found
===>   Returning to build of py27-apscheduler-3.0.3
===>   Generating temporary packing list
running install
running build
running build_py
running install_lib
creating /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7
creating /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages
creating /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler
copying build/lib/apscheduler/__init__.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler
creating /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/jobstores
copying build/lib/apscheduler/jobstores/sqlalchemy.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/jobstores
copying build/lib/apscheduler/jobstores/redis.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/jobstores
copying build/lib/apscheduler/jobstores/__init__.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/jobstores
copying build/lib/apscheduler/jobstores/base.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/jobstores
copying build/lib/apscheduler/jobstores/memory.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/jobstores
copying build/lib/apscheduler/jobstores/mongodb.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/jobstores
copying build/lib/apscheduler/util.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler
copying build/lib/apscheduler/events.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler
copying build/lib/apscheduler/job.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler
creating /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers
copying build/lib/apscheduler/triggers/base.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers
copying build/lib/apscheduler/triggers/interval.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers
copying build/lib/apscheduler/triggers/date.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers
creating /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers/cron
copying build/lib/apscheduler/triggers/cron/fields.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers/cron
copying build/lib/apscheduler/triggers/cron/__init__.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers/cron
copying build/lib/apscheduler/triggers/cron/expressions.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers/cron
copying build/lib/apscheduler/triggers/__init__.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers
creating /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers
copying build/lib/apscheduler/schedulers/tornado.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers
copying build/lib/apscheduler/schedulers/blocking.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers
copying build/lib/apscheduler/schedulers/qt.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers
copying build/lib/apscheduler/schedulers/gevent.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers
copying build/lib/apscheduler/schedulers/base.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers
copying build/lib/apscheduler/schedulers/asyncio.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers
copying build/lib/apscheduler/schedulers/background.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers
copying build/lib/apscheduler/schedulers/twisted.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers
copying build/lib/apscheduler/schedulers/__init__.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers
creating /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors
copying build/lib/apscheduler/executors/base.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors
copying build/lib/apscheduler/executors/twisted.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors
copying build/lib/apscheduler/executors/asyncio.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors
copying build/lib/apscheduler/executors/debug.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors
copying build/lib/apscheduler/executors/gevent.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors
copying build/lib/apscheduler/executors/pool.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors
copying build/lib/apscheduler/executors/__init__.py -> /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/__init__.py to __init__.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/jobstores/sqlalchemy.py to sqlalchemy.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/jobstores/redis.py to redis.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/jobstores/__init__.py to __init__.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/jobstores/base.py to base.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/jobstores/memory.py to memory.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/jobstores/mongodb.py to mongodb.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/util.py to util.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/events.py to events.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/job.py to job.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers/base.py to base.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers/interval.py to interval.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers/date.py to date.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers/cron/fields.py to fields.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers/cron/__init__.py to __init__.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers/cron/expressions.py to expressions.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/triggers/__init__.py to __init__.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers/tornado.py to tornado.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers/blocking.py to blocking.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers/qt.py to qt.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers/gevent.py to gevent.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers/base.py to base.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers/asyncio.py to asyncio.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers/background.py to background.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers/twisted.py to twisted.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/schedulers/__init__.py to __init__.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors/base.py to base.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors/twisted.py to twisted.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors/asyncio.py to asyncio.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors/debug.py to debug.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors/gevent.py to gevent.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors/pool.py to pool.pyc
byte-compiling /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/apscheduler/executors/__init__.py to __init__.pyc
writing byte-compilation script '/tmp/tmp07JgRR.py'
/usr/local/bin/python2.7 -O /tmp/tmp07JgRR.py
removing /tmp/tmp07JgRR.py
running install_egg_info
running egg_info
writing requirements to APScheduler.egg-info/requires.txt
writing APScheduler.egg-info/PKG-INFO
writing top-level names to APScheduler.egg-info/top_level.txt
writing dependency_links to APScheduler.egg-info/dependency_links.txt
writing entry points to APScheduler.egg-info/entry_points.txt
reading manifest file 'APScheduler.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'APScheduler.egg-info/SOURCES.txt'
Copying APScheduler.egg-info to /wrkdirs/usr/ports/devel/py-apscheduler/work/stage/usr/local/lib/python2.7/site-packages/APScheduler-3.0.3-py2.7.egg-info
running install_scripts
writing list of installed files to '/wrkdirs/usr/ports/devel/py-apscheduler/work/.PLIST.pymodtmp'
====> Compressing man pages (compress-man)
===>  Testing for py27-apscheduler-3.0.3
===>   py27-apscheduler-3.0.3 depends on package: py27-pytest>=2.5.1 - not found
===>   Installing existing package /packages/All/py27-pytest-2.7.1.txz
[93i386-default] Installing py27-pytest-2.7.1...
[93i386-default] `-- Installing py27-py-1.4.26...
[93i386-default] `-- Extracting py27-py-1.4.26: 100%
[93i386-default] Extracting py27-pytest-2.7.1: 100%
===>   py27-apscheduler-3.0.3 depends on package: py27-pytest>=2.5.1 - found
===>   Returning to build of py27-apscheduler-3.0.3
===>   py27-apscheduler-3.0.3 depends on package: py27-mock>0 - not found
===>   Installing existing package /packages/All/py27-mock-1.0.1_1.txz
[93i386-default] Installing py27-mock-1.0.1_1...
[93i386-default] Extracting py27-mock-1.0.1_1: 100%
===>   py27-apscheduler-3.0.3 depends on package: py27-mock>0 - found
===>   Returning to build of py27-apscheduler-3.0.3
running test
running egg_info
writing requirements to APScheduler.egg-info/requires.txt
writing APScheduler.egg-info/PKG-INFO
writing top-level names to APScheduler.egg-info/top_level.txt
writing dependency_links to APScheduler.egg-info/dependency_links.txt
writing entry points to APScheduler.egg-info/entry_points.txt
reading manifest file 'APScheduler.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'APScheduler.egg-info/SOURCES.txt'
running build_ext
=============================== test session starts ================================
platform freebsd9 -- Python 2.7.10 -- py-1.4.26 -- pytest-2.7.1
rootdir: /wrkdirs/usr/ports/devel/py-apscheduler/work/APScheduler-3.0.3, inifile: setup.cfg
collected 401 items

tests/test_executors.py ..........
tests/test_expressions.py ..........................
tests/test_job.py .........................................
tests/test_jobstores.py .sss.sss.sss.sss.sss.sss.sss.sss..ssssss......ssssssssssssssssss.ssssss.sss.sss.sss.sss.sssss
tests/test_schedulers.py .....................................................................................................sssssssssssssss
tests/test_triggers.py ........................................
tests/test_util.py ...................................................ss....................ss
============================= short test summary info ==============================
SKIP [1] /usr/local/lib/python2.7/site-packages/_pytest/skipping.py:140: Requires Python >= 3
SKIP [1] /wrkdirs/usr/ports/devel/py-apscheduler/work/APScheduler-3.0.3/tests/test_jobstores.py:286: could not import 'apscheduler.jobstores.mongodb'
SKIP [3] /wrkdirs/usr/ports/devel/py-apscheduler/work/APScheduler-3.0.3/tests/test_schedulers.py:990: could not import 'tornado.ioloop'
SKIP [1] /wrkdirs/usr/ports/devel/py-apscheduler/work/APScheduler-3.0.3/tests/test_jobstores.py:269: could not import 'apscheduler.jobstores.sqlalchemy'
SKIP [3] /wrkdirs/usr/ports/devel/py-apscheduler/work/APScheduler-3.0.3/tests/test_schedulers.py:1015: could not import 'twisted.internet.selectreactor'
SKIP [1] /wrkdirs/usr/ports/devel/py-apscheduler/work/APScheduler-3.0.3/tests/test_jobstores.py:296: could not import 'apscheduler.jobstores.mongodb'
SKIP [3] /wrkdirs/usr/ports/devel/py-apscheduler/work/APScheduler-3.0.3/tests/test_schedulers.py:971: could not import 'apscheduler.schedulers.gevent'
SKIP [3] /wrkdirs/usr/ports/devel/py-apscheduler/work/APScheduler-3.0.3/tests/test_schedulers.py:1041: could not import 'PySide.QtCore'
SKIP [22] /wrkdirs/usr/ports/devel/py-apscheduler/work/APScheduler-3.0.3/tests/test_jobstores.py:58: could not import 'apscheduler.jobstores.redis'
SKIP [3] /wrkdirs/usr/ports/devel/py-apscheduler/work/APScheduler-3.0.3/tests/test_schedulers.py:946: could not import 'apscheduler.schedulers.asyncio'
SKIP [1] /wrkdirs/usr/ports/devel/py-apscheduler/work/APScheduler-3.0.3/tests/test_jobstores.py:279: could not import 'apscheduler.jobstores.sqlalchemy'
SKIP [22] /wrkdirs/usr/ports/devel/py-apscheduler/work/APScheduler-3.0.3/tests/test_jobstores.py:46: could not import 'apscheduler.jobstores.mongodb'
SKIP [2] /usr/local/lib/python2.7/site-packages/_pytest/skipping.py:140: Requires __qualname__ (Python 3.3+)
SKIP [22] /wrkdirs/usr/ports/devel/py-apscheduler/work/APScheduler-3.0.3/tests/test_jobstores.py:34: could not import 'apscheduler.jobstores.sqlalchemy'
SKIP [1] /wrkdirs/usr/ports/devel/py-apscheduler/work/APScheduler-3.0.3/tests/test_jobstores.py:302: could not import 'apscheduler.jobstores.mongodb'
SKIP [1] /usr/local/lib/python2.7/site-packages/_pytest/skipping.py:140: Requires Python >= 3.4

====================== 311 passed, 90 skipped in 3.71 seconds ======================

Diff Detail

Repository
rP FreeBSD ports repository
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 666
Build 666: arc lint + arc unit

Event Timeline

jbeich retitled this revision from to Simplify testing for some py-* ports.
jbeich updated this object.
jbeich edited the test plan for this revision. (Show Details)
jbeich added reviewers: Python, portmgr, AMDmi3.
AMDmi3 requested changes to this revision.Oct 4 2015, 12:52 AM
AMDmi3 edited edge metadata.

(only reviewed a few bits, will have a time to look through remaining part later)

Mk/bsd.port.mk
1510 ↗(On Diff #9099)

Please don't. This way you have less control and it's also inconsistent with _ARGS.

For instance, if for some reason you want TEST_ENV empty, with your change you can't make it so unless you also make MAKE_ENV empty, and you may not want that as you need MAKE_ENV for build phase. With ?= you can do whatever you need - you can set TEST_ENV it to empty, to ${MAKE_ENV}, to foo ${MAKE_ENV} or to ${MAKE_ENV} foo for any kind of override logic.

databases/py-Pyrseas/Makefile
25–28

I think you need NO_TEST here, else the test will always fail. Not yet sure, but either

  • We may ignore such kind of tests completely, e.g. state that tests which cannot run automatically are out of portstree scope.
  • We may add MANUAL_TEST similar to MANUAL_PACKAGE_BUILD, which sets NO_TEST if PACKAGE_BUILDING is set or something like that.
This revision now requires changes to proceed.Oct 4 2015, 12:52 AM
jbeich edited edge metadata.
  • bsd.port.mk patch dropped as it needs more work but ENOTIME. TEST_ENV= ${MAKE_ENV} ... should suffice here to run setup.py commands with consistent environment. Still a shot in the dark without automation help.
  • Adapted databases/py-Pyrseas to not run for PACKAGE_BUILDING. However, more ports here have broken tests. Identifying them is outside of scope until poudriere catches up. Their tests can be disabled individually with errors documented in commit messages. regression-test implies it worked fine in the past as tinderbox runs the target by default.

I'm -1 on this so soon after the test framework landed. I think a review of the current test framework for applicability to Python ports is warranted before we go changing a large number of Python ports.

See:

Mk/Uses/python.mk
117

I would prefer to see this as a generic variable, overriding the default TEST_TARGET from bsd.port.mk.

The canonical 'test' target is a setuptools target, so naming it 'distutils' is technically incorrect.

Also, my gut feeling tells me we want to explicitly opt-in/enable tests, indepently to whether other TEST_* variables are set. Many upstream test suites break/fail from one version to the next.

koobs requested changes to this revision.Oct 4 2015, 3:01 AM
koobs added a reviewer: koobs.

Let's catch up on #freebsd-python (freenode IRC) so we can discuss and elicit feedback from others

This revision now requires changes to proceed.Oct 4 2015, 3:01 AM
jbeich removed reviewers: koobs, portmgr.
jbeich edited subscribers, added: portmgr; removed: koobs.
jbeich requested a review of this revision.Oct 4 2015, 3:10 AM
jbeich added a reviewer: koobs.

Sorry, that was an accident. I've intentionally moved portmgr from reviewers to subscribers as the patch against bsd.port.mk is gone in new version.

Mk/Uses/python.mk
117

I would prefer to see this as a generic variable, overriding the default TEST_TARGET from bsd.port.mk.

That will be inconsistent with other PYDISTUTILS_*_TARGETs. TEST_TARGET is intended primarily for make, so there's nothing wrong if different kind of target variables are used for pydistutils. Otoh, we may consider switching these to normal *_TARGETs.

Also, my gut feeling tells me we want to explicitly opt-in/enable tests, indepently to whether other TEST_* variables are set. Many upstream test suites break/fail from one version to the next.

I'd say opt-out. We want tests to run by default, and disable them only if they are useless and cannot be fixed. Every failure should be fixed this way or another, as it's either problem of software or test itself.

jbeich added inline comments.
Mk/Uses/python.mk
117

Many upstream test suites break/fail from one version to the next.

How existing regression-test in the py-* ports survived tinderbox run then? In pointyhat.freebsd.org days such ports would be quickly flagged as BROKEN or the target would be disabled somehow.

Since poudriere became the tool of choice testing automation gradually regressed. Until it catches up tests are unreliable. They can fail on some cpython versions, different freebsd releases or architectures. Only maintainer knows where or when tests are supposed to work.

Otherwise, NO_TEST works fine as a tool to disable broken tests without also disabling package generation.

sunpoet added a subscriber: sunpoet.
sunpoet added inline comments.
Mk/Uses/python.mk
610

How about ".if !target(do-test) && !defined(NO_PYTHON_TEST)"? I would suggest to add check for NO_PYTHON_TEST. It allows ports with USES=python to run default test (defined in bsd.port.mk) instead of python one. I did a similar one in perl5.mk.

612

PYDISTUTILS_TESTARGS is undefined. Why not use TEST_ARGS?

jbeich edited the test plan for this revision. (Show Details)
jbeich edited edge metadata.
  • Added NO_PYDISTUTILS_TEST like rP398474. To avoid consistency mess a la Uses/perl5.mk the change is expanded to include other targets as well. The naming continues the tradition of confusing distutils and setuptools.
  • Added description for PYDISTUTILS_TESTARGS
  • PYDISTUTILS_TESTARGS default remains as is. setuptools cannot handle make/gmake arguments:
$ make test PYDISTUTILS_TESTARGS='${TEST_ARGS}' -C devel/py-apscheduler
===>  Testing for py27-apscheduler-3.0.3
===>   py27-apscheduler-3.0.3 depends on package: py27-pytest>=2.5.1 - found
===>   py27-apscheduler-3.0.3 depends on package: py27-mock>0 - found
invalid command name 'DESTDIR=/usr/ports/devel/py-apscheduler/work/stage'
*** Error code 1
jbeich edited edge metadata.

After checking a few hybrid ports and thinking more:

  • Drop NO_PYDISTUTILS_BUILD and NO_PYDISTUTILS_INSTALL as they're unlikely to work. NO_PYDISTUTILS_CONFIGURE is already handled but in different way.
  • Use !defined(TEST_TARGET) similar to !defined(HAS_CONFIGURE) as make-style tests have to be explicitly enabled similar to configure.
jbeich edited edge metadata.
  • Sync commit message as NO_PYDISTUTILS_TEST is gone. No need to mention TEST_TARGET explicitly as it's already documented in bsd.port.mk
AMDmi3 requested changes to this revision.Oct 6 2015, 4:53 PM
AMDmi3 edited edge metadata.
AMDmi3 added inline comments.
Mk/Uses/python.mk
610

This is not needed as we have NO_TEST.
NO_PERL_TEST is wrong as well, see mail.

This revision now requires changes to proceed.Oct 6 2015, 4:53 PM
Mk/Uses/python.mk
610

No, they are different. IIUC NO_TEST is used to bypass test framework entirely. A port requires perl would add USES=perl5 but it does not mean that it wants to run perl tests. Since we do not support multiple tests currently, we need to check NO_PERL5_TEST in order to skip do-test: from perl5.mk but run the do-test: from bsd.port.mk or other Uses/*.mk (e.g. USES=python).

Please try ftp/curl with/without NO_PERL5_TEST.

BTW, I think Jan's idea is much better. Checking TEST_TARGET should have same effect but avoids new knob like NO_foo_TEST.

Mk/Uses/python.mk
610

Please try ftp/curl with/without NO_PERL5_TEST.

https://reviews.freebsd.org/D3830

BTW, I think Jan's idea is much better. Checking TEST_TARGET should have same effect but avoids new knob like

No. It won't work e.g. with perl5 as perl5 tests require TEST_TARGET, and here it's basically the same thing, but python duplicates *_TARGET by PYDISTUTILS_*_TARGET. Which feels wrong by itself.

AMDmi3 edited edge metadata.

As it currently is, LGTM. && !defined(TEST_TARGET) check looks a bit pointless though. It is only useful for cases where pydistutils are used, however there's also Makefile specifically for running tests. Are there any real cases like that?

Mk/Uses/python.mk
612

As per original diff comment, I would much prefer to use TEST_* variables in python.mk and python ports. These test framework bits ought to have nothing to do (and dont) with distutils/setuptools. The setuptools test command is just one possible invocation command.

This allows us to use the plethora of python testing frameworks, with granularity in setting/overriding defaults from bsd.port.mk and python.mk if necessary. For example:

  • Some python packages use (and sometimes force use of) nosetests/tox, with and without setuptools integration (test command)
    • @cd ${WRKSRC} && nosetests [optional args]
    • @cd ${WRKSRC} && ${PYTHON_CMD} ${PYDISTUTILS_SETUP} nosetests (a setuptools command, if the package integrates nose well with setup.py)
    • @cd ${WRKSRC} && tox [optional args]
    • @cd ${WRKSRC} && py.test [optional args]

Thus, I'd like to see, in Python ports, and all ports more generally that support test suites, use something like:

TEST_ENV TEST_CMD TEST_TARGET TEST_ARGS

If the defaults (make test, python setup.py test, perl whatever) don't happen to just work for any package, it's then trivial to override the appropriate variable(s) for that port, while staying entirely faithful to a generic test framework.

Mk/Uses/python.mk
612

As I've noted, I don't really like TEST_CMD.

  • If you need custom command to run tests, you should do custom do-test: in the port
  • If there's some (another) standard way of running python tests, there should be USES (or USE_PYDISTUTILS, or USE_PYTHON) bit which defines do-test for that.

Trying to sticking everything into "TEST_ENV TEST_CMD TEST_TARGET TEST_ARGS" or similar pattern will lead to ugliness like

TEST_CMD=${python_CMD}
TEST_TARGET=./run_tests
TEST_ARGS=

which is both longer and less readable than a custom do-test.