Convert bsd.python.mk to USES=python

[mva]

Summary

This converts bsd.python.mk into a USES file.

Deprecations

• USE_PYTHON: USES=python, USES=python:2.7+, USES=python:3, etc.
• USE_PYTHON_BUILD: USES=python:build
• USE_PYTHON_RUN: USES=python:run
• PYTHON_MASTER_SITES: only five ports use it and it's exactly the same as MASTER_SITE_PYTHON
• PYDISTUTILS_INSTALLNOSINGLE: four ports use it

The following knobs are replaced:

• USE_PYDISTUTILS: see below
• PYDISTUTILS_AUTOPLIST: see below
• PYDISTUTILS_NOEGGINFO: see below
• USE_PYTHON_PREFIX: see below
• PYTHON_CONCURRENT_INSTALLL: see below
• PYTHON_PY3K_PLIST_HACK: see below

New Features

A new PYTHON_FEATURES variable was added which enables certain features for a
port:

• PYTHON_FEATURES=distutils replaces USE_PYDISTUTILS
• PYTHON_FEATURES=autoplist replaces PYDISTUTILS_AUTOPLIST
• PYTHON_FEATURES=py3kplist replaces PYTHON_PY3K_PLIST_HACK
• PYTHON_FEATURES=noegginfo replaces PYDISTUTILS_NOEGGINFO
• PYTHON_FEATURES=concurrent replaces PYTHON_CONCURRENT_INSTALL
• PYTHON_FEATURES=pythonprefix replaces USE_PYTHON_PREFIX

Other changes

Rework the settings for a specific python version to be set dynamically,
instead of hardcoded. Only a single place needs to be touched on updates of
supported Python versions now (_PYTHON_VERSIONS).

[bapt]

Thank you for working on this one! i wasn t expecting it before a while ;)
I do like the PYTHON_FEATURES/USES split looks pretty straight forward, I ll make a full review as soon as possible

[mva]

In D399#6, @bapt wrote:

Thank you for working on this one! i wasn t expecting it before a while ;)
I do like the PYTHON_FEATURES/USES split looks pretty straight forward, I ll make a full review as soon as possible

Feel free to do so. I doubt, that we can integrate it the way it is done right now, though, since some key internals were changed that make breaking ports even easier.

As I wrote in the Notes, I'd rather go with a 3-step approach and remove port-specific bits completely out of the .mk file prior to converting it to a USES. The problem is the mapping for different Python versions, which would require a back-reference or two files to maintain. I do have some ideas about it, on which I'd like to hear your opinion:

Let's assume the whole "which python versions are supported" thing should be made availabe to ports and the infrastructure through a single solution:

1. should we go with a port-centric local Makefile solution maintained under lang/python, which is used for looking up things in the .mk infrastructure (I would not, but like to hear @portmgr's opinion)?
2. should we go with a separate ("private") Makefile under Mk/, which can be referenced by both, python ports and infrastructure .mk files?

I'd rather go with the second solution, since a change can have an impact on the overall python port infrastructure, which seems to be more clear to me, if the file is maintained below Mk/. Let's call the file python-versions.mk.

The file itself (indifferent of the targeted way to solve things) would contain:

• version number magic (_REL, _ABIVER, _VER, _MAJOR_VER, etc.) along with the supported port directories, interpreter names, etc. Everything that's necessary to
• build lang/pythonXX (fetch, set PORTVERSION, etc.)
• allow packages for and using a specific python version to query the correct version number(s)

but no logic that influences a build of a port directly.

Mk/Uses/python.mk would include Mk/python-versions.mk. The same goes for the lang/pythonXX ports. which do not pull in Mk/Uses/python.mk via a USES=python line or similar approach.

[wg]

Nice work, I love it! Very straightforward and clean.

Mk/Uses/python.mk

290

Is there a reason for := and not just =?

I really don't know so just a dumb question =)

[mva]

Mk/Uses/python.mk

290

Yes. I want to avoid that a dependency has those already set, which may cause problems and pollutes the build.
Actually this should read

.undef _PYTHON_BUILD_DEP
.undef _PYTHON_RUN_DEP
_PYTHON_ARGS=		${python_ARGS:S/,/ /g}
.if ${python_ARGS:Mbuild}
_PYTHON_BUILD_DEP=	yes
...

[mva]

A reworked version, which is completely decoupled from lang/pythonXX and
vice-versa

[mva]

We need to preserve DEPENDS_ARGS to avoid using different python versions
within a build chain A -> B -> C

[mva]

• Fix PYTHON_REL/PYTHON_VER
• Make porters using DEVELOPER=y very unhappy :-)

[mva]

• Incorporate INSTALLS_EGGINFO bugfix from r362364

[mva]

• Repo cleanup to allow proper patch application

[mva]

• Fix PYTHON_PORTSDIR

[mva]

• Fix RUN_DEPENDS for _PYTHON_RUN_DEP

[mva]

• Fix PYDISTUTILS_INSTALLARGS

[mva]

• Fix _PYTHON_DEFAULT_VERSION detection, if a meta port version is selected

[antoine]

A preliminary list of ports/files that will have to be fixed:

audio/rhythmbox/Makefile:.include "${PORTSDIR}/Mk/bsd.python.mk"
biology/libsbml/Makefile:.include "../../Mk/bsd.python.mk"
databases/rrdtool/Makefile:.include "${PORTSDIR}/Mk/bsd.python.mk"
databases/metakit/Makefile:.include "${PORTSDIR}/Mk/bsd.python.mk"
databases/rrdtool12/Makefile:.include "${PORTSDIR}/Mk/bsd.python.mk"
editors/gedit/Makefile:.include "${PORTSDIR}/Mk/bsd.python.mk"
japanese/libtomoe-gtk/Makefile:.include "${PORTSDIR}/Mk/bsd.python.mk"
multimedia/libkate/Makefile:.include "${PORTSDIR}/Mk/bsd.python.mk"
security/libpreludedb/Makefile:.include "${PORTSDIR}/Mk/bsd.python.mk"
x11/gnome-applets/Makefile:.include "${PORTSDIR}/Mk/bsd.python.mk"

Mk/bsd.sites.mk: PYTHON:${PYTHON_MASTER_SITE_SUBDIR} \

databases/py-gdbm/Makefile:MASTER_SITES= ${PYTHON_MASTER_SITES}
databases/py-gdbm/Makefile:MASTER_SITE_SUBDIR= ${PYTHON_MASTER_SITE_SUBDIR}
databases/py-gdbm/Makefile:DISTNAME= ${PYTHON_DISTNAME}
databases/py-gdbm/Makefile:WRKSRC= ${PYTHON_WRKSRC}/Modules
databases/py-bsddb/Makefile:MASTER_SITE_SUBDIR= ${PYTHON_MASTER_SITE_SUBDIR}
databases/py-bsddb/Makefile:DISTNAME= ${PYTHON_DISTNAME}
databases/py-bsddb/Makefile:WRKSRC= ${PYTHON_WRKSRC}/Modules
databases/py-sqlite3/Makefile:MASTER_SITES= ${PYTHON_MASTER_SITES}
databases/py-sqlite3/Makefile:MASTER_SITE_SUBDIR= ${PYTHON_MASTER_SITE_SUBDIR}
databases/py-sqlite3/Makefile:DISTNAME= ${PYTHON_DISTNAME}
databases/py-sqlite3/Makefile:WRKSRC= ${PYTHON_WRKSRC}/Modules
x11-toolkits/py-tkinter/Makefile:MASTER_SITES= ${PYTHON_MASTER_SITES}
x11-toolkits/py-tkinter/Makefile:MASTER_SITE_SUBDIR= ${PYTHON_MASTER_SITE_SUBDIR}
x11-toolkits/py-tkinter/Makefile:DISTNAME= ${PYTHON_DISTNAME}
x11-toolkits/py-tkinter/Makefile:WRKSRC= ${PYTHON_WRKSRC}/Modules

[mva]

• Rework PYTHON_REL and fix PYTHON_PORTVERSION to provide the correct values
• Fix wrong .include "${PORTSDIR}/Mk/bsd.python.mk" lines in several ports
• Remove PYTHON_MASTER_SITES, PYTHON_MASTER_SITE_SUBDIR, PYTHON_DISTNAME and PYTHON_WRKSRC from Python-bound ports

• Attempt to fix broken ports from exp-run #167368 (remove PYTHON_VERSION from .MAKEFLAGS)

[mva]

• Do not set any python related variables on a global port scale via bsd.port.subdir.mk. This just will pollute ports builds.

[antoine]

Mk/Uses/python.mk

298

This doesn't seem to work (make -V_PYTHON_ARGS -C devel/googletests)
Maybe remove this line and replace
353 .if ${_PYTHON_ARGS} == "2"
with
.if ${_PYTHON_ARGS:M2}
356 .elif ${_PYTHON_ARGS} == "3"
with
.elif ${_PYTHON_ARGS:M3}

302

same here

[mva]

Mk/Uses/python.mk

298

This is weird. It works in every other Makefile - could it be a bug in make?

[mva]

Mk/Uses/python.mk

298

.if ${python_ARGS:Mbuild} should actually be .if ${_PYTHON_ARGS:Mbuild} ...

[mva]

• Fix build/run flag removal from _PYTHON_ARGS

[antoine]

Below is a list of ports that will break because they USE_PYTHON* after including bsd.port.pre.mk (doesn't work with USES, doing it after bsd.port.options.mk would be ok)

databases/tdb
databases/ldb
databases/ntdb
devel/talloc
devel/tevent
editors/gedit
japanese/libtomoe-gtk
x11/gnome-applets

[antoine]

So the exp-run results for Diff12 is available at: http://package22.nyi.freebsd.org/build.html?mastername=91amd64-default-PR192357&build=2014-08-04_18h37m52s

textproc/py2html does something crazy: textproc/py2html/work/py2html-0.7/Makefile:.include "${PORTSDIR}/Mk/bsd.python.mk"
devel/py3-gobject3 still failed to package because of the egginfo

The rest should be fixed by Diff13 (devel/googletest, devel/libopencm3, devel/mongo-cxx-driver) or by moving the USE_PYTHON before bsd.port.pre.mk (japanese/libtomoe-gtk, x11/gnome-applets, and the others that didn't break but may not get some things right)

[mva]

• Patch USE_PYTHON* out of <bsd.port.pre.mk> in several ports
• Teach textproc/py2html not to use bsd.python.mk
• Fix the plist of py3-gobject3

[antoine]

looks good!

[mva]

Committed in rP364450