Add Q/A tests and shebang fixes for python[23] shebangs

[AMDmi3]

Summary

This is kind of followup to D9332: we've added checks and fixes for
python, but not for python2/3 interpreters. Currently there's an ongoing
effort to switch instances of e.g. USES=python:2 to USES=python:2.7,
which makes ports no longer depend on python2 metaport and thus allows
a situation where python2/python3 executable is not available. This
can be tested with a simple test port:

PORTNAME=       testport
PORTVERSION=    0.0.0
CATEGORIES=     sysutils
DISTFILES=      #

MAINTAINER=     [e-mail]
COMMENT=        #

USES=           python
SHEBANG_FILES=  *.py
NO_BUILD=       yes

PLIST_FILES=    bin/python.py \
                bin/env_python.py \
                bin/python2.py \
                bin/env_python2.py \
                bin/python3.py \
                bin/env_python3.py \
                bin/python_cmd.py

OPTIONS_DEFINE= SHEBANGFIX
SHEBANGFIX_USES=shebangfix

do-extract:
        @${MKDIR} ${WRKSRC}
        echo '#!/usr/local/bin/python' > ${WRKSRC}/python.py
        echo '#!/usr/bin/env python' > ${WRKSRC}/env_python.py
        echo '#!/usr/local/bin/python2' > ${WRKSRC}/python2.py
        echo '#!/usr/bin/env python2' > ${WRKSRC}/env_python2.py
        echo '#!/usr/local/bin/python3' > ${WRKSRC}/python3.py
        echo '#!/usr/bin/env python3' > ${WRKSRC}/env_python3.py
        echo '#!${python_CMD}' > ${WRKSRC}/python_cmd.py

do-install:
        ${INSTALL_SCRIPT} ${WRKSRC}/*.py ${STAGEDIR}${PREFIX}/bin
.for f in ${PLIST_FILES}
        @if ${STAGEDIR}${PREFIX}/${f}; then \
                echo ${f}: OK; \
        else \
                echo ${f}: Failed; \
        fi 2>/dev/null
.endfor

It installs python scripts with different shebangs so one could
examing stage Q/A output and the actual ability to run scripts.

So, currently the output is the following:

SHEBANG=off
bin/python.py: Failed
bin/env_python.py: Failed
bin/python2.py: Failed
bin/env_python2.py: Failed
bin/python3.py: Failed
bin/env_python3.py: Failed
bin/python_cmd.py: OK
Error: '/usr/local/bin/python' is an invalid shebang you need USES=shebangfix for 'bin/python.py'
Error: '/usr/bin/env python' is an invalid shebang you need USES=shebangfix for 'bin/env_python.py'

SHEBANG=on
bin/python.py: OK
bin/env_python.py: OK
bin/python2.py: Failed
bin/env_python2.py: Failed
bin/python3.py: Failed
bin/env_python3.py: Failed
bin/python_cmd.py: OK

As you can see, only scripts with /usr/local/bin/python and
/usr/bin/env python interpreters were detected as incorrect
shebangs and fixed by shebangfix. python2/pytnon3 are neither
detected neither runnable.

So I propose to add corresponding checks and fixes.

Test Plan

After the patch applied,

SHEBANG=off
Error: '/usr/bin/env python3' is an invalid shebang you need USES=shebangfix for 'bin/env_python3.py'
Error: '/usr/local/bin/python' is an invalid shebang you need USES=shebangfix for 'bin/python.py'
Error: '/usr/local/bin/python2' is an invalid shebang you need USES=shebangfix for 'bin/python2.py'
Error: '/usr/local/bin/python3' is an invalid shebang you need USES=shebangfix for 'bin/python3.py'
Error: '/usr/bin/env python' is an invalid shebang you need USES=shebangfix for 'bin/env_python.py'
Error: '/usr/bin/env python2' is an invalid shebang you need USES=shebangfix for 'bin/env_python2.py'

SHEBANG=on
bin/python.py: OK
bin/env_python.py: OK
bin/python2.py: OK
bin/env_python2.py: OK
bin/python3.py: OK
bin/env_python3.py: OK
bin/python_cmd.py: OK

That is, all incorrect shebangs are either detected by stage-qa or fixed by shebangfix.

[mat]

I had a thing similar to that, but please go ahead.

[AMDmi3]

Thanks! Out of curiocity, was the possibility of automatically adding dependency on python2/3 metaports considered?

[mat]

In D13571#283855, @AMDmi3 wrote:

Thanks! Out of curiosity, was the possibility of automatically adding dependency on python2/3 metaports considered?

How would one guess that one needs the metaport, and what good would it achieve ?

[AMDmi3]

How would one guess that one needs the metaport, and what good would it achieve ?

I guess it makes sense to always add a metaport for corresponding major python version (2 or 3), and it will save us from caring of shebangs, at least partially.
There's an issue of multiple minor versions (3.4, 3.5, 3.6) this won't fix though...

[mat]

Those two metaports never really made sense, they were transitional to help while we cleaned the tree, they should probably be removed, or really made as meta ports, empty shells with only USES=python:${PYTHONX_DEFAULT}.

[AMDmi3]

In D13571#283859, @mat wrote:

Those two metaports never really made sense, they were transitional to help while we cleaned the tree, they should probably be removed, or really made as meta ports, empty shells with only USES=python:${PYTHONX_DEFAULT}.

There is sense in having python/python2/python3 links as that's what most people interact with and use in both command line and shebangs.

[adamw]

In D13571#283860, @AMDmi3 wrote:

There is sense in having python/python2/python3 links as that's what most people interact with and use in both command line and shebangs.

We have that ability now in USES=uniquefiles. Ruby does it well, and python could leverage it too. At build-time, python knows whether it's the default python and/or default python2/3.