security/py-obfsproxy-tor: Fix runtime dependency error

[loader]

Summary

devel/py-twistedCore was removed since r450958:
https://svnweb.freebsd.org/ports?view=revision&revision=450958

# /usr/local/bin/obfsproxy
Traceback (most recent call last):
  File "/usr/local/bin/obfsproxy", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3195, in <module>
    @_call_aside
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3179, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3208, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 681, in _build_master
    ws.require(__requires__)
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 999, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 885, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'Twisted_Core' distribution was not found and is required by obfsproxy

Proposed commit log message:

security/py-obfsproxy-tor: Fix runtime dependency error

- Change runtime dependency entry point Twisted_Core back to Twisted 
- Update checksum for CHEESESHOP
- Add test target

Reviewed_by: cs, koobs
Approved by: cs (maintainer), koobs (ports)
Differential_Revision: D13452
MFH: 2018Q2

Test Plan

• portlint: OK (looks fine.)
• testport: OK (poudriere: 1200060, [armv7, aarch64], '', py27 tested)
• unittest: OK (passed: 80, skipped: 0, failed: 0)

[mat]

security/obfsproxy/Makefile

25 ↗

(On Diff #36475)

Could this work with Python 3 ?

[loader]

security/obfsproxy/Makefile

25 ↗

(On Diff #36475)

No, this port couldn't work with Python 3.

This reminds me that I made a mistake in D11562, the RUN_DEPENDS security/py-pyptlib
couldn't work with Python 3 either. Sorry ...

[loader]

Fixed in r463377.

[loader]

Fix runtime dependency error

[koobs]

LGTM, commit should be merged to quarterly if quarterly version of the port is also broken (MFH: 2018Q2)

Also: I would also check that argparse isn't actually required. Just removing it from setup.py stops a runtime error from occurring at startup, but doesn't prevent functionality loss if the code actually references it, or requires the third party package.

Just because argparse is included in the Python stdlib does not by itself mean that python packages can/do use the stdlib version. They are not the same thing, and removal of argparse from dependencies has caused broken ports in the past.

See Also, among others: http://svnweb.freebsd.org/changeset/ports/440641

[loader]

Hi koobs@, thanks for the note, I wasn't aware that the newer version of
devel/py-argarse might be required. The argparse module from stdlib works
for security/py-obfsproxy-tor, should I change it back to use devel/py-argparse?

% obfsproxy -v
0.2.13

% obfsproxy -h
usage: obfsproxy [-h] [-v] [--log-file LOG_FILE]
                 [--log-min-severity {error,warning,info,debug}] [--no-log]
                 [--no-safe-logging] [--data-dir DATA_DIR] [--proxy PROXY]
                 {managed,obfs2,dummy,obfs3,scramblesuit,b64} ...

py-obfsproxy: A pluggable transports proxy written in Python

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  --log-file LOG_FILE   set logfile
  --log-min-severity {error,warning,info,debug}
                        set minimum logging severity (default: None)
  --no-log              disable logging
  --no-safe-logging     disable safe (scrubbed address) logging
  --data-dir DATA_DIR   where persistent information should be stored.
  --proxy PROXY         Outgoing proxy (<proxy_type>://[<user_name>][:<passwor
                        d>][@]<ip>:<port>)

supported transports:
  {managed,obfs2,dummy,obfs3,scramblesuit,b64}
    managed             managed mode
    obfs2               obfs2 help
    dummy               dummy help
    obfs3               obfs3 help
    scramblesuit        scramblesuit help
    b64                 b64 help

[loader]

Another question is how can we make sure Python scripts load
the argparse module from PYTHON_SITELIBDIR instead of PYTHON_LIBDIR.

There are five ports which have devel/py-argparse in RUN_DEPENDS:

converters/py-zfec
devel/py-dateutils
devel/youcompleteme
security/py-gixy
textproc/pootle

I just checked converters/py-zfec, devel/py-dateutils and security/py-gixy,
it seems they just load the python2.7/argparse.py{,c} and doesn't check for
the python2.7/site-packages/argparse.py{,c}:

py27-zfec-1.5.2:

% truss /usr/local/bin/zfec 2>&1 | grep '\/argparse\.'
open("/usr/local/lib/python2.7/site-packages/zfec/argparse.so",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/lib/python2.7/site-packages/zfec/argparse.py",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/lib/python2.7/site-packages/zfec/argparse.pyc",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/bin/argparse.so",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/bin/argparse.py",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/bin/argparse.pyc",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/lib/python2.7/argparse.so",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/lib/python2.7/argparse.py",O_RDONLY,0666) = 5 (0x5)
open("/usr/local/lib/python2.7/argparse.pyc",O_RDONLY,0666) = 6 (0x6)

py27-dateutils-0.6.6:

% truss /usr/local/bin/datediff 2>&1 | grep '\/argparse\.'
open("/usr/local/lib/python2.7/site-packages/dateutils/argparse.so",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/lib/python2.7/site-packages/dateutils/argparse.py",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/lib/python2.7/site-packages/dateutils/argparse.pyc",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/bin/argparse.so",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/bin/argparse.py",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/bin/argparse.pyc",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/lib/python2.7/argparse.so",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/lib/python2.7/argparse.py",O_RDONLY,0666) = 4 (0x4)
open("/usr/local/lib/python2.7/argparse.pyc",O_RDONLY,0666) = 5 (0x5)

py27-gixy-0.1.3_1:

% truss /usr/local/bin/gixy 2>&1 | grep '\/argparse\.'
open("/usr/local/bin/argparse.so",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/bin/argparse.py",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/bin/argparse.pyc",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/lib/python2.7/argparse.so",O_RDONLY,0666) ERR#2 'No such file or directory'
open("/usr/local/lib/python2.7/argparse.py",O_RDONLY,0666) = 6 (0x6)
open("/usr/local/lib/python2.7/argparse.pyc",O_RDONLY,0666) = 7 (0x7)

[koobs]

In D13452#319545, @loader wrote:

Hi koobs@, thanks for the note, I wasn't aware that the newer version of
devel/py-argarse might be required. The argparse module from stdlib works
for security/py-obfsproxy-tor, should I change it back to use devel/py-argparse?

If it indeed works without it in this case (ie: the install_requires is actually spurious), then its fine to stay as is. However: some code may have try/fallback code to use the package if it exists, falling back to the stdlib if it doesn't, and/or vice versa.

If packages have spurious (unnecessary) argparse, or any other package requirements set (but don't use it), sure they are OK to be removed (and patched out), BUT its better removed upstream.

The bottom line is we shouldn't be doing anything in our ports which deviates from, or beyond what the upstream package says it wants/needs, because if we do, we *must* take on the *full* burden of doing so.

If the metadata (like requirements) isn't reflected in actual code (use), then upstream changes should be submitted, with us carrying a local ports patch (if absolutely necessary), until the changes are released. In most if not all cases, the benefit does not come close to outweighing that extra burden, vs just submitting changes upstream.

Finally, every time I've seen it happen in freebsd ports, it has been wrong and broken things in non-obvious ways.

[loader]

Thank you @koobs for your help and guidance. :)