graphics/py-mayavi: Application and library for 3D visualization and plotting in Python

[Vladimir.Chukharev_gmail.com]

Summary

A port of a new Mayavi application and libraries for scientific 3D visualization
with 4 depend ports.

All 5 ports are in Bugzilla: bug 204430 devel/py-envisage, bug 204431 devel/py-apptools,
bug 204432 devel/py-pyface, bug 204433 graphics/py-traitsui,
and bug 204435 graphics/py-mayavi.

The port devel/py-envisage in bug 204430 has been commited while this review was
under preparation. There is at least one minor change to it in this review.

Test Plan

Unswers to comments made in Bugzilla by Kubilay Kocak:

• Using PyPI (CHEESESHOP) for distribution files is preferred unless desirable files are missing from the sdist (LICENSE, test suite, etc)

I did not manage to fetch the distro using CHEESESHOP, perhaps due to recent switch
to GitHub made by Enthought. I'm bit reluctant to go away from currently working
method. Are there big advantages of CHEESESHOP?

• If a LICENSE file exists in WRKSRC, add LICENSE_FILE=${WRKSRC}/path/to/LICENSE

Done for graphics/py-mayavi, yet to be done for other ports.
A question. It turns out that within mayavi sources there are few more license file
for some internally used files (like colormap and icons). What to do in this case?
What to read on the subject?

• I'm not confident that the declared dependencies are all BUILD_DEPENDENCIES. See:

https://github.com/enthought/mayavi/blob/master/mayavi/__init__.py#L10

You are right, only one port is nesessary for build.

• GH_PROJECT defaults to PORTNAME, this entry is not needed

Removed from all 5 ports. I swear I tried to do this earlier and it did not work :)

• Instead of a pre-configure check for the/a python vtk module, there should be a (slave) port for either vtk with the option enabled (vtkX-python) OR, a port for py-pyvtk (https://pypi.python.org/pypi/PyVTK) if this is the module that is used. A pre-configure check assists port users who can rebuild vtk w/ PYTHON, but not package users.

Well, I do not want to maintain any of vtk ports, they are too complex to my taste...

• Group USE{S_*} lines with USES at the top, and USE_* sorted

Done for all 5 ports.

Before submitting the ports to Bugzilla, I asked for help with 4 issues. I added
a link to that request in bug 204435 graphics/py-mayavi. I recieved no comment
till now, probably, google groups is not a good place to ask this kind of questions.
So, I repeat the request here.
For Issue 2, I have found that some ports use pkg info for this purposes.
The solution still needs some adaptaion to my case, so it's not included in this
review.

Issue 1. Plist.

I cannot get plist working correctly.
If I use a static file, pkg-plist, then it fails either during my normal
installing or in poudrier. Seemingly, the installation distutils script
of the package detects difference of options of depend packages and installs or
skips some parts of html documentation.

So, it looks like I need an autoplist. I have in my port a line
USE_PYTHON= autoplist distutils
But with this the port also fails, and I think this indicates a bug in autoplist
mechanics.

How to fix it? Or how to debug this mechanics?

Issue 2. Depends on an option.

The port depends on math/vtk6 and needs it installed with option PYTHON
selected. I found in discussions that the only way is to inform the user about
the option and ask to re-install the dependence. I adapted the solution. But
later I learned that this method of checking a file for existence does not work
with changed WORKDIR and PREFIX (BUILDROOT in porttools).

Currently I have in Makefile:

pre-configure:
        if [ ! -e ${PREFIX}/lib/python2.7/site-packages/vtk/__init__.py ]; then \
                ${ECHO_MSG} "math/vtk6 was installed without PYTHON option; "; \
                ${ECHO_MSG} "please reinstall with PYTHON selected"; \
                ${FALSE}; \
        fi

What can be a better way to check for an option?

Issue 3. portlint and BUILD_DEPENDS= ${MY_DEPENDS}.

Porter's Handbook recommends to use a variable if the same list of depends
is to be assigned to two *_DEPENDS. But portlint complains:
WARN: Makefile: wrong dependency value for RUN_DEPENDS. RUN_DEPENDS requires 2 or 3 colon-separated tuples.

Is it a bug in portlint or am I missing something?

Issue 4. LD_LIBRARY_PATH.

If the port is installed and I start the script mayavi2, it fails:

$ mayavi2 
Warning: Unable to import the wx backend for pyface due to traceback: Traceback (most recent call last): 
          File "/usr/local/lib/python2.7/site-packages/pyface/toolkit.py", line 49, in _init_toolkit 
            be = import_toolkit(tk) 
          File "/usr/local/lib/python2.7/site-packages/pyface/toolkit.py", line 31, in import_toolkit 
            __import__(be + 'init') 
          File "/usr/local/lib/python2.7/site-packages/pyface/ui/wx/init.py", line 14, in <module> 
            import wx 
          File "/usr/local/lib/python2.7/site-packages/wx-3.0-gtk2/wx/__init__.py", line 45, in <module> 
            from wx._core import * 
          File "/usr/local/lib/python2.7/site-packages/wx-3.0-gtk2/wx/_core.py", line 4, in <module> 
            import _core_ 
        ImportError: /usr/lib/libstdc++.so.6: version GLIBCXX_3.4.11 required by /usr/local/lib/libwx_baseu-3.0.so.0 not found 

Traceback (most recent call last): 
  File "/usr/local/bin/mayavi2", line 9, in <module> 
    load_entry_point('mayavi==4.4.3', 'gui_scripts', 'mayavi2')() 
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 552, in load_entry_point 
    return get_distribution(dist).load_entry_point(group, name) 
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2672, in load_entry_point 
    return ep.load() 
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2345, in load 
    return self.resolve() 
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2351, in resolve 
    module = __import__(self.module_name, fromlist=['__name__'], level=0) 
  File "/usr/local/lib/python2.7/site-packages/mayavi/scripts/mayavi2.py", line 493, in <module> 
    raise ImportError(msg) 
ImportError: cannot import name scimath 
________________________________________________________________________________ 
Could not load envisage. You might have a missing dependency. 
Do you have the EnvisageCore and EnvisagePlugins installed? 

If you installed Mayavi with easy_install, try 'easy_install <pkg_name>'. 
'easy_install Mayavi[app]' will also work. 

If you performed a source checkout and installed via 'python setup.py develop', 
be sure to run the same command in the EnvisageCore and EnvisagePlugins folders. 

If these packages appear to be installed, check that your numpy and 
configobj are installed and working. If you need numpy, 'easy_install numpy' 
will install numpy. Similarly, 'easy_install configobj' will install 
configobj. 

I have read a number of PRs with similar problems e.g. bug 196703, bug 192928,
bug 192821. Unfortunately, I cannot identify a port which is the source of the
problem. Any help will be greatly appreciated.

A work-around from one of the PRs helps. Start as follows:
$ LD_LIBRARY_PATH=/usr/local/lib/gcc48 mayavi2
The program emits warnings about some deprecated things but works alright.

[Vladimir.Chukharev_gmail.com]

I do not understand why all the files are empty. Is it related to the fact, that I use svn while 'arc diff --create' uses git in its commands:
diff --git a/graphics/py-traitsui/pkg-descr b/graphics/py-traitsui/pkg-descr

How to fix this?

[Vladimir.Chukharev_gmail.com]

OK, I add the review via web interface.

[Vladimir.Chukharev_gmail.com]

graphics/py-mayavi/Makefile

32–37

For this block I have found a better way, suitable for build with non-default places for everything.

pre-configure:
        @opt=`${PKG_QUERY} "%Ok %Ov" ${DEP1} | ${AWK} "/^PYTHON on/{print $$2}"`
        @if [ -z "@${opt}" ] ; then \
                ${ECHO_MSG} "Port math/vtk6 is installed without PYTHON option."; \
                ${ECHO_MSG} "Please reinstall with PYTHON selected."; \
                ${FALSE}; \
        else \
                ${ECHO_MSG} "Good, math/vtk6 is installed with PYTHON option!"; \
        fi

[koobs]

For all ports:

• Update COMMENT to match (modified if necessary) setup.py:description=
• Add correct/appropriate LICENSE_FILE, LICENSE_COMB if necessary and license files exist
• pkg-descr WWW: URL should match setup.py:homepage= field

devel/py-apptools/Makefile

10

Rule of thumb:

• Use the setup.py:description= field for COMMENT, making changes to satisfy portlint/other conventions if or as necessary.

Currently, upstream has this as the description

description='application tools',

I would temporarily use:

COMMENT= Enthough application tools

And submit an upstream issue for them to modify the description= to something better/more descriptive (< 72 chars)

12

LICENSE.txt exists in WRKSRC, so add LICENSE_FILE

I can also see an image_LICENSE.txt file.

If multiple LICENSES are declared, use LICENSE_COMB (See Mk/bsd.licenses.mk for details)

20

Not necessary, this is the default.

23

Why NO_BUILD?

graphics/py-mayavi/Makefile

6

Put virtual categories (python) last

15–16

Depend on a 'version' not a 'port/package' version. If upstream doesn't declare any particular versions, depend on >0 (and get upstream to be clear about what it wants in the docs)

32–37

I'm not sure whether this is allowed/correct. We should ask someone from portmgr

[koobs]

CHEESESHOP

• Globally distributed CDN
• Confirms upstream is using good/correct packaging practices
• No reason not to

LICENSES

• LICENSE_COMB=multi, see Mk/bsd.licenses.mk

GH_PROJECT

• Still exists in current review changeset

VTK

• Not suggesting you maintain it, I am suggesting a 'root cause' fix, rather than 'workarounds'.

Checking for presense (or not) of an OPTION is a known 'issue' and usually an indication that a package/port could benefit from refactoring/splitting

PLIST Issues

I think this is because you're using NO_BUILD, but i dont know why. Jump on IRC so we can isolate why autoplist/distutils is breaking

PORTLINT (MY_DEPENDS)

This is a false positive because portlint is reading the variable as a string, not parsing its contents/values (which are 2-tuples). Open an issue under ports-mgmt/portlint, hopefully Marcus (maintainer) can find a good way to solve it

LB_LIBRARY_PATH

We've seen this happen a lot of numpy/scipy dependent ports. It needs to be isolated and resolved after root cause is identified.

[Vladimir.Chukharev_gmail.com]

Most of he issues are resolved in the new revision, with two exceptions.

1. LICENSE work is made only in commented out lines. I think, without ready documentation it's too complex job for these ports. Or, a least, I need more guidance in this area.
2. The dependence from a port with certain OPTION is reverted to what I had in the first revision. I'm still not ready to work on a new (sub)port of math/vtk6. On the other hand, the method I tried in my comment to graphycs/py-mayavi/Makefile (using pkg query) gives answer about packages installed on the system, but not in a jail of poudriere.

Please make your comments. Frankly speaking, I expect you will find find much more in addition to the above.

[koobs]

@Vladimir.Chukharev_gmail.com if you could tick the inline items that have been done, leaving the undone items unticked, that would help me more quickly figure out whats changed between diffs. Thanks!

[Vladimir.Chukharev_gmail.com]

graphics/py-mayavi/Makefile

16–17

I'm not sure what you mean.

32–37

Reverted to previous review, since in poudriere this method gives OPTIONs from system packages, not from the jail. Perhaps, pkg query can be instructed where to look via environment variables, I don't know.

[Vladimir.Chukharev_gmail.com]

devel/py-apptools/Makefile

13

This particular port uses one license, 'Done' is about this port only.
Others do use more than one, despite docs say (quite vaguely) it's BSD licensed. Internally, other ports include icons, color maps and other similar files with different licenses. I collected the information in comments.

I'm not sure in correctness of the declarations made. Awaiting for additions by A.J. "Fonz" van Werven to Porter's Handbook, see topic 'License info Q' in ports maillist: https://lists.freebsd.org/pipermail/freebsd-ports/2015-November/101198.html

[Vladimir.Chukharev_gmail.com]

Review 10830 contains graphics/py-mayavi/files/patch-* files, forgotten in the previous review. These patches fix the problem with plist.

[Vladimir.Chukharev_gmail.com]

Change USES=python:2 to USES=python:-2.7

See bug 205239 for discussions.

[Vladimir.Chukharev_gmail.com]

Update devel/py-pyface to 5.0.0.

[Vladimir.Chukharev_gmail.com]

graphics/py-mayavi/Makefile

33–38

About a week ago the default for PYTHON option of math/vtk6 has been changed to ON.

[Vladimir.Chukharev_gmail.com]

Update to the current state.
All license information is filled.
Corresponding PRs in bugzilla are updated, too.

[Vladimir.Chukharev_gmail.com]

Probably, better to say it explicitly:
I do not know about any issues with any of the five ports in this review. All five are ready to commit IMO.

[Vladimir.Chukharev_gmail.com]

Sorry, forgot to "svn add" the new ports after getting a new svn working copy (due to OS upgrade).

[Vladimir.Chukharev_gmail.com]

Everything has been committed.