Page MenuHomeFreeBSD

Trying to rescue the port math/sage
Needs ReviewPublic

Authored by thierry on Mar 26 2020, 4:34 PM.

Details

Summary

math/sage, also known as SageMath is a very important application for people working in mathematics or sciences.
Unfortunately, this must-have is also a nightmare for packagers, and ATM the FreeBSD port is lagging and broken.

This patch is a work-in-progress to try to fix it.

/!\ This is just a WIP! Not [yet] committable at this point, and any help is welcome!

This is a kind of meta-application, which builds as many math-related packages as possible to give them a common interface, and all these sub-packages are bundled. Upstream knows about this problem, and a ticket has been opened to "use as many system packages as possible": see #27330 at https://trac.sagemath.org/ticket/27330.

But this is a long task, and a lot of packages are still bundled and built by Sage, and some of them do not respect DESTDIR (i.e. STAGEDIR). The hereunder patch allow to build everything, but several objects are referring to the staging directory.
E.g.:
Error: 'lib/liblinbox.so.0.0.0' is referring to /usr/ports/math/sage/work/stage

Every sub-package can be patched, e.g. see files/psutilarchbsd__freebsd_socks.c.patch, but this would be a time consuming task. Maybe someone™ could have an idea to fix this globally?

Another related problem is the installation directory. The current port installed everything under ${PREFIX}/${PORTNAME}-${PORTVERSION}.
If we keep this, every installed object should know about this directory to find its related objects.
If we try to install it directly under ${PREFIX}, it would create a lot of conflicts: a possible solution would be to remove these potential conflicts from the staging directory and use the external dependencies, hoping they have been built with similar options and compatible.

Test Plan

First, if I ever recreated this port, I would rename it to sagemath. Also, if someone else wants to formally take over as maintainer, I am completely fine with that.

Next, when I created the original port I tried to follow their original vision of having the port build its own dependencies. But it was very difficult to figure out how to apply patches to tarballs. Hence the patches like patch-build_pkgs_*_spkg-install. Also, I had to find the patches applied to the freebsd port, and transfer them to the sage port. This was very time consuming. Transferring the patches from python was by far the hardest. Not all the patches applied cleanly, or were even appropriate, and I had to go through the patches one by one.

The reason I stopped actively maintaining this port is that it used to only use python2. Then it started using python3 as well. I just didn't have enough time. I always planned to take it up again, but I never did.

So if you can make it depend upon existing FreeBSD ports, that would really make things a lot easier.

Diff Detail

Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

thierry created this revision.Mar 26 2020, 4:34 PM
lwhsu added a subscriber: lwhsu.Mar 26 2020, 4:59 PM
arrowd added a subscriber: arrowd.Mar 28 2020, 4:45 PM
arrowd added inline comments.Mar 28 2020, 6:17 PM
math/sage/files/patch-build_pkgs_gfan_spkg-install
1 ↗(On Diff #69895)

There is math/gfan port in our tree. Installing it and running Sage's configure script makes it picked up:

gfan-0.6.2.p1: using system package; SPKG will not be installed

I haven't tried building Sage with system gfan yet. Maybe dependending on this package will make this patch not needed.

stephen_missouri.edu edited the test plan for this revision. (Show Details)Mar 28 2020, 6:59 PM
thierry added inline comments.Mar 28 2020, 6:59 PM
math/sage/files/patch-build_pkgs_gfan_spkg-install
1 ↗(On Diff #69895)

AFAIK for a system package to be picked up, a sage_require_xxx=no must be set for configure (unless patching Sage's build mechanism), and there is no sage_require_gfan in 9.0.

Meanwhile, it has been added in their ticket #28985 (See https://trac.sagemath.org/ticket/28985 ) and should be available in 9.1.

Many others are planned, and the full list is available at https://trac.sagemath.org/ticket/27330 .

arrowd added inline comments.Mar 28 2020, 7:09 PM
math/sage/files/patch-build_pkgs_gfan_spkg-install
1 ↗(On Diff #69895)

Oh, right, I'm building it from git. Ok, let's keep this in mind and try to make a switch in 9.1

math/sage/files/patch-build_pkgs_tachyon_patches_Make-arch.patch
2

I just created a port for tachyon - graphics/tachyon. Consider using it to fulfill the dependency.

thierry added inline comments.Mar 29 2020, 10:53 AM
math/sage/files/patch-build_pkgs_tachyon_patches_Make-arch.patch
2

👍

arrowd added a comment.EditedApr 5 2020, 11:20 AM

For some reason, I can't apply this diff onto current tree. Maybe rebasing it will help?

For some reason, I can't apply this diff onto current tree. Maybe rebasing it will help?

AFAIK the only change is from Antoine:

+ DEPRECATED= Broken for more than 6 months
+ EXPIRATION_DATE= 2020-05-05

I'm working on an update to catch sage-9.1.beta9, but it is not yet ready…

lwhsu added a comment.Apr 7 2020, 9:52 AM

AFAIK the only change is from Antoine:

+ DEPRECATED= Broken for more than 6 months
+ EXPIRATION_DATE= 2020-05-05

I'm working on an update to catch sage-9.1.beta9, but it is not yet ready…

I think you can extend this date if you're working on it.

thierry updated this revision to Diff 70333.EditedApr 8 2020, 10:41 AM

New diff, created when using sage-9.1.beta9.

This is still a WIP, but it is much better than the previous one: all the possible external dependencies ATM are taken from the ports tree.

Note: it uses a lot of dependencies! To test it, `make install-missing-packages' is your friend!

But unfortunately, there are still many problems, mostly caused by staging,e.g.
Error: 'lib/libldl.so.2.2.6' is referring to /usr/ports/math/sage/work/stage
Error: 'lib/libfflas_c.so.1.0.0' is referring to /usr/ports/math/sage/work/stage
...

I forgot to add: the patch included in PR 245109 for math/ntl must be applied.

AFAIK the only change is from Antoine:

+ DEPRECATED= Broken for more than 6 months
+ EXPIRATION_DATE= 2020-05-05

I'm working on an update to catch sage-9.1.beta9, but it is not yet ready…

I think you can extend this date if you're working on it.

For this WIP, I totally removed it. Let's see if everything can be fixed before May the 5th (may be 4th be with us!)

thierry updated this revision to Diff 71192.Thu, Apr 30, 3:38 PM

Many progresses with 9.1.rc1: more modules are no more built by Sage but taken from system packages, and this solves the staging problems with them.

But also a bad news: I tried to take ECL and Maxima from our packages, but without success, see https://trac.sagemath.org/ticket/26249#comment:40
Maybe a Lisp guru could have an idea? Else let's it for now and just register the conflicts.

Now that gfan is borrowed from ports, we don't need files/patch-build_pkgs_gfan_spkg-install.in, I presume?

Now that gfan is borrowed from ports, we don't need files/patch-build_pkgs_gfan_spkg-install.in, I presume?

Good catch!

thierry updated this revision to Diff 71203.Thu, Apr 30, 6:54 PM

Switch to 9.1-rc2, with some minor fixes.

thierry updated this revision to Diff 71251.Fri, May 1, 8:20 PM

At some point I was fighting against libomp, brought directly clang from devel/llvm90 for testing purpose… and forgot to remove it! This version is much cleaner.

Also remove two more bundled packages by replacing them by math/libhomfly and math/linbox.

thierry updated this revision to Diff 71351.Mon, May 4, 10:03 AM

More progresses: now Maxima and GAP are fixed.

Prerequisites: apply the patches in PR 242509 (devel/isl) and PR 246149 (math/gap) (or wait for them to be committed)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242509
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246149

thierry updated this revision to Diff 71385.Mon, May 4, 9:06 PM

Latest evolution, with zn_poly taken from ports.

thierry updated this revision to Diff 71609.Sun, May 10, 12:44 PM
  • Upgrade to 9.1.rc4;
  • Also use a Python library (e.g. with math/py-cvxopt): it opens a promising way, because all the stuff built by Sage around cvxopt is now clean (no more references to STAGEDIR), but unfortunately upstream does not like it…
thierry updated this revision to Diff 71828.Fri, May 15, 3:25 PM

Some news:
+ r535281 on math/giacxcas fixed problems in libgiac and indirectly in other libs (e.g. mpfi)

+ added a generic way to use Python modules from our packages: using py-numpy and py-scipy solves many staging problems: to be continued!

thierry updated this revision to Diff 71894.Sun, May 17, 8:37 PM

Update to 9.1.rc5 and use more Python modules from the ports tree.

thierry updated this revision to Diff 72123.Fri, May 22, 4:56 PM

With the two new ports lang/ecl-sage and math/maxima-sage (See D24958 and D24959), the main problem of this port are solved!

thierry updated this revision to Diff 72194.Sun, May 24, 6:57 PM

Update to the 9.1 release.

With the last pieces (JSmol - see PR 246703, MathJax and threejs), it builds and installs (almost) cleanly, without staging problems.

Why "almost"? Because its plist contains several files previously installed by its dependencies (e.g. ipython, sphynx, etc.)!
Next step: try to use external packages for these ones, or remove them for STAGEDIR if not possible.

Many other ports could also be used from external packages, to avoid conflicts, even if they are not a dependency (e.g. devel/py-babel, textproc/py-rst2html5, etc.).