Trying to rescue the port math/sage

[thierry]

Summary

math/sage, also known as SageMath is a very important application for people working in mathematics or sciences.
Unfortunately, this must-have is also a nightmare for packagers, and ATM the FreeBSD port is lagging and broken.

This patch fixes it and upgrades it to the latest stable version (9.1).

This is a kind of meta-application, which builds as many math-related packages as possible to give them a common interface, and all these sub-packages are bundled. Upstream knows about this problem, and a ticket has been opened to "use as many system packages as possible": see #27330 at https://trac.sagemath.org/ticket/27330.

The proposed patch uses system packages from this list, completed with others which are not stage-clean when bundled, and others common ports to avoid as many conflicts as possible.

Test Plan

First, if I ever recreated this port, I would rename it to sagemath. Also, if someone else wants to formally take over as maintainer, I am completely fine with that.

Next, when I created the original port I tried to follow their original vision of having the port build its own dependencies. But it was very difficult to figure out how to apply patches to tarballs. Hence the patches like patch-build_pkgs_*_spkg-install. Also, I had to find the patches applied to the freebsd port, and transfer them to the sage port. This was very time consuming. Transferring the patches from python was by far the hardest. Not all the patches applied cleanly, or were even appropriate, and I had to go through the patches one by one.

The reason I stopped actively maintaining this port is that it used to only use python2. Then it started using python3 as well. I just didn't have enough time. I always planned to take it up again, but I never did.

So if you can make it depend upon existing FreeBSD ports, that would really make things a lot easier.

[arrowd]

math/sage/files/patch-build_pkgs_gfan_spkg-install

1 ↗

(On Diff #69895)

There is math/gfan port in our tree. Installing it and running Sage's configure script makes it picked up:

gfan-0.6.2.p1: using system package; SPKG will not be installed

I haven't tried building Sage with system gfan yet. Maybe dependending on this package will make this patch not needed.

[thierry]

math/sage/files/patch-build_pkgs_gfan_spkg-install

1 ↗

(On Diff #69895)

AFAIK for a system package to be picked up, a sage_require_xxx=no must be set for configure (unless patching Sage's build mechanism), and there is no sage_require_gfan in 9.0.

Meanwhile, it has been added in their ticket #28985 (See https://trac.sagemath.org/ticket/28985 ) and should be available in 9.1.

Many others are planned, and the full list is available at https://trac.sagemath.org/ticket/27330 .

[arrowd]

math/sage/files/patch-build_pkgs_gfan_spkg-install

1 ↗

(On Diff #69895)

Oh, right, I'm building it from git. Ok, let's keep this in mind and try to make a switch in 9.1

math/sage/files/patch-build_pkgs_tachyon_patches_Make-arch.patch

2

I just created a port for tachyon - graphics/tachyon. Consider using it to fulfill the dependency.

[thierry]

math/sage/files/patch-build_pkgs_tachyon_patches_Make-arch.patch

2

👍

[arrowd]

For some reason, I can't apply this diff onto current tree. Maybe rebasing it will help?

[thierry]

In D24195#534537, @arrowd wrote:

For some reason, I can't apply this diff onto current tree. Maybe rebasing it will help?

AFAIK the only change is from Antoine:

+ DEPRECATED= Broken for more than 6 months
+ EXPIRATION_DATE= 2020-05-05

I'm working on an update to catch sage-9.1.beta9, but it is not yet ready…

[lwhsu]

In D24195#534541, @thierry wrote:

AFAIK the only change is from Antoine:

+ DEPRECATED= Broken for more than 6 months
+ EXPIRATION_DATE= 2020-05-05

I'm working on an update to catch sage-9.1.beta9, but it is not yet ready…

I think you can extend this date if you're working on it.

[thierry]

New diff, created when using sage-9.1.beta9.

This is still a WIP, but it is much better than the previous one: all the possible external dependencies ATM are taken from the ports tree.

Note: it uses a lot of dependencies! To test it, `make install-missing-packages' is your friend!

But unfortunately, there are still many problems, mostly caused by staging,e.g.
Error: 'lib/libldl.so.2.2.6' is referring to /usr/ports/math/sage/work/stage
Error: 'lib/libfflas_c.so.1.0.0' is referring to /usr/ports/math/sage/work/stage
...

I forgot to add: the patch included in PR 245109 for math/ntl must be applied.

[thierry]

In D24195#535056, @lwhsu wrote:

In D24195#534541, @thierry wrote:

AFAIK the only change is from Antoine:

+ DEPRECATED= Broken for more than 6 months
+ EXPIRATION_DATE= 2020-05-05

I'm working on an update to catch sage-9.1.beta9, but it is not yet ready…

I think you can extend this date if you're working on it.

For this WIP, I totally removed it. Let's see if everything can be fixed before May the 5th (may be 4th be with us!)

[thierry]

Many progresses with 9.1.rc1: more modules are no more built by Sage but taken from system packages, and this solves the staging problems with them.

But also a bad news: I tried to take ECL and Maxima from our packages, but without success, see https://trac.sagemath.org/ticket/26249#comment:40
Maybe a Lisp guru could have an idea? Else let's it for now and just register the conflicts.

[arrowd]

Now that gfan is borrowed from ports, we don't need files/patch-build_pkgs_gfan_spkg-install.in, I presume?

[thierry]

In D24195#542301, @arrowd wrote:

Now that gfan is borrowed from ports, we don't need files/patch-build_pkgs_gfan_spkg-install.in, I presume?

Good catch!

[thierry]

Switch to 9.1-rc2, with some minor fixes.

[thierry]

At some point I was fighting against libomp, brought directly clang from devel/llvm90 for testing purpose… and forgot to remove it! This version is much cleaner.

Also remove two more bundled packages by replacing them by math/libhomfly and math/linbox.

[thierry]

More progresses: now Maxima and GAP are fixed.

Prerequisites: apply the patches in PR 242509 (devel/isl) and PR 246149 (math/gap) (or wait for them to be committed)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242509
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246149

[thierry]

Latest evolution, with zn_poly taken from ports.

[thierry]

• Upgrade to 9.1.rc4;

• Also use a Python library (e.g. with math/py-cvxopt): it opens a promising way, because all the stuff built by Sage around cvxopt is now clean (no more references to STAGEDIR), but unfortunately upstream does not like it…

[thierry]

Some news:
+ r535281 on math/giacxcas fixed problems in libgiac and indirectly in other libs (e.g. mpfi)

+ added a generic way to use Python modules from our packages: using py-numpy and py-scipy solves many staging problems: to be continued!

[thierry]

Update to 9.1.rc5 and use more Python modules from the ports tree.

[thierry]

With the two new ports lang/ecl-sage and math/maxima-sage (See D24958 and D24959), the main problem of this port are solved!

[thierry]

Update to the 9.1 release.

With the last pieces (JSmol - see PR 246703, MathJax and threejs), it builds and installs (almost) cleanly, without staging problems.

Why "almost"? Because its plist contains several files previously installed by its dependencies (e.g. ipython, sphynx, etc.)!
Next step: try to use external packages for these ones, or remove them for STAGEDIR if not possible.

Many other ports could also be used from external packages, to avoid conflicts, even if they are not a dependency (e.g. devel/py-babel, textproc/py-rst2html5, etc.).

[thierry]

Here we are!

With this patch, math/sage can be installed on a normal workstation without overwriting files of its dependencies, so that it's more easy to test. My first tests are OK.

It's also possible to build it in poudriere: it's OK for 12.1/amd64.

Next step: use more ports as dependencies to avoid building the bundled stuff by SageMath and to avoid conflicts.

[arrowd]

Awesome work, greatly appreciated!

Are there any open Bugzilla PRs that should be resolved first, or this diff can be committed right away?

[thierry]

In D24195#551218, @arrowd wrote:

Are there any open Bugzilla PRs that should be resolved first, or this diff can be committed right away?

It's not yet ready to be committed: the main problem is with lang/ecl and math/maxima. I proposed to fork these ports in D24959 and D24958, but salvadore@ might have a better solution.
Meanwhile I'm still working on reducing the amount of bundled sub-packages.

[thierry]

With this version, I would say that it's now committable (pending D24958 and D24959, and blocked by PR 242509):

• the build is no more done directly in $STAGEDIR

• more dependencies but a smaller plist

• CONFLICTS are registered when it is not possible to use an external package

• basic tests are OK, and also use case involving external tools like GAP, Maxima, Singular, etc.

Note 1: stephen@, I have noted your wish to repocopy this port to math/sagemath, but I think it's better to do that afterwards.

Note 2: thanks to lock-down, I have been able to find the time to work on this, but on normal conditions, it's too much time consuming for one maintainer! I guess that it should be maintained by a team (math or something like that).

[thierry]

For those interested, a poudriere log is available at https://people.freebsd.org/~thierry/sage-math-9.1.log.bz2

Of course, as always, there remains some possible improvements. E.g. it should be possible make a port of exportSageNB and of sagetex in order to use them as external dependencies, but they were not on my top priorities.

[thierry]

Do not depend on ecl-sage but directly on lang/ecl after D25096 has been committed.

Also there in no more problem with devel/isl (thanks to arrowd@ for committing PR 242509).

[thierry]

Follow-up D24959 and directly depends on math/maxima. The proposed math/maxima-sage is no more needed!

[thierry]

With r538922 (math/maxima) committed by salvadore, this revision becomes committable.

[thierry]

Just committed.

Note: I took the maintainership, but I'd like to create a team for it!

[arrowd]

A big thank you for pulling this over. I'll look into improving the port after I play with it a bit. Again, your work is greatly appreaciated.