Page MenuHomeFreeBSD

sysutils/apache-mesos: Add option to enable Java bindings
ClosedPublic

Authored by james.wright_digital-chaos.com on Jun 19 2020, 2:14 PM.

Details

Summary

Added new JAVA port option (on by default) to enable Java bindings for libmesos.

Issue: Maven Build Dependencies
Enabling Java bindings causes Maven to be invoked mid-way through the build process to build the mesos.jar artifact.
On a ports build system this will fail as internet connectivity is only available during the fetch phase.

Resolution
Pushed all required Maven deps into GitHub and created a release tarball, so they can be fetched as a normal distfile;
https://github.com/Digital-Chaos/mesos-maven-deps

Issue: Mismatched Protobuf.jar version
The mesos build already has a dependency on devel/protobuf from ports (currently version 3.12.2). However enabling java bindings in the build also requires protobuf.jar from Maven which has the version hardcoded to 3.5.0 within mesos.pom.in. This version mismatch will cause the build to fail.

Resolution
Created a patch (files/extra-patch-src_java_mesos.pom.in) to replace the harcoded protobuf version with a placeholder @PROTOBUF_VERSION@ string. During the post-patch phase, this placeholder is replaced inline from the PROTOBUF_VERSION Makefile variable which is set to the current ports version of protobuf (3.12.2). Would be better if PROTOBUF_VERSION could be set to the current version of devel/protobuf in the Makefile rather than hardcoding, but I'm not sure how that would be done, any advice appreciated!

Bugzilla
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247419

Test Plan

portlint
make stage-qa

Diff Detail

Repository
rP FreeBSD ports repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

lwhsu retitled this revision from apache/mesos: Add option to enable Java bindings to sysutils/apache-mesos: Add option to enable Java bindings.Jun 19 2020, 2:21 PM
Makefile
88 ↗(On Diff #73333)

${HOME}/.m2 is not writeable when built with poudriere (see link below), could we use ${TMPDIR}/.m2 instead?

https://gist.github.com/lwhsu/7298ec52c6c4dad0c7af1cc5792cebdf

lwhsu added inline comments.
Makefile
88 ↗(On Diff #73333)

TBH I don't know what's the right solution of this. We probably need to consult people on -java@.

Makefile
88 ↗(On Diff #73333)

Agreed. Although it does seem ${HOME}/.m2 is not going to work whatever is decided. Could we try another build if I change the local Maven repo to ${TMPDIR}/.m2?

Makefile
88 ↗(On Diff #73333)

Oh I totally forgot one thing, if we're using maven to download things to .m2, then it mean a violation of the rule of not downloading anything in the build phase. We need to check how other java ports resolve this, and/or ask on -ports@ and -java@.

Makefile
88 ↗(On Diff #73333)

It's downloading all the dependencies upfront in the fetch phase not the build phase (see line 96 of the Makefile). Some other ports I looked at have bundled the dependencies into a tarball and hosted on a personal/private repo, which IMHO is suboptimal compared with fetching the artifacts directly from the offical Maven repo upfront in the fetch phase.

Makefile
88 ↗(On Diff #73333)

Oh ok this sounds better, but this files are not verified with checksum, which might not be good enough. Is there a way to download them to ${DISTDIR} or maybe ${DISTDIR}/m2 and put the checksum in distinfo for verifying in the checksum phase?

Makefile
88 ↗(On Diff #73333)

I had considered downloading the maven deps into ${DISTDIR}/.m2, but then didn't know how i'd go about generating the distinfo with all the checksums (there are a lot of artifacts downloaded). Also it seems redundant doing our own checksumming/verifying when Maven itself is checksumming/verifying everything it does from the official repo. If we can't trust Maven to get it right, then we have bigger problem. :-)

Makefile
88 ↗(On Diff #73333)

Indeed. I have this question for a long time, but still don't have a good solution. Do you mind to start a thread on -ports@ to discuss how should we deal with those java ports with maven?

I've updated the patch to pull all the required Maven dependencies from a GitHub repo, as suggested by the feedback from -ports@
I would like to look into the wider issue of resolving/fetching dependencies for Maven based ports, but is easier to try that with a smaller and simpler project first, rather than Mesos which is quite complex.

distinfo
4 ↗(On Diff #73735)

Do you think we can have a more unique name? This looks too general and I am worried that it could conflict with others in $DISTDIR, or its suggested to set $DIST_SUBDIR

New Changes;
Added DIST_SUBDIR
Download Maven deps from GitHub with more descriptive filename (mesos-maven-deps-1.10.0.tar.gz)

james.wright_digital-chaos.com added inline comments.
distinfo
4 ↗(On Diff #73735)

Completely agree, have updated patch with both DIST_SUBDIR and a more unique filename as suggested.

This revision is now accepted and ready to land.Jun 27 2020, 7:15 PM
This revision was automatically updated to reflect the committed changes.