Upgrade net/ceph to version 12.2.2
ClosedPublic

Authored by wjw_digiware.nl on Dec 30 2017, 11:02 PM.

Details

Summary

In order to support Boost 1.66 this is in preparation for the upgrade
port will build with both Boost 1.65 and 1.66
But the Boost 1.66 libraries will only recognised correctly IFF when
the CMake port is patched to actually recognised this Boost version in
Modules.

Test Plan

Ceph has its onw testset included, and is correctly executed on this
version.

Diff Detail

Repository
rP FreeBSD ports repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
wjw_digiware.nl edited the summary of this revision. (Show Details)Dec 30 2017, 11:09 PM

Since my port was change in between by ports-managers
there were some unneeded changes.
So reduce the difference between version.

Accidently include the Boost 1.66 updates.
Try to remove them from this PR

Reduce diffs on white spaces and trivia

jbeich added a subscriber: jbeich.Dec 30 2017, 11:35 PM
jbeich added inline comments.
net/ceph/Makefile
65 ↗(On Diff #37258)

Why do you need LLD? If it's just to fix build maybe try the following:

.if ${/usr/bin/ld:L:tA} != "/usr/bin/ld.lld"
# <describe rationale or error message briefly>
USE_BINUTILS=		yes
LDFLAGS+=		-B${LOCALBASE}/bin
.endif
  • LLD before 5.0.1 (e.g., on FreeBSD 11.1) maybe unstable
  • -fuse-ld=lld would be a shorter way to use it
  • Did you check CMAKE_*_LINK_FLAGS doesn't override LDFLAGS from make.conf?
jbeich added inline comments.Dec 30 2017, 11:37 PM
net/ceph/Makefile
21 ↗(On Diff #37261)

Convert to USE_PYTHON=cython, see rP455210.

@jbeich

  1. I need LLD because otherwise one off the executables does not link. Seems to be a size limitation in the regular ld And after discussion with Ed Maste this was the solution that worked. Not sure how that impacts on 11.x. But my Ceph testruns do seem to work.
  2. I tried all kinds of incantations to get LLD to work, and this was the one that actually worked. It was painfull since it needs a poudriere rerun for every attempt. And even with ccache Ceph is big to build.
  3. Not really sure on how to verify that. But I would expect LDFLAGS to end up in LINK_FLAGS
net/ceph/Makefile
21 ↗(On Diff #37261)

Ah, I did add that line from the online version but did
not remove this line.

Sort of a pitty that changes by others to the port I maintain
are not reported to me. So I'm notified automatically.
And can update my own kept files.

Will change

Move cython usage to USE_PYTHON

Remove verbose output during linking
leftover from getting lld to work

net/ceph/Makefile
65 ↗(On Diff #37258)

@jbeich
This .if-.endif construct prevents misery on 11.1?
Because it seems to work for me on 12-CURRENT.

jbeich accepted this revision.Dec 31 2017, 12:36 AM

Is it OK to land before Boost 1.66.0 for 2018Q1 users given Ceph 12.2.2 builds fine against Boost 1.65.1? Or did you disable some features (e.g., WITH_RADOSGW_BEAST_FRONTEND) for the sake of Boost 1.66?

net/ceph/Makefile
65 ↗(On Diff #37258)

I haven't tested in net/ceph yet but the conditional is based on rP457430 which itself is based on rP448576 and relies on ld.lld being a symlink (unlike ld.bfd). -B${LOCALBASE}/bin doesn't force LLD, so is less likely to break custom LDFLAGS in make.conf or on architectures still stuck with GCC (powerpc*).

This revision is now accepted and ready to land.Dec 31 2017, 12:36 AM

Is it OK to land before Boost 1.66.0 for 2018Q1 users given Ceph 12.2.2 builds fine against Boost 1.65.1? Or did you disable some features (e.g., WITH_RADOSGW_BEAST_FRONTEND) for the sake of Boost 1.66?

Exactly,
There is a small issue in Beast, and that is with a command for which there is an alternative.
But it can land before Boost 1.66 lands, giving you the option to pull the trigger on Boost whenever you have also fixed the Cmake FindBoost module.
And net/Ceph should build on both sides of the barrier.

net/ceph/Makefile
65 ↗(On Diff #37258)

Would not be able to help with PPC stuff if it broke. :)
Running it now in my 11.1 poudriere jail, but it is a large build, and all kinds of dependancies need to be upgraded as well.

ultima added a subscriber: ultima.Dec 31 2017, 1:16 AM
ultima added inline comments.
net/ceph/Makefile
5 ↗(On Diff #37265)

DISTVERSIONPREFIX= v

net/ceph/Makefile
5 ↗(On Diff #37265)

@ultima
Looks like a sensible one, will test once poudriere is done running the 11.1 version

Cleanup of versionstring in package.
And remove the v from v12.2.2

This revision now requires review to proceed.Dec 31 2017, 2:03 AM
wjw_digiware.nl marked 2 inline comments as done.Dec 31 2017, 2:03 AM

Reorder to make portlint happy

@jbeich
Could you approve once again, fixed the way the versioning was done.
This makes more sense IMHO

jbeich accepted this revision.Dec 31 2017, 2:27 AM
This revision is now accepted and ready to land.Dec 31 2017, 2:27 AM
jbeich added inline comments.Dec 31 2017, 3:03 AM
net/ceph/Makefile
65 ↗(On Diff #37258)

ceph-12.2.2 built fine on 11.1 against boost 1.65.1 with the above workaround instead of -fuse-ld=.

jbeich added a comment.EditedDec 31 2017, 5:56 AM
  1. I need LLD because otherwise one off the executables does not link. Seems to be a size limitation in the regular ld And after discussion with Ed Maste this was the solution that worked. Not sure how that impacts on 11.x. But my Ceph testruns do seem to work.

I can't reproduce. Even without -fuse-ld= and -B${LOCALBASE}/bin ceph-12.2.2 builds fine: boost-libs-1.65.1_1 (11.1 amd64, 12.0 amd64), boost-libs-1.66.0 (11.1 amd64, 12.0 amd64). Can you provide more details e.g., poudriere log?

  1. I tried all kinds of incantations to get LLD to work, and this was the one that actually worked. It was painfull since it needs a poudriere rerun for every attempt. And even with ccache Ceph is big to build.

Have you tried disabling ccache?

  1. I need LLD because otherwise one off the executables does not link. Seems to be a size limitation in the regular ld And after discussion with Ed Maste this was the solution that worked. Not sure how that impacts on 11.x. But my Ceph testruns do seem to work.

I can't reproduce. Even without -fuse-ld= and -B${LOCALBASE}/bin ceph-12.2.2 builds fine: boost-libs-1.65.1_1 (11.1 amd64, 12.0 amd64), boost-libs-1.66.0 (11.1 amd64, 12.0 amd64). Can you provide more details e.g., poudriere log?

Oke, interesting since it has been bothering me for quite some time.
And I spent plenty of poudriere runs to find out what to use here.

I'll see if I can get a poudriere run going today with regular ld.bfd as it is.

  1. I tried all kinds of incantations to get LLD to work, and this was the one that actually worked. It was painfull since it needs a poudriere rerun for every attempt. And even with ccache Ceph is big to build.

Have you tried disabling ccache?

On and off, but not systematically. But (test)building ceph without ccache is a long
process on the limited hardware I have.

I'll run some more tests, but it seems that you are no longer held back by ceph
in your migration?

net/ceph/Makefile
65 ↗(On Diff #37258)

Yup, this is what I found in my poudriere this morning.
So there is no problem there for 11.1

Closed by commit rP457752: net/ceph: update to 12.2.2 (authored by jbeich, committed by ). · Explain WhyJan 1 2018, 1:36 AM
This revision was automatically updated to reflect the committed changes.