Page MenuHomeFreeBSD

New port: science/g2o: General framework for graph optimization
ClosedPublic

Authored by yuri on Dec 24 2017, 8:23 PM.

Details

Summary

This port can be built for systems with different levels of SSE.
It uses flavors for this.

g2o-sse2 would be for SSE2 system, g2o-sse4a would be for SSE4A system.
I made one flavor defined for non-Intel platforms (nosimd), because in general SIMD is available on other platforms too.

If some port needs to depend on science/g2o, it would need to depend using the SIMD support level: g2o-${SIMD}>0:science/g2o.
The way how the value ${SIMD} is determined isn't specified. This should ultimately be supported by the framework and pkg tools, and should be based on the current system support.

There are certainly many other ports that need such SIMD-based flavors.

Diff Detail

Repository
rP FreeBSD ports repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

science/g2o/Makefile
26 ↗(On Diff #36976)

Why not enable SSE_AUTODETECT instead of doing all the flavours?

science/g2o/pkg-descr
22 ↗(On Diff #36976)

This is a really long pkg-descr. Does this "Based on the paper" block give our end-users anything?

yuri marked 2 inline comments as done.

.

science/g2o/Makefile
26 ↗(On Diff #36976)

With SSE_AUTODETECT the current SSE support on the package building server will be recorded, and such package will generally break on systems with lower SSE support in CPU.

This is why it isn't a trivial thing, and I did it this way. I tried to make this package to support SIMD properly, as opposed to just doing SSE2, or just doing autodetect. Both of these approaches are generally incorrect.

adamw added inline comments.
science/g2o/Makefile
26 ↗(On Diff #36976)

Ah, got it. It's a build-time autodetection. I'd assumed it was a runtime autodetection. Flavours makes sense.

This revision is now accepted and ready to land.Dec 24 2017, 8:41 PM
svn: E165001: Commit failed (details follow):
svn: E165001: Commit blocked by pre-commit hook (exit code 1) with output:
Do not commit a port with FLAVORS without first
getting approval from portmgr.
This revision now requires review to proceed.Dec 24 2017, 9:11 PM
mat requested changes to this revision.Dec 27 2017, 2:30 PM

Like I already told you in another review, FLAVORS are not for this.
Always build with the lowest available instructions.

This revision now requires changes to proceed.Dec 27 2017, 2:30 PM
In D13610#285332, @mat wrote:

Like I already told you in another review, FLAVORS are not for this.
Always build with the lowest available instructions.

Why lowest? Performance will suffer.
How do we support other SSE levels then?

For other SSE levels, the whole project needs to be rebuilt. (unless they do dynamic SSE detection and switching, which g2o does not)

I will submit this question for discussion on ports@.

In D13610#285332, @mat wrote:

Like I already told you in another review, FLAVORS are not for this.
Always build with the lowest available instructions.

Why lowest? Performance will suffer.
How do we support other SSE levels then?

For other SSE levels, the whole project needs to be rebuilt. (unless they do dynamic SSE detection and switching, which g2o does not)

I will submit this question for discussion on ports@.

In D13610#285369, @yuri wrote:

Why lowest?

So that packages work on all machines from that architecture. For amd64, for example, it is SSE2. And nobody has i386 any more and wants to use this.

Performance will suffer.

Anyone who wants edge performances can rebuild the port. I highly doubt anyone will.

How do we support other SSE levels then

Make them options, like I had you do for the other port a week or so ago.

In D13610#285370, @mat wrote:
In D13610#285369, @yuri wrote:

Make them options, like I had you do for the other port a week or so ago.

Mat, it's not that options can't be used here. The problem is that with SIMD supported through options it is much harder for users. User needs to identify which ports use SIMD, rebuild each of them manually. Then repeat the procedure after each system upgrade.
This is too much hassle for SIMD support which is a mainstream CPU feature today.

The point of this suggestion is to make a step towards the proper SIMD support, which will be very easy or automatic for users.

I would suggest patching the project (or requsesting the project devs make the change) so that it builds all available options and chooses which code path to use at runtime, this way the best option is always available with only one build.

Google "choose sse avx code path at runtime" for some suggestions on how to do this.

While I'm not sure where the decision is made, the cycles render engine in blender is an example that compiles all available options, regardless of builder's cpu features.

https://developer.blender.org/diffusion/B/browse/master/intern/cycles/kernel/kernels/cpu/

I would suggest patching the project (or requsesting the project devs make the change) so that it builds all available options and chooses which code path to use at runtime, this way the best option is always available with only one build.

Google "choose sse avx code path at runtime" for some suggestions on how to do this.

While I'm not sure where the decision is made, the cycles render engine in blender is an example that compiles all available options, regardless of builder's cpu features.

https://developer.blender.org/diffusion/B/browse/master/intern/cycles/kernel/kernels/cpu/

Thanks.
So far, I asked the upstream author how/where does he use SIMD.

yuri planned changes to this revision.Jan 12 2018, 8:38 AM
This revision was not accepted when it landed; it landed in state Changes Planned.Mar 28 2018, 9:18 AM
This revision was automatically updated to reflect the committed changes.