Switch C compiler used by GHC to Clang.

[arrowd]

Summary

This diff changes the C compiler used by GHC to base clang.

• Completed poudriere bulk -t lang/ghc */hs-* devel/stack on 10.3 and 11.0 jails.
• Built https://github.com/haskell/haskell-ide-engine using stack and GHC built with this change. HIE is a pretty heavy dependence-wise and depends on foreign libraries too. Compiled binary works and tests pass.
• Built https://github.com/llvm-hs/llvm-hs with some patching (see [1] below). Before there was a nasty problem with this package, as it used llvm-config to configure itself and llvm* packages on FreeBSD are compiled with clang. This causes clang-specific flags (various -W ones) to be present in llvm-config output. When GHC tries to compile C++ bits with GCC, it chokes on these flags. The only way to build llvm-hs with GCC-orientired GHC I found was to temporalily symlink clang to gcc. With this change only minor patching is required to build this package.
• Built pandoc with stack and system GHC, and verified it is working.

[1] Some haskell packages that are not yet in ports tree depend on C++ foreign libraries and have extra-libraries: stdc++ in their .cabal files. This causes them to fail with my patch, but it is easily fixed. An example of such package and corresponding change is https://github.com/arrowd/charsetdetect-ae/commit/7f7e8b15080006cb4e285c62b4fada3011878142 . Another example is llvm-hs mentioned before. I plan to upstream fixes to these packages once this diff is landed.

[arrowd]

lang/ghc/files/patch-compiler_main_SysTools.hs

1 ↗

(On Diff #32117)

This patch is upstreamed in https://phabricator.haskell.org/D3840

lang/ghc/files/patch-rts_Linker.c

1 ↗

(On Diff #32117)

This patch is upstreamed in https://phabricator.haskell.org/D3841

[mat]

The two commented out USE_GCC lines should probably be removed.

[tcberner]

devel/hs-c2hs/files/patch-src_C2HS_Config.hs

26 ↗

(On Diff #32117)

Were there build failures when you left that in? If not I would opt to keep it to be on the safe side.

lang/ghc/bsd.cabal.mk

103 ↗

(On Diff #32117)

This comment can now go too ...

[tcberner]

Also

Warning: Couldn't figure out C compiler information!
         Make sure you're using GNU gcc, or clang

I think you still need to modify one of the files I mentioned in the 8.0.2 review to get rid of that.

[arrowd]

In D12043#250952, @tcberner wrote:

Also

Warning: Couldn't figure out C compiler information!
         Make sure you're using GNU gcc, or clang

I think you still need to modify one of the files I mentioned in the 8.0.2 review to get rid of that.

If I understand it right, this message is produced by bootstrap GHC, which is not yet aware of FreeBSD clang (patch-compiler_main_SysTools.hs patch). The resulting GHC does not produce these messages anymore.

[arrowd]

Remove comments, revert changes to C2HS patches.

[tcberner]

In D12043#250992, @6yearold_gmail.com wrote:

In D12043#250952, @tcberner wrote:

Also

Warning: Couldn't figure out C compiler information!
         Make sure you're using GNU gcc, or clang

I think you still need to modify one of the files I mentioned in the 8.0.2 review to get rid of that.

If I understand it right, this message is produced by bootstrap GHC, which is not yet aware of FreeBSD clang (patch-compiler_main_SysTools.hs patch). The resulting GHC does not produce these messages anymore.

Ah, great :)

[pgj]

Would not it be possible to make this optional (like it was before)? I am not endorsing GCC here, but in my experience it is much more supported in Haskell circles. So, I would keep that as the default, and make the use of Clang optional as an option to the port. Also, we would not have to be so strict on build failures that way.

[arrowd]

In D12043#251038, @pgj wrote:

Would not it be possible to make this optional (like it was before)? I am not endorsing GCC here, but in my experience it is much more supported in Haskell circles. So, I would keep that as the default, and make the use of Clang optional as an option to the port. Also, we would not have to be so strict on build failures that way.

The whole point of this patch was to get rid from GCC completely. Clang from its inception aimed to be highly compatible with GCC. I really doubt that any discrepancies left between them today could really affect GHC working somehow. RTS is written in C (which is a lot simpler that C++) and doesn't contain any subtle code features. After all, how Clang could become more tested with GHC, if no distro or OS would use it with GHC? We can't measure the GHC-on-Clang stability while we have Clang option off by default.

So, my points (subjective, of course):

1. I really doubt that differencies between GCC and Clang can really affect GHC and code generated by it nowadays.
2. The GCC dependency is heavy and you have to pull it in even for running Haskell programs (not all, of course, but at least ones that use FFI to C++) code.
3. It seems plain wrong to me not to use system C compiler.

I will understand, if you wouldn't approve this change, but I'm not going to work on making this optional.

[pgj]

In D12043#251046, @6yearold_gmail.com wrote:

The whole point of this patch was to get rid from GCC completely.

But you cannot. This is not a base system, it is ports, where it is free to prefer any compiler the maintainer sees feasible.

Clang from its inception aimed to be highly compatible with GCC.

Unfortunately GHC could only reliably build with Clang more or the less since around version 7.10. I can accept that the situation has improved a lot in the last few years, but you have not given any concrete evidences on that. Building some ports successfully with this change is not something that would warrant making this a default.

I really doubt that any discrepancies left between them today
could really affect GHC working somehow. RTS is written in C [..]
and doesn't contain any subtle code features.

This is not about the run-time system. GHC applies the associated C compiler and linker in building Haskell programs, including all the hs- ports in the tree. Many Haskell package authors do not really care about portability and Clang, they often develop on GNU/Linux with GCC in mind. I am not happy with this situation either but I do not have the time to continuously struggle with this.

After all, how Clang could become more tested with GHC, if no distro
or OS would use it with GHC? We can't measure the GHC-on-Clang
stability while we have Clang option off by default.

FreeBSD takes only a very small share in the population of GHC users.

1. I really doubt that differencies between GCC and Clang can really affect GHC and code generated by it nowadays.

See my comment above.

2. The GCC dependency is heavy and you have to pull it in even for running Haskell programs (not all, of course,

but at least ones that use FFI to C++) code.

GCC has to be installed once, I do not see why this introduces any extra burden than that.

3. It seems plain wrong to me not to use system C compiler.

Why? I tried to respect this principle back in about 2010 with GCC 4.2.1 in the base, but I quickly had to realize that was just too old for GHC, so I introduced the dependency on GCC from ports. Anyway, I do not tell you to not to add this patch, but not make it default. If somebody wants to use the system C compiler, she is free to implement that with your changes. Common sense, that is all.

I will understand, if you wouldn't approve this change, but I'm not going to work on making this optional.

I am fine with disagreeing on this. The problem is that you will not be the one who will have to answer the mails and problem reports on the fallout after this is committed. I want approve patches only that I could take the responsibility / blame for. I believe I have the freedom to chose my preferences on this subject as the maintainer and user of this port for 7 years.

[arrowd]

In D12043#251047, @pgj wrote:

In D12043#251046, @6yearold_gmail.com wrote:

The whole point of this patch was to get rid from GCC completely.

But you cannot. This is not a base system, it is ports, where it is free to prefer any compiler the maintainer sees feasible.

Clang from its inception aimed to be highly compatible with GCC.

Unfortunately GHC could only reliably build with Clang more or the less since around version 7.10. I can accept that the situation has improved a lot in the last few years, but you have not given any concrete evidences on that. Building some ports successfully with this change is not something that would warrant making this a default.

And what would make enough evidence? Building whole stackage? I can give it a try, if you ask.

I really doubt that any discrepancies left between them today
could really affect GHC working somehow. RTS is written in C [..]
and doesn't contain any subtle code features.

This is not about the run-time system. GHC applies the associated C compiler and linker in building Haskell programs, including all the hs- ports in the tree. Many Haskell package authors do not really care about portability and Clang, they often develop on GNU/Linux with GCC in mind.

More generally, many software authors don't care about whole FreeBSD OS, or about Clang. But FreeBSD project somehow managed to survive until now and even grow up its ports tree to ~25k packages. Moreover, that "dont care" stance is the one of the reasons to the Ports existance - we had to patch software to make it work. And after patching we try reach out developers asking them to not be too Linux-centric, or GCC-centric, or whatever. I see this as our mission, what makes FreeBSD Ports useful not to only FreeBSD, but to the whole open-source community.

You may argue that when FreeBSD was choosing between GCC 4.2.1 and Clang, it was different - GCC 4.2.1 was no better that Clang in terms of support. I'd agree, but the main point still holds - since we are sort of outsiders in *nix world anyway, we need to aggressively push upstreams instead of submitting to their ways.

I am not happy with this situation either but I do not have the time to continuously struggle with this.

After all, how Clang could become more tested with GHC, if no distro
or OS would use it with GHC? We can't measure the GHC-on-Clang
stability while we have Clang option off by default.

FreeBSD takes only a very small share in the population of GHC users.

So, there is no way to make Clang "more supported in Haskell circles"? I read it like that, at least.

1. I really doubt that differencies between GCC and Clang can really affect GHC and code generated by it nowadays.

See my comment above.

2. The GCC dependency is heavy and you have to pull it in even for running Haskell programs (not all, of course,

but at least ones that use FFI to C++) code.

GCC has to be installed once, I do not see why this introduces any extra burden than that.

The burden is that it weighs 420 Mb. Add 1.1 Gb of ghc package to that and we get pretty fat runtime requirement.

Another problem is packages like llvm-hs. These have C++ cbits, which gets linked to GCC libstdc++, while the main C++ library (LLVM.so) is linked to libc++. These standard libraries are not binary compatible, so things break. Using system compiler for everything ensures, we won't get into such situation.

3. It seems plain wrong to me not to use system C compiler.

Why? I tried to respect this principle back in about 2010 with GCC 4.2.1 in the base, but I quickly had to realize that was just too old for GHC, so I introduced the dependency on GCC from ports.

GCC 4.2.1 was totally outdated, which is not true for current Clang. Both GHC and LLVM community are open to our patches, they don't have "oh, these BSD guys again" stance.

Anyway, I do not tell you to not to add this patch, but not make it default. If somebody wants to use the system C compiler, she is free to implement that with your changes. Common sense, that is all.

I will understand, if you wouldn't approve this change, but I'm not going to work on making this optional.

I am fine with disagreeing on this. The problem is that you will not be the one who will have to answer the mails and problem reports on the fallout after this is committed. I want approve patches only that I could take the responsibility / blame for. I believe I have the freedom to chose my preferences on this subject as the maintainer and user of this port for 7 years.

What makes you the only one having to answer the mails? I was having subtle problems with GHC FFI some time ago, and in the end it was me who solved it and upstreamed the fix. I really appreciate your work, but you can't do all the work alone, as the latest GHC update has shown. Similarly, I and hopefully @tcberner will be around when problems arise (if they do, BTW, because I still think it is not that bad).

To sum this up, I think switching to Clang wouldn't raise problems rate significantly (that is, we had subtle problem just because "we are FreeBSD", now it would be the same because of "GHC-on-Clang"), and you think it would. Since it is you who make decisions here, I'm falling back to the last resort - let me compile whole stackage using GHC-On-Clang. Would this convince you?

[pgj]

Let me add a recent example of why I am not entirely satisfied with base system Clang.

After the GHC 8.0.2 update, haskell@ got an email from a guy who tried to build lang/ghc on an i386 old machine, something that lacks SSE2 (perhaps that was Pentium III) but failed. The reason was that the run-time system of GHC had been built with SSE2-enabled floating-point builtins, such as __floatdidf() in this specific case. After some investigation, I managed to figure out that because lang/ghc links to compiler-rt disguised as libgcc in the base system that has an SSE2-enabled version of the builtins. That is possibly because the i386 base system was built on a much more modern machine (perhaps in a jail on a 64-bit system) so libgcc was built with SSE2-enabled code, which is something that one would not expect. Let me quote the minimum requirements from the FreeBSD Handbook: "Almost all i386-compatible processors with a floating point unit are supported. All Intel® processors 486 or higher are supported."

I worked this problem around by avoiding linking to libgcc from the base system, and use the one that is shipped with GCC.

Note that different base systems come with different Clang versions as well. 10.3-RELEASE has 3.4.1, 11.0-RELEASE has 3.8.0, 11.1-RELEASE has 4.0.0, and 12-CURRENT may have a more recent version. If one installs lang/gcc one will get the same GCC version on every supported FreeBSD version. It may change over time, but it will then change everywhere. This makes testing much easier in my opinion.

[arrowd]

In D12043#251052, @pgj wrote:

Let me add a recent example of why I am not entirely satisfied with base system Clang.

After the GHC 8.0.2 update, haskell@ got an email from a guy who tried to build lang/ghc on an i386 old machine, something that lacks SSE2 (perhaps that was Pentium III) but failed. The reason was that the run-time system of GHC had been built with SSE2-enabled floating-point builtins, such as __floatdidf() in this specific case. After some investigation, I managed to figure out that because lang/ghc links to compiler-rt disguised as libgcc in the base system that has an SSE2-enabled version of the builtins. That is possibly because the i386 base system was built on a much more modern machine (perhaps in a jail on a 64-bit system) so libgcc was built with SSE2-enabled code, which is something that one would not expect. Let me quote the minimum requirements from the FreeBSD Handbook: "Almost all i386-compatible processors with a floating point unit are supported. All Intel® processors 486 or higher are supported."

I worked this problem around by avoiding linking to libgcc from the base system, and use the one that is shipped with GCC.

I don't see why is this Clang's fault. The FreeBSD build system for compiler-rt lacks some flags, and that's it no?

But we digress. Is there something I can do make you consider defaulting to clang, or you just don't want to hear about it?

[pgj]

In D12043#251051, @6yearold_gmail.com wrote:

And what would make enough evidence? Building whole stackage? I can give it a try, if you ask.

It is enough if you just build all the hs- ports in the tree on all currently supported FreeBSD release, on all supported platforms, i386 and amd64.

GCC has to be installed once, I do not see why this introduces any extra burden than that.

The burden is that it weighs 420 Mb. Add 1.1 Gb of ghc package to that and we get pretty fat runtime requirement.

I do not think this should be a real problem in the world of multi-terabyte disks. I never felt this a burden, even when I am typing these words on machine with a 320-GB HDD.

Another problem is packages like llvm-hs.

I do not see a problem there as well: that package should link to LLVM and Clang, as it name suggests.

What makes you the only one having to answer the mails?

Because the maintainer of those is [e-mail], on which is internal mailing list I am on. This is list of committers who once had to do something with the maintenance of FreeBSD Haskell ports, but I am usually the one who spends time on working with the issues. Perhaps this should be replaced with a open mailing list on lists.freebsd.org, so other non-committer contributors, like you could join and reply?

I really appreciate your work, but you can't do all the work alone, as the latest GHC update has shown.

I could if I wished, as many earlier GHC updates has already shown. Actually, I had the opposite feeling with you and the latest GHC update: if I did not join, it could have taken months until the change reaches the tree in a reasonable shape. I did not work on the update by myself because I could not, but because that was not a priority to me. I was happy to see others being interested in the cause, that is why I helped in the first place.

Similarly, I and hopefully @tcberner will be around when problems arise (if they do, BTW, because I still think it is not that bad).

I have different experiences, but I truly hope you are right.

Since it is you who make decisions here, I'm falling back to the last resort - let me compile whole stackage using GHC-On-Clang. Would this
convince you?

I want only a trivial thing: I do not want to deal with the resulting fallout. I do not want to be the blocker of those efforts either, so what I need from you is to declare the you will take care of that without involving me.

[pgj]

In D12043#251056, @6yearold_gmail.com wrote:

I don't see why is this Clang's fault. The FreeBSD build system for compiler-rt lacks some flags, and that's it no?

Because the system C compiler would use that library for building binaries therefore it becomes the compiler's fault. That is an example how unreliable it could possibly be.

[arrowd]

In D12043#251057, @pgj wrote:

Because the maintainer of those is [e-mail], on which is internal mailing list I am on. This is list of committers who once had to do something with the maintenance of FreeBSD Haskell ports, but I am usually the one who spends time on working with the issues. Perhaps this should be replaced with a open mailing list on lists.freebsd.org, so other non-committer contributors, like you could join and reply?

I've been wondering why it wasn't done this way for a long time. Of course, making an open mailing list is a great idea, I'd immediately join it.

I want only a trivial thing: I do not want to deal with the resulting fallout. I do not want to be the blocker of those efforts either, so what I need from you is to declare the you will take care of that without involving me.

Of course I'll take care of the related bug reports.

Ok, so /me is off testing all the various combinations of OS version and architecture.

[arrowd]

Update the revision. With this poudriere bulk lang/ghc */hs-* devel/stack compiles without error on 11.1-RELEASE {i386,amd64}.

During compilation following warning shows up:

 Linking ./setup ...

 In file included from /tmp/ghc96530_0/ghc_4.c:1:0: error: 

 In file included from /usr/local/lib/ghc-8.0.2/include/Rts.h:217:0: error:
    

/usr/local/lib/ghc-8.0.2/include/rts/storage/ClosureMacros.h:505:5: error:
     warning: macro expansion producing 'defined' has undefined behavior [-Wexpansion-to-defined]
#if ZERO_SLOP_FOR_LDV_PROF || ZERO_SLOP_FOR_SANITY_CHECK
    ^

/usr/local/lib/ghc-8.0.2/include/rts/storage/ClosureMacros.h:502:37: error:
     note: expanded from macro 'ZERO_SLOP_FOR_LDV_PROF'
#define ZERO_SLOP_FOR_LDV_PROF     (defined(PROFILING))

This is an upstream bug that already has been fixed. It is harmless despite "error" message. I can pull the patch fixing this issue too, but I plan to update GHC to 8.2 adter this lands anyway.

I can do 10.3-RELEASE exp runs too, but it'd be nice to get a helping hand with it.

[pgj]

In D12043#284778, @6yearold_gmail.com wrote:

Update the revision. With this poudriere bulk lang/ghc */hs-* devel/stack compiles without error on 11.1-RELEASE {i386,amd64}.

Well done! However, please note that the "fun" starts with making this work on 10.x as it was shipped with a badly broken tool chain in base.

During compilation [a] warning shows up [which] is an upstream bug that already has been fixed. [..] I can pull the patch fixing this issue too, but I plan to update GHC to 8.2 after this lands anyway.

All right, that sounds like a good idea.

I can do 10.3-RELEASE exp runs too, but it'd be nice to get a helping hand with it.

What kind of help would you need? Note that you can do the testing of that release with poudriere on a later system only by setting up a 10.3-RELEASE jail. My development hardware is still in pieces (literally) so I cannot help with dedicating computational capacity to this effort.

[arrowd]

In D12043#286756, @pgj wrote:

All right, that sounds like a good idea.

Updating to 8.2 or just pulling in the patch?

What kind of help would you need? Note that you can do the testing of that release with poudriere on a later system only by setting up a 10.3-RELEASE jail. My development hardware is still in pieces (literally) so I cannot help with dedicating computational capacity to this effort.

My dev machine is 11.1, so I can't run 12-CURRENT jails. I'll do runs for 10.3 myself, but would like to ask someone to run 12 ones.

[pgj]

In D12043#286791, @6yearold_gmail.com wrote:

In D12043#286756, @pgj wrote:

All right, that sounds like a good idea.

Updating to 8.2 or just pulling in the patch?

Sorry, I meant pulling the patch in. But if you can do the 8.2 update at the same time that would be more awesome. I did not want to ask for this as that may require you to re-run all the tests that you have done earlier.

My dev machine is 11.1, so I can't run 12-CURRENT jails. I'll do runs for 10.3 myself, but would like to ask someone to run 12 ones.

Perhaps we can ask for help from the Ports Management Team -- they often do exp-runs with patches developers send them. Another solution could be to install bhyve and create a 12-CURRENT FreeBSD guest. A couple of months ago I have used bhyve to run a Windows 10 guest on my development machine and it performed quite well. But you can also try to write about this to the FreeBSD Haskell mailing list.

[arrowd]

Fix Wexpansion-to-defined warnings.

[arrowd]

Runs for 10.3-RELEASE on i386 and amd64 are done without any errors.

[arrowd]

As reported by @tcberner the revision builds fine on 12-CURRENT {i386,amd64} too. Is it ready to be committed now?

[pgj]

Sure. But I will not commit it myself as I do not have the resources to double check the changes (so I could not take the responsibility for them, including committing further fixes for potential fallout).

[arrowd]

Should all PORTREVISIONs for lang/ghc, */hs-* and devel/stack be bumped before commiting?

[arrowd]

Bump PORTREVISION's of lang/ghc, */hs-* and devel/stack.

[arrowd]

Fix some PORTREVISION vars going before PORTVERSION.

[tcberner]

Awesome. Can you please ask for an exp-run of this?