Page MenuHomeFreeBSD

www/qt6-webengine: NEW PORT
ClosedPublic

Authored by kai on Dec 22 2022, 7:45 PM.
Tags
None
Referenced Files
Unknown Object (File)
Wed, Feb 28, 9:27 AM
Unknown Object (File)
Mon, Feb 12, 9:16 AM
Unknown Object (File)
Mon, Feb 12, 1:55 AM
Unknown Object (File)
Mon, Feb 12, 1:54 AM
Unknown Object (File)
Mon, Feb 12, 1:54 AM
Unknown Object (File)
Mon, Feb 12, 1:54 AM
Unknown Object (File)
Mon, Feb 12, 1:43 AM
Unknown Object (File)
Jan 17 2024, 7:40 PM

Details

Summary
The Qt WebEngine module provides a web browser engine that makes it easy
to embed content from the World Wide Web into your Qt application on
platforms that do not have a native web engine.

Qt WebEngine provides C++ classes and QML types for rendering HTML,
XHTML, and SVG documents, styled using Cascading Style Sheets (CSS) and
scripted with JavaScript. HTML documents can be made fully editable by
the user through the use of the contenteditable attribute on HTML
elements.

Most of the patches were taken from Chromium 102.0.5005.115
(6fbdece63f38) and a small portion also from Chromium 105.0.5195.102
(96b9a69e7010) , which has been required since Qt 6.4.1.

In contrast to www/qt5-webengine, some things have changed in the build
process:

* Python 3.x is used
* CMake maintains the whole build process at the top.
* devel/binutils is no longer required.
Test Plan

Known issues at the moment:

  • Building with DEBUG requires a lot of memory (~ 32+GB). This should be verified with a better builder, see below [*]
  • If aarch64 builds is not known, yet and therefor some more tweaks might be required.

Building:

  • Poudriere -> OK (12.3-, 12.4- 13.1-RELEASE, 14.0-CURRENT@356724fc931 amd64 i386 with ALSA/PULSE/SNDIO and with/-out DEBUG [*])

Runtime:

  • Tests with following consumers (needs to be ported/updated, once qt6-webengine lands) looked fine so far:
    • deskutils/calibre (via py-qt6-webengine, kudos to @madpilot)
    • editors/ghostwriter (kudos to @madpilot)
    • www/dooble-qt6

Following features were tested with dooble-qt6:

  • Webcam
  • Working multiple tabs, running videos in background tab
  • Running WebGL via hardware acceleration

Other stuff:

  • With FreeBSD 12.4 and later llvm-strip is used whereas FreeBSD 12.3 uses strip. The latter one doesn't have the --keep-section parameter.
  • Building with DEBUG on FreeBSD 12.3 fails as the debug code requires a more recent version of clang.

Diff Detail

Lint
Lint Skipped
Unit
Tests Skipped

Event Timeline

kai requested review of this revision.Dec 22 2022, 7:45 PM

Macro thankyou_dog:

Mk/Uses/qt.mk
370

%%QT_LIBDIR%%/libQt5WebEngineCore.so is in qt5-webengine/pkg-plist -- so this could be unified to just:

qt-webengine_LIB=	libQt${_QT_LIBVER}WebEngineCore.so

without the if, no?

www/qt6-webengine/Makefile
53

^ QT_BINARY is not needed, I think

117

Does

BINARY_ALIAS=  python=${PYTHON_CMD}

not work`

Looks like the patch is missing a required change to Mk/Uses/qt-dist.mk.

webengine should be moved from _QT5_DISTS to _COMMON_DISTS I think.

Otherwise I get a failure:

[00:00:06] Ignoring www/qt6-webengine | qt6-webengine-6.4.1: Unsupported qt-dist webengine for qt:6

Further testing ongoing.

This revision now requires changes to proceed.Dec 28 2022, 3:50 PM

Hi again,

I've tested this in poudriere, it worked like a charm on head, but I got a failure for 13.1 and 12.4:

--- src/core/CMakeFiles/QtWebEngineCore_Release_amd64.dir/all ---
[  0% 31/30420] AR obj/base/third_party/symbolize/libsymbolize.a
FAILED: obj/base/third_party/symbolize/libsymbolize.a 
rm -f obj/base/third_party/symbolize/libsymbolize.a && "/usr/bin/llvm-ar" -T -r -c -s -D obj/base/third_party/symbolize/libsymbolize.a @"obj/base/third_party/symbolize/libsymbolize.a.rsp"
/usr/bin/llvm-ar: error: @obj/base/third_party/symbolize/libsymbolize.a.rsp: No such file or directory
--- src/core/CMakeFiles/convert_dict_Release_amd64.dir/all ---

P.S. obviously after fixing locally the qt-dist.mk issue I already explained

  • Add missing Mk/Uses/qt-dist.mk
  • Remove conditional in Mk/Uses/qt.mk (build works with www/qt5-webengine and www/qt6-webengine)
  • Remove custom pre-configure target and use BINARY_ALIAS instead.
kai marked 3 inline comments as done.Dec 29 2022, 2:11 PM

Looks like the patch is missing a required change to Mk/Uses/qt-dist.mk.

webengine should be moved from _QT5_DISTS to _COMMON_DISTS I think.

Oops, good catch! This should be fixed now.

I've tested this in poudriere, it worked like a charm on head, but I got a failure for 13.1 and 12.4:

--- src/core/CMakeFiles/QtWebEngineCore_Release_amd64.dir/all ---
[  0% 31/30420] AR obj/base/third_party/symbolize/libsymbolize.a
FAILED: obj/base/third_party/symbolize/libsymbolize.a 
rm -f obj/base/third_party/symbolize/libsymbolize.a && "/usr/bin/llvm-ar" -T -r -c -s -D obj/base/third_party/symbolize/libsymbolize.a @"obj/base/third_party/symbolize/libsymbolize.a.rsp"
/usr/bin/llvm-ar: error: @obj/base/third_party/symbolize/libsymbolize.a.rsp: No such file or directory
--- src/core/CMakeFiles/convert_dict_Release_amd64.dir/all ---

P.S. obviously after fixing locally the qt-dist.mk issue I already explained

That's the second issue listed at Known issues at the moment:

Sometime the build fails in an early stage if using poudriere testport or poudriere bulk -J1. But it seems to be not reproducible.

So it's a not a local problem and happens in other environments as well. I assume that the workaround for the missing convert_dict.stamp in cmake/Functions.cmake still needs some adjustements because the issue has appeared since in a sporadic manner.

As a "workaround" it should help to try another build. (But it's definitely not the solution for it).

@tcberner : The items you noted should be fixed/adapted.

In D37859#861089, @kai wrote:

I've tested this in poudriere, it worked like a charm on head, but I got a failure for 13.1 and 12.4:

--- src/core/CMakeFiles/QtWebEngineCore_Release_amd64.dir/all ---
[  0% 31/30420] AR obj/base/third_party/symbolize/libsymbolize.a
FAILED: obj/base/third_party/symbolize/libsymbolize.a 
rm -f obj/base/third_party/symbolize/libsymbolize.a && "/usr/bin/llvm-ar" -T -r -c -s -D obj/base/third_party/symbolize/libsymbolize.a @"obj/base/third_party/symbolize/libsymbolize.a.rsp"
/usr/bin/llvm-ar: error: @obj/base/third_party/symbolize/libsymbolize.a.rsp: No such file or directory
--- src/core/CMakeFiles/convert_dict_Release_amd64.dir/all ---

P.S. obviously after fixing locally the qt-dist.mk issue I already explained

That's the second issue listed at Known issues at the moment:

Sometime the build fails in an early stage if using poudriere testport or poudriere bulk -J1. But it seems to be not reproducible.

I see, sorry I failed to make the connection myself.

So it's a not a local problem and happens in other environments as well. I assume that the workaround for the missing convert_dict.stamp in cmake/Functions.cmake still needs some adjustements because the issue has appeared since in a sporadic manner.

For testing purposes I can try to build more times.

Looks like some race condition, but happening during the convert_dict part, but I know nothing about this software build system so I'm just guessing here.

Apart from this everything looks fine now.

www/qt6-webengine/Makefile
48

^ @madpilot maybe try cmake:noninja to see if when using make instead of ninja the build gets through reproducibly.

If not, maybe MAKE_JOBS_UNSAFE is required but that would hurt alot...

www/qt6-webengine/Makefile
48

^ @madpilot maybe try cmake:noninja to see if when using make instead of ninja the build gets through reproducibly.

Unluckily adding cmake:noninja made no difference.

Maybe it's just a coincidence, but in my testing the build is much more likely to fail on releases than on head.

If not, maybe MAKE_JOBS_UNSAFE is required but that would hurt alot...

That would really hurt, since it's already taking almost two hours to build here.

www/qt6-webengine/Makefile
48

^ @madpilot maybe try cmake:noninja to see if when using make instead of ninja the build gets through reproducibly.

Unluckily adding cmake:noninja made no difference.

Actually continued testing shows some improvements. I'm going on running these tests.

Maybe my first run did not include the patch to remove ninja and sent off my results

Maybe it's just a coincidence, but in my testing the build is much more likely to fail on releases than on head.

Please ignore, this was definitely just a coincidence.

www/qt6-webengine/Makefile
48

^ @madpilot maybe try cmake:noninja to see if when using make instead of ninja the build gets through reproducibly.

Unluckily adding cmake:noninja made no difference.

Actually continued testing shows some improvements. I'm going on running these tests.

Maybe my first run did not include the patch to remove ninja and sent off my results

After some more tests I confirm using cmake:noninja makes little or no difference. it keeps failing around half the time.

kai edited the summary of this revision. (Show Details)
kai edited the test plan for this revision. (Show Details)
  • Fix issue with intermittent build failures due race conditions
  • Fix packaging issue with different Qt6 versions
  • Fix build issue on i386 with compress_files.js
  • Place the unbundling of libraries to pre-configure target
  • Chase Qt6 in the ports tree by updating www/qt6-webengine to 6.4.2
www/qt6-webengine/Makefile
48

^ @madpilot maybe try cmake:noninja to see if when using make instead of ninja the build gets through reproducibly.

Unluckily adding cmake:noninja made no difference.

Actually continued testing shows some improvements. I'm going on running these tests.

Maybe my first run did not include the patch to remove ninja and sent off my results

After some more tests I confirm using cmake:noninja makes little or no difference. it keeps failing around half the time.

Indeed, cmake:noninja didn't resolve the issue. I also tried to use devel/samurai but without any luck, so it isn't an issue with the implementation of ninja or samurai. I was able to fix the issue by coercing make(1) to use only one job, but pass the available MAKE_JOBS_NUMBER to ninja.

With this approach, all targets generated by GN are built sequentially and already compiled objects are re-used during build. The builds should now run considerably faster than before as well.

Great work!

It works fine here, and I also was able to run my updated calibre port on it.

Thanks!

This revision is now accepted and ready to land.Jan 12 2023, 9:28 PM