Differential D50911
Mk: reproducible_builds: set SOURCE_DATE_EPOCH for TIMESTAMPed ports Authored by guest-svmhdvn on Jun 17 2025, 3:41 PM. Tags None Referenced Files
Subscribers
Details Continuing the work from D24586. Original summary: Pass SOURCE_DATE_EPOCH in build environnement if the timestamp is Changes since previous patch:
Currently writing some tooling around Poudriere to verify reproducible builds across the ports tree in an automated way. I've verified the following:
Diff Detail
Event TimelineComment Actions before requesting an exp-run or anything like that, have you tested on a bulk of known problematic ports ? python for example or emacs? Also part of this will only be visible at runtime, like build your python package, install them on a machine, run some python things, just importing the library is enough to trigger the issue (run as root) and check with pkg check if some files have been rebuilt Comment Actions I'll do that now, is there a list of problematic ports already known? I see the history of python and emacs, but if you have a quick list of some others that are widely used, I'll test them now.
Sorry I didn't understand this part. "just importing the library is enough to trigger the issue" -> what issue are you referring to? Comment Actions
Here is a reproducibility test of a random list of ports taken from a typical VPS installation of FreeBSD with a bunch of python, PHP, and other ports. This test ran two builds of all ports, separated by a few seconds, on two different Poudriere jails (with different hostnames). TAP version 14 ok 1 - acme.sh-3.1.1.pkg ok 2 - autoconf-2.72.pkg not ok 3 - autoconf-switch-20220527.pkg ok 4 - automake-1.17.pkg ok 5 - bash-5.3_1.pkg ok 6 - bison-3.8.2_2,1.pkg ok 7 - brotli-1.1.0,1.pkg ok 8 - ccache-3.7.12_8.pkg ok 9 - cmake-core-3.31.7.pkg ok 10 - curl-8.14.1.pkg ok 11 - cyrus-imapd310-3.10.2_1.pkg ok 12 - cyrus-sasl-2.1.28_5.pkg ok 13 - doas-6.3p12.pkg not ok 14 - docbook-1.5.pkg not ok 15 - docbook-sgml-4.5_1.pkg not ok 16 - docbook-xml-5.0_3.pkg ok 17 - docbook-xsl-1.79.1_1,1.pkg ok 18 - dokuwiki-php83-20250514a.pkg ok 19 - expat-2.7.1.pkg ok 20 - fontconfig-2.15.0_3,1.pkg ok 21 - freetype2-2.13.3.pkg ok 22 - gettext-runtime-0.23.1.pkg ok 23 - gettext-tools-0.23.1_1.pkg ok 24 - giflib-5.2.2.pkg not ok 25 - glib-2.84.1_3,2.pkg not ok 26 - glib-bootstrap-2.84.1_3,2.pkg ok 27 - gmake-4.4.1.pkg not ok 28 - gobject-introspection-1.84.0,1.pkg not ok 29 - gobject-introspection-bootstrap-1.84.0,1.pkg ok 30 - gperf-3.2.1_1.pkg ok 31 - groff-1.23.0_5.pkg not ok 32 - gsfonts-8.11_8.pkg not ok 33 - gtk-doc-1.33.2_3.pkg ok 34 - help2man-1.49.3_1.pkg ok 35 - htop-3.4.0.pkg ok 36 - icu-76.1,1.pkg ok 37 - indexinfo-0.3.1_1.pkg not ok 38 - iso8879-1986_3.pkg ok 39 - itstool-2.0.7_2.pkg ok 40 - jansson-2.14.1.pkg not ok 41 - jbigkit-2.1_3.pkg ok 42 - jpeg-turbo-3.1.1.pkg ok 43 - jsoncpp-1.9.6_1.pkg ok 44 - lerc-4.0.0.pkg ok 45 - libargon2-20190702_1.pkg ok 46 - libdeflate-1.24.pkg ok 47 - libedit-3.1.20250104,1.pkg ok 48 - libevent-2.1.12.pkg ok 49 - libffi-3.5.1.pkg ok 50 - libgcrypt-1.11.1.pkg ok 51 - libgd-2.3.3_13,1.pkg ok 52 - libgpg-error-1.55.pkg ok 53 - libical-3.0.20_1.pkg ok 54 - libiconv-1.17_1.pkg ok 55 - libidn2-2.3.8.pkg ok 56 - libinotify-20240724_2.pkg ok 57 - liblz4-1.10.0,1.pkg ok 58 - libnghttp2-1.66.0.pkg ok 59 - libpaper-1.1.28_1.pkg ok 60 - libpsl-0.21.5_2.pkg ok 61 - libsodium-1.0.19.pkg not ok 62 - libsrs2-1.0.18_4.pkg ok 63 - libssh2-1.11.1,3.pkg ok 64 - libtextstyle-0.23.1.pkg ok 65 - libtool-2.5.4.pkg ok 66 - libunistring-1.3.pkg ok 67 - libunwind-20240221_2.pkg ok 68 - libuuid-2.41.1_1.pkg ok 69 - libuv-1.51.0.pkg ok 70 - libxml2-2.14.5.pkg ok 71 - libxslt-1.1.43_1.pkg ok 72 - libyaml-0.2.5.pkg ok 73 - m4-1.4.19_1,1.pkg ok 74 - meson-1.7.0.pkg ok 75 - mpdecimal-4.0.1.pkg ok 76 - nasm-2.16.03,2.pkg ok 77 - nginx-lite-1.28.0_2,3.pkg ok 78 - ninja-1.11.1,4.pkg ok 79 - oniguruma-6.9.10.pkg ok 80 - openldap26-client-2.6.10.pkg ok 81 - opensmtpd-7.6.0,1.pkg ok 82 - p5-Locale-gettext-1.07.pkg ok 83 - p5-Locale-libintl-1.35.pkg ok 84 - p5-Term-ReadLine-Gnu-1.47.pkg ok 85 - p5-Text-Unidecode-1.30.pkg ok 86 - p5-Unicode-EastAsianWidth-12.0.pkg ok 87 - pcre2-10.45_1.pkg not ok 88 - perl5-5.40.2_2.pkg ok 89 - php83-8.3.23.pkg ok 90 - php83-ctype-8.3.23.pkg ok 91 - php83-filter-8.3.23.pkg ok 92 - php83-gd-8.3.23.pkg ok 93 - php83-iconv-8.3.23.pkg ok 94 - php83-ldap-8.3.23.pkg ok 95 - php83-mbstring-8.3.23.pkg ok 96 - php83-pdo-8.3.23.pkg ok 97 - php83-pdo_mysql-8.3.23.pkg ok 98 - php83-session-8.3.23.pkg ok 99 - php83-simplexml-8.3.23.pkg ok 100 - php83-xml-8.3.23.pkg ok 101 - php83-zlib-8.3.23.pkg not ok 102 - pkg-2.2.1.pkg ok 103 - pkgconf-2.4.3,1.pkg ok 104 - png-1.6.49.pkg not ok 105 - psutils-1.17_6.pkg ok 106 - public_suffix_list-20250626.pkg ok 107 - py311-Babel-2.17.0_1.pkg ok 108 - py311-Jinja2-3.1.6.pkg ok 109 - py311-build-1.2.2_2.pkg ok 110 - py311-calver-2025.4.17.pkg ok 111 - py311-cython-0.29.37_2.pkg ok 112 - py311-docutils-0.21.2,1.pkg not ok 113 - py311-flit-core-3.12.0.pkg ok 114 - py311-gi-docgen-2025.4.pkg ok 115 - py311-hatchling-1.27.0.pkg ok 116 - py311-installer-0.7.0.pkg ok 117 - py311-libxml2-python-2.14.5.pkg ok 118 - py311-lxml-4.9.3_2.pkg ok 119 - py311-markdown-3.8.2.pkg ok 120 - py311-markupsafe-3.0.2.pkg ok 121 - py311-packaging-25.0.pkg ok 122 - py311-pathspec-0.12.1.pkg ok 123 - py311-pluggy-1.6.0.pkg ok 124 - py311-pygments-2.19.2.pkg ok 125 - py311-pyproject-hooks-1.2.0.pkg ok 126 - py311-setuptools-63.1.0_3.pkg ok 127 - py311-setuptools-scm-8.0.4_1.pkg ok 128 - py311-smartypants-2.0.1_1.pkg ok 129 - py311-toml-0.10.2_1.pkg ok 130 - py311-trove-classifiers-2025.5.9.12.pkg ok 131 - py311-typing-extensions-4.14.1.pkg ok 132 - py311-typogrify-2.1.0.pkg ok 133 - py311-wheel-0.45.1.pkg ok 134 - py311-wheel044-0.44.0_1.pkg ok 135 - python311-3.11.13.pkg ok 136 - re2c-4.1.pkg ok 137 - readline-8.2.13_2.pkg ok 138 - rhash-1.4.4_1.pkg ok 139 - rsync-3.4.1_2.pkg ok 140 - ruby-3.3.8,1.pkg ok 141 - ruby33-gems-3.6.9.pkg ok 142 - rubygem-asciidoctor-2.0.23.pkg not ok 143 - sdocbook-xml-1.1_2,2.pkg ok 144 - socat-1.8.0.3.pkg ok 145 - sqlite3-3.50.2_1,1.pkg ok 146 - tcl86-8.6.16_2.pkg ok 147 - texinfo-7.1_11,1.pkg ok 148 - tiff-4.7.0.pkg ok 149 - tmux-3.5a_1.pkg ok 150 - tree-2.2.1.pkg ok 151 - uchardet-0.0.8_1.pkg ok 152 - unbound-1.23.0_1.pkg ok 153 - vala-0.56.18,1.pkg ok 154 - webp-1.6.0.pkg not ok 155 - xmlcatmgr-2.2_4.pkg not ok 156 - xmlcharent-0.3_2.pkg ok 157 - xxhash-0.8.3.pkg ok 158 - yelp-tools-42.1.pkg ok 159 - yelp-xsl-42.4.pkg ok 160 - zstd-1.5.7.pkg 1..160 If I did not apply this patch, all of these ports show up as not ok in the test.
Comment Actions @bapt any updates on reviewing v2 and the results of the reproducibility test of 160 packages from above? Comment Actions someone needs to understand why the not ok package are not ok, is this is related to the timestamp or not. also some packages needs to be run tested, for example install the python package on a system. As root in a python cli import anything which is installed by the package, and run pkg check to see of this pyc were regenerated. Comment Actions
Here is an updated analysis with a fresh build as of today. I have manually looked into each failing case to determine the reason for the irreproducibility using diffoscope. Please see below and let me know if you want me to send you the diffoscope results.
I can do this run test soon. However, in the interrim, I don't think this run testing blocks the acceptance of this patch. It would be good to make incremental progress towards getting ports to build reproducibly, rather than waiting on the results of verification tests such as the python import test. I do agree that the pyc files might be regenerated at runtime, but in the meantime, can we merge this patch so that we can start publishing weekly/monthly status numbers on the percentage of ports that are reproducible across the Ports collection? TAP version 14 ok 1 - acme.sh-3.1.1.pkg ok 2 - autoconf-2.72.pkg not ok 3 - autoconf-switch-20220527.pkg (mtimes differ in .pkg-content tar listing) ok 4 - automake-1.17.pkg ok 5 - bash-5.3.3_1.pkg ok 6 - bison-3.8.2_2,1.pkg ok 7 - brotli-1.1.0,1.pkg ok 8 - cmake-core-3.31.7.pkg ok 9 - curl-8.15.0.pkg ok 10 - cyrus-imapd310-3.10.2_2.pkg ok 11 - cyrus-sasl-2.1.28_5.pkg ok 12 - doas-6.3p12.pkg not ok 13 - docbook-1.5.pkg (mtimes differ in .pkg-content tar listing) not ok 14 - docbook-sgml-4.5_1.pkg (mtimes differ in .pkg-content tar listing) not ok 15 - docbook-xml-5.0_3.pkg (mtimes differ in .pkg-content tar listing) ok 16 - docbook-xsl-1.79.1_1,1.pkg ok 17 - dokuwiki-php83-20250514a.pkg ok 18 - expat-2.7.1.pkg ok 19 - fontconfig-2.15.0_3,1.pkg ok 20 - freetype2-2.13.3.pkg ok 21 - gettext-runtime-0.23.1.pkg ok 22 - gettext-tools-0.23.1_1.pkg ok 23 - giflib-5.2.2.pkg not ok 24 - glib-2.84.1_3,2.pkg (moddate differs in bundled __pycache__ pyc files) not ok 25 - glib-bootstrap-2.84.1_3,2.pkg (moddate differs in bundled __pycache__ pyc files) ok 26 - gmake-4.4.1.pkg not ok 27 - gobject-introspection-1.84.0,1.pkg (moddate differs in bundled __pycache__ pyc files) not ok 28 - gobject-introspection-bootstrap-1.84.0,1.pkg (moddate differs in bundled __pycache__ pyc files) ok 29 - gperf-3.2.1_1.pkg ok 30 - groff-1.23.0_5.pkg not ok 31 - gsfonts-8.11_8.pkg (mtimes differ in .pkg-content tar listing) not ok 32 - gtk-doc-1.33.2_3.pkg (moddate differs in bundled __pycache__ pyc files) ok 33 - help2man-1.49.3_1.pkg ok 34 - htop-3.4.0.pkg ok 35 - icu-76.1,1.pkg ok 36 - indexinfo-0.3.1_1.pkg not ok 37 - iso8879-1986_3.pkg (mtimes differ in .pkg-content tar listing) ok 38 - itstool-2.0.7_2.pkg ok 39 - jansson-2.14.1.pkg not ok 40 - jbigkit-2.1_3.pkg (mtimes differ in .pkg-content tar listing) ok 41 - jpeg-turbo-3.1.1.pkg ok 42 - jsoncpp-1.9.6_1.pkg ok 43 - lerc-4.0.0.pkg ok 44 - libargon2-20190702_1.pkg ok 45 - libdeflate-1.24.pkg ok 46 - libedit-3.1.20250104,1.pkg ok 47 - libevent-2.1.12.pkg ok 48 - libffi-3.5.1.pkg ok 49 - libgcrypt-1.11.2.pkg ok 50 - libgd-2.3.3_13,1.pkg ok 51 - libgpg-error-1.55.pkg ok 52 - libical-3.0.20_1.pkg ok 53 - libiconv-1.17_1.pkg ok 54 - libidn2-2.3.8.pkg ok 55 - libinotify-20240724_3.pkg ok 56 - liblz4-1.10.0,1.pkg ok 57 - libnghttp2-1.66.0.pkg ok 58 - libpaper-1.1.28_1.pkg ok 59 - libpsl-0.21.5_2.pkg ok 60 - libsodium-1.0.19.pkg not ok 61 - libsrs2-1.0.18_4.pkg (mtimes differ in .pkg-content tar listing) ok 62 - libssh2-1.11.1,3.pkg ok 63 - libtextstyle-0.23.1.pkg ok 64 - libtool-2.5.4_1.pkg ok 65 - libunistring-1.3.pkg ok 66 - libunwind-20240221_2.pkg ok 67 - libuuid-2.41.1_1.pkg ok 68 - libuv-1.51.0.pkg ok 69 - libxml2-2.14.5.pkg ok 70 - libxslt-1.1.43_1.pkg ok 71 - libyaml-0.2.5.pkg ok 72 - m4-1.4.19_1,1.pkg ok 73 - meson-1.7.0.pkg ok 74 - mpdecimal-4.0.1.pkg ok 75 - nasm-2.16.03,2.pkg ok 76 - nginx-lite-1.28.0_3,3.pkg ok 77 - ninja-1.11.1,4.pkg ok 78 - oniguruma-6.9.10.pkg ok 79 - openldap26-client-2.6.10.pkg ok 80 - opensmtpd-7.6.0,1.pkg ok 81 - p5-Locale-gettext-1.07.pkg ok 82 - p5-Locale-libintl-1.35.pkg ok 83 - p5-Term-ReadLine-Gnu-1.47.pkg ok 84 - p5-Text-Unidecode-1.30.pkg ok 85 - p5-Unicode-EastAsianWidth-12.0.pkg ok 86 - pcre2-10.45_1.pkg not ok 87 - perl5-5.42.0_1.pkg (embedded hostname and timestamps in bundled files) not ok 88 - php83-8.3.24.pkg (embedded hostname and timestamps in bundled files) ok 89 - php83-ctype-8.3.24.pkg ok 90 - php83-filter-8.3.24.pkg ok 91 - php83-gd-8.3.24.pkg ok 92 - php83-iconv-8.3.24.pkg ok 93 - php83-ldap-8.3.24.pkg ok 94 - php83-mbstring-8.3.24.pkg ok 95 - php83-pdo-8.3.24.pkg ok 96 - php83-pdo_mysql-8.3.24.pkg ok 97 - php83-session-8.3.24.pkg ok 98 - php83-simplexml-8.3.24.pkg ok 99 - php83-xml-8.3.24.pkg ok 100 - php83-zlib-8.3.24.pkg not ok 101 - pkg-2.2.2.pkg (libpkg.a not reproducible due to embedded timestamp) ok 102 - pkgconf-2.4.3,1.pkg ok 103 - png-1.6.49.pkg not ok 104 - psutils-1.17_6.pkg (mtimes differ in .pkg-content tar listing) ok 105 - public_suffix_list-20250626.pkg ok 106 - py311-Babel-2.17.0_1.pkg ok 107 - py311-Jinja2-3.1.6.pkg ok 108 - py311-build-1.2.2_2.pkg ok 109 - py311-calver-2025.4.17.pkg ok 110 - py311-cython3-3.1.2.pkg ok 111 - py311-docutils-0.21.2,1.pkg not ok 112 - py311-flit-core-3.12.0.pkg (moddate differs in bundled __pycache__ pyc files) ok 113 - py311-gi-docgen-2025.4.pkg ok 114 - py311-hatchling-1.27.0.pkg ok 115 - py311-installer-0.7.0.pkg ok 116 - py311-libxml2-python-2.14.5.pkg ok 117 - py311-lxml5-5.4.0_2.pkg ok 118 - py311-markdown-3.8.2.pkg ok 119 - py311-markupsafe-3.0.2.pkg ok 120 - py311-packaging-25.0.pkg ok 121 - py311-pathspec-0.12.1.pkg ok 122 - py311-pluggy-1.6.0.pkg ok 123 - py311-pygments-2.19.2.pkg ok 124 - py311-pyproject-hooks-1.2.0.pkg ok 125 - py311-setuptools-63.1.0_3.pkg ok 126 - py311-setuptools-scm-8.0.4_1.pkg ok 127 - py311-smartypants-2.0.2.pkg ok 128 - py311-toml-0.10.2_1.pkg ok 129 - py311-trove-classifiers-2025.5.9.12.pkg ok 130 - py311-typing-extensions-4.14.1.pkg ok 131 - py311-typogrify-2.1.0.pkg ok 132 - py311-wheel-0.45.1.pkg ok 133 - py311-wheel044-0.44.0_1.pkg ok 134 - python311-3.11.13_1.pkg ok 135 - re2c-4.1.pkg ok 136 - readline-8.2.13_2.pkg ok 137 - rhash-1.4.4_1.pkg ok 138 - rsync-3.4.1_3.pkg ok 139 - ruby-3.3.9,1.pkg ok 140 - ruby33-gems-3.7.1.pkg ok 141 - rubygem-asciidoctor-2.0.23.pkg not ok 142 - sdocbook-xml-1.1_2,2.pkg (mtimes differ in .pkg-content tar listing) ok 143 - socat-1.8.0.3.pkg ok 144 - sqlite3-3.50.2_1,1.pkg not ok 145 - tcl86-8.6.17.pkg (embedded hostname and timestamps in bundled files) ok 146 - texinfo-7.1_12,1.pkg ok 147 - tiff-4.7.0.pkg ok 148 - tmux-3.5a_1.pkg ok 149 - tree-2.2.1.pkg ok 150 - uchardet-0.0.8_1.pkg ok 151 - unbound-1.23.1.pkg ok 152 - vala-0.56.18,1.pkg ok 153 - webp-1.6.0.pkg not ok 154 - xmlcatmgr-2.2_4.pkg (mtimes differ in .pkg-content tar listing) not ok 155 - xmlcharent-0.3_2.pkg (mtimes differ in .pkg-content tar listing) ok 156 - xxhash-0.8.3.pkg ok 157 - yelp-tools-42.1_1.pkg ok 158 - yelp-xsl-42.4.pkg ok 159 - zstd-1.5.7.pkg 1..159 Comment Actions @bapt any info on the above? Can we get this merged first (since it would be harmless) so that we can start publishing reproducible builds stats for the Ports collection? The python ports can be looked into after that. Btw, the ports that are root-caused with "(mtimes differ in .pkg-content tar listing)" are due to them not having a distinfo entry, which is a trivial fix. Comment Actions It does as it will break something working today if it does not work. Comment Actions Looks like https://github.com/freebsd/pkg/pull/2577 is needed first after https://github.com/freebsd/pkg/commit/08bf9b3d79b127b127ecf597286ba14c016af8c4 was merged.
I'll do the necessary testing, report back, then file a bugzilla bug to request an exp-run with this patch. Comment Actions
As a good comprehensive example to test: I took the py311-gi-docgen pkg. This python package depends on the following packages: root@rb1:/tmp # pkg info -d py311-gi-docgen py311-gi-docgen-2025.5: python311-3.11.14 py311-typogrify-2.1.0 py311-toml-0.10.2_1 py311-smartypants-2.0.2 py311-pygments-2.19.2 py311-packaging-25.0 py311-markupsafe-3.0.3 py311-markdown-3.10 py311-Jinja2-3.1.6 To test this, I built all of these packages using the patch from this review and this command: doas poudriere testport -NNi -j rb1 textproc/py-gi-docgen I did a before/after test of all the checksums of all installed files by these packages (including their generated .pyc files) with this command: root@rb1:/tmp # pkg info -d py311-gi-docgen | grep -v ':$' | xargs pkg check -s Checking python311-3.11.14: 100% Checking py311-typogrify-2.1.0: 100% Checking py311-toml-0.10.2_1: 100% Checking py311-smartypants-2.0.2: 100% Checking py311-pygments-2.19.2: 100% Checking py311-packaging-25.0: 100% Checking py311-markupsafe-3.0.3: 100% Checking py311-markdown-3.10: 100% Checking py311-Jinja2-3.1.6: 100% I tested the checksums before and after running this command to trigger importing all its dependencies: gi-docgen index Gdk-4.0.gir (using a sample Gdk-4.0.gir file I took from online). To sanity check my findings, I also inserted a space character in a random .py and .pyc file installed by one of its dependencies py311-markdown, then tested with the same checksum command. This was an example output from a .py file with an extra space character: root@rb1:/usr/ports/textproc/py-gi-docgen # pkg check -s py311-markdown Checking py311-markdown: 0% py311-markdown-3.10: checksum mismatch for /usr/local/lib/python3.11/site-packages/markdown/extensions/admonition.py Checking py311-markdown: 100% Is this testing sufficient, since it is mostly generalizable for all python programs like this? I think we are good to do an exp-run of this patch, since the pkg patch has been merged and is out in 2.5.1. I will file a bug for the exp-run request shortly. Comment Actions Could you make it so that lang/python27 + the ports using python27 do not use this feature? | ||||||||||||||||||||||||||||||||||