LGTM, but other than the relocation; I see no difference. Tested with

In D34739#895778, @vishwin wrote:

In D34739#895772, @bapt wrote:

As far as I know from the time when I participated to the events about reproducible build the issues has been fixed, at least instead of claiming it is not deterministic, can you state what makes it non deterministic?

It's still not, never has and probably never will. While PEP-552 adds an alternative to mtime comparisons (a computed hash), the use of a magic number that can vary between environments (in addition to interpreter versions) already makes it non-deterministic. Further, PEP-552 acknowledges other facets of non-determinism, particularly inconsistent marshalling of set objects (such an implementation is necessary by default for security reasons).

Increased file sizes in the upcoming CPython 3.12 is also concerning.

I also wonder why we don't install a 3.7+ compatible Python script and call that from the lua trigger. It would be far more useful (as in count of available developers) to write this in Python and not Lua, and we'd get proper debugging and logging libs for free. Lua is a bit minimalistic for the rather complex task at hand.

As a more general comment, I removed some immature code. I am not too troubled about *how* we solve it, but we need robust code that properly traps errors, properly reports them, and is fully tested, and DOES NOT GET COMMITTED BEFORE it's properly tested and approved by all. Python isn't your average leaf port sandbox in the playground, but high-profile stuff. And we surely should not comment anything in the next few days (hours) before 2023Q2 has branched. If something is mature and does not break semantics of other ports, we can still MFH to quarterly later, or just really clean up for 2023Q3 before July.

In D34739#895719, @vishwin wrote:

In D34739#895706, @bapt wrote:

To be honnest I am not convinced we should pursue in that direction, only debian seems to be doing something like that and they use a python script to compile and cleanup called by a trigger. All other OSes I have checked are not at all doing anything in that direction.

Because nearly nobody else has a trigger functionality like we do.

In D39261#893754, @matthew wrote:

I believe there was a recent, possibly contraversial, change to generate .pyc files dynamically at pkg install time using a pkg script, rather than including those in the main package. This apparently parallels the way python wheels work, and has a number of advantages.

One side effect has been that all of the pycache directories now show up as "checksum mismatch" in the daily security e-mails.

In D39004#888511, @vishwin wrote:

Doing it this way, when building meson-python on a non-default Python, fails because literally bin/meson is expected, not bin/meson-${PYTHON_VER}.

Don't even think about un-flavouring everything, near future consumers need to use this with all supported Python.

@vishwin please fix this up and after that also D34739, before working on any new stuff.

@jbeich wrote to the mailing lists:

this isn't working properly. Reopening

Also, there is practically no error checking and handling. This seems inappropriate for code that is applied to high-profile ports with lots of users.

we cannot have duplicated code all around. This is unmaintainable, error-prone, and Charlie has not given any non-refuted reason that would justify code duplication.

@vishwin remove me again from reviewers and that will have grave consequences.

In D34739#883876, @vishwin wrote:

In D34739#883869, @mandree wrote:

So, the trigger is redundantly duplicated across all Python versions. This is bad style and error-prone.

Please refactor this to a common script (either we keep it in Tools/ or else in a separate port), and just run it from the trigger (which has the port-specific %%PYTHON_*%% variables, with the necessary arguments.
That also makes it amenable to separate testing and maintenance independent of the Python package.

No.

The initial initial pass had this as a separate port that the Python ports themselves depended on. It was discarded for a couple reasons. Bytecode is a CPython implementation detail and thus only applicable for it; no other Python implementation employs bytecode. Also, every CPython version differs somewhat, and even for compileall, 3.9 gained a flag that allows for one invocation instead of three, which also shortens the trigger. Thus there is no technical justification to separating this from the CPython ports themselves, however redundant things may appear.

It is not in our best interest to have even a relatively minor implementation detail discourage someone else from exploring porting other Python implementations here. For instance, PyPy has been a wishlist item for years.

So, the trigger is redundantly duplicated across all Python versions. This is bad style and error-prone.

In D34739#883579, @vishwin wrote:

now that we apparently have to appease non-fatal errors not part of the build, as if package builders are sentient beings

In D34739#883400, @vishwin wrote:

For those who've lived under a rock, I'm not under mentorship.

reopening

@tcberner please make sure that and how your mentees request -exp runs for major changes. Let me know if you need a hand.

In D34739#882066, @antoine wrote:

py39-sphinx package seems to exhibit some bug in the trigger

pkg-static: Failed to execute lua trigger: attempt to index a nil value
...
pkg-static: Failed to execute lua trigger: [string "function cleanup(directory)..."]:21: attempt to get length of a nil value (local 'res')

Note this introduces an extra PATH search for systems that do NOT yet have nproc so perhaps we need to add some OSVERSION switch here.