Page MenuHomeFreeBSD

Emacs infrastructure: Overhaul
Needs ReviewPublic

Authored by jrm on Fri, Mar 20, 8:45 PM.
Tags
None
Referenced Files
F151070917: D56001.diff
Sun, Apr 5, 7:07 PM
Unknown Object (File)
Sat, Apr 4, 9:28 AM
Unknown Object (File)
Sat, Apr 4, 8:50 AM
Unknown Object (File)
Fri, Apr 3, 12:49 AM
Unknown Object (File)
Thu, Apr 2, 11:39 PM
Unknown Object (File)
Thu, Apr 2, 11:36 PM
Unknown Object (File)
Wed, Apr 1, 12:44 AM
Unknown Object (File)
Wed, Apr 1, 12:27 AM

Details

Summary

This overhaul introduces four key changes to the elisp ports infrastructure:

  1. Remove support for packaged byte-compiled elisp. The primary motivation is to eliminate a large number of flavor-specific packages and to simplify the ports tree. For example, supporting byte-compiled elisp for print/pdf-tools required six packages, one for each flavor of editors/emacs and editors/emacs-devel. With over 100 elisp ports and requests for new Emacs flavors, this was unmanageable.
  1. Install configuration that integrates with Emacs's native compilation machinery, allowing elisp from ports to be compiled into the standard cache under the user's home directory. This matches the behavior of GNU ELPA packages and generally results in a faster experience. Reported speedups vary depending on the characteristics of the elisp code, but are typically in the range of 2.5 to 5 times faster compared to byte-compiled code.
  1. Perform byte compilation on the target host, but only when native compilation is unavailable. Compilation is initiated when Emacs starts, and the resulting .elc files are cached under the user's home directory.
  1. Load all autoloads files installed by FreeBSD elisp ports. This mirrors what package.el does for ELPA packages, ensuring that autoloaded functions are available without requiring users to explicitly load each package.

Key changes to individual elisp ports:

  • Stop installing .elc files. My first choice was to update the build so no .elc is ever created or put in ${STAGEDIR}, but with over 100 elisp ports, for practical purposes (and my sanity), I had to limit the time I spent on each port. In some cases, I gave up and added a simple post-install target to remove any .elc files from the stage directory. I can revisit this later.
  • Update or remove any pkg-message that instructs users to require or load components when the port includes an autoloads file. As with ELPA, autoloads files are now loaded automatically. While here, If I encountered text such as "in your ~/.emacs", I updated it to something more general like "in your Emacs configuration", since ~/.emacs is now considered legacy in favor of ~/.emacs.d/init.el or ~/.config/emacs/init.el.

Elisp port maintainers: I have build-tested everything, but if you could run-test your port(s) to confirm that everything behaves as expected, that would be helpful.

Diff Detail

Repository
R11 FreeBSD ports repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 71613
Build 68496: arc lint + arc unit

Event Timeline

jrm requested review of this revision.Fri, Mar 20, 8:45 PM
jrm created this revision.

This is still in the design stage, so I haven't:

  • made the changes needed for editors/emacs
  • made the changes needed to most USES=emacs ports

I'm testing it with my installation of editors/emacs-devel, and so far so good. I considered doing something similar for byte-compiled elisp, but I don't think that will work because the documentation for byte-compile-dest-file-function says

Note that the assumption that the source and compiled files are found in the same directory is hard-coded in various places in Emacs.

If we really wanted byte-compiled files, we would have to do something at package installation time as root, so the .elc files can be installed alongside the .el files. I'm leaning against that approach because it requires running things as root on the target system and would scatter untracked files under ${LOCALBASE}. Given those concerns, native compilation is the right path for users who care about performance.

Thoughts?

Incorporate necessary changes to editors/emacs

  • Remove an old refernce to EMACS_LIBDIR_WITH_VER in emacs.mk
  • Tweak variable order in emacs.mk

I understand the motivation for this. I don't understand auto compiling on behalf of the user.

Given those concerns, native compilation is the right path for users who care about performance.

Seems like we should just ship the elisp and let user configure emacs to native compile if they want? Which I'm pretty sure it does by default anyway.

I understand the motivation for this. I don't understand auto compiling on behalf of the user.

Given those concerns, native compilation is the right path for users who care about performance.

Seems like we should just ship the elisp and let the user configure Emacs to native compile if they want? Which I'm pretty sure it does by default anyway.

Providing natively compiled elisp by default aligns with upstream behavior, both for elisp shipped with Emacs itself and for packages installed via elpa. Elisp files installed by ports aren't native-compiled by default because of their install location. The short configuration snippet essentially tells Emacs to treat that location the same way it treats elpa-installed packages. We also include a pkg-message in both editors/emacs and editors/emacs-devel that explains how users can disable native compilation if they prefer.

Can you elaborate on your concerns for turning on native compilation by default? Maybe I'm missing something obvious. If there are strong reasons to deviate from upstream on this, now that we've fixed the flavor/package explosion issue, we could, in theory, create a new flavor for editors/emacs and editors/emacs-devel that disables native compilation.

  • Move elisp port changes to a separate reivew.
  • Only set EMACS_CMD, EMACS_MAJOR_VER, and EMACS_VER when the build argument is supplied.
jrm edited the summary of this revision. (Show Details)
jrm edited the summary of this revision. (Show Details)

Guard native-compile-async in site-start.el so it only runs in interactive sessions, not batch mode.

In D56001#1282338, @jrm wrote:

Can you elaborate on your concerns for turning on native compilation by default? Maybe I'm missing something obvious. If there are strong reasons to deviate from upstream on this, now that we've fixed the flavor/package explosion issue, we could, in theory, create a new flavor for editors/emacs and editors/emacs-devel that disables native compilation.

You're right. I've spent some more time looking at and thinking through this and it's very nice. It should simplify things quite a bit. Thank you!

  • Add support for byte compilation
  • Add variables to allow users to easily disable either byte compilation or native compilation
  • Put the variable definitions in site-start.el, and the rest of the startup code in default.el. This separation makes it easier for users to selectively disable native or byte compilation.
  • Rebase.

Load all autoloads files installed by FreeBSD elisp ports. This mirrors what package.el does for ELPA packages, ensuring that autoloaded functions are available without requiring users to explicitly load each package.

  • Regexps like \\.el$ should be anchored at the end of the string rather the end of the line (\\.el\\).
  • Move the elsip port changes from D56116 to here; it's better to make this one commit.

Elisp port maintainers: I have build-tested everything, but if you could run-test your port(s) to confirm that everything behaves as expected, that would be helpful.

These all work, thanks.

  • comms/emacs-eat
  • devel/go-mode.el
  • devel/ob-rust
  • devel/rust-mode
  • editors/consult.el
  • editors/marginalia.el
  • editors/orderless.el
  • editors/perspective.el
  • editors/vertico.el
  • textproc/ox-gfm.el
This revision is now accepted and ready to land.Fri, Apr 3, 4:05 AM
jrm edited the summary of this revision. (Show Details)
  • Fix a regexp issue in default.el.in. When I switched some regexps to be anchored at the end of string rather than end of line, I switched, e.g., "\\.elc$" to "\\.el\\" when I should have done "\\.el\\'".
This revision now requires review to proceed.Sat, Apr 4, 4:54 PM