Retire dialog

[des]

Summary

This is the last remaining piece of GPL software in the base system.
The installer transitioned to bsddialog four years ago, and the last
remaining dialog consumer, dpv, was turned off more than two years ago.
Retire dpv, libdpv, libfigpar (used only by dpv), and dialog itself.

[emaste]

Why cdialog and not libdialog in the message?

[des]

In D55424#1269013, @emaste wrote:

Why cdialog and not libdialog in the message?

That's the real name of the software package from which we got libdialog and the dialog utility.

[emaste]

As far as I can tell Thomas E. Dickey just calls it dialog. cdialog appears in the "Early History" section in https://invisible-island.net/dialog/#history_details, in our ports tree, and in some tooling in the package to support renaming.

Ah, from the readme,

-(1999-12-25)-------------------------------------------------------------------
This version of dialog, formerly known as cdialog is based on the Debian
package for dialog 0.9a (see CHANGES for recent modifications)

[des]

The port is still called cdialog.

[emaste]

Ok, but we're removing it from the base system, not the port.

[jhb]

I also don't think anyone would know what "cdialog" is, but we all know what dialog is. Do we want to move dpv and friends to ports first? Maybe Cy can create new git repos for dpv + libdpv + libfigpar and a port to install them?

[des]

In D55424#1269838, @jhb wrote:

Do we want to move dpv and friends to ports first?

I'm waiting to hear back from @dteske before I do anything.

[des]

.github/CODEOWNERS

65 ↗

(On Diff #172403)

Note, I was never the maintainer of libfigpar. I have no idea why I'm listed as such here.

[des]

rebase

[des]

rebase

[des]

@dteske have you given this any further thought?

[dteske]

In D55424#1327904, @des wrote:

@dteske have you given this any further thought?

Still working on shipping the ACPI patches that I developed during BSDCan. I need a couple more days and can then have a look at this

[dteske]

I obviously do not want to see dpv, libdpv, and libfigpar removed from base. I would like to update them.

[des]

rebase

[des]

In D55424#1327922, @dteske wrote:

I obviously do not want to see dpv, libdpv, and libfigpar removed from base. I would like to update them.

To what end? Nothing in base uses them and they've been disconnected since 2023. They're gone from 14 and 15 already. We offered to move them to ports months ago.

[dteske]

In D55424#1327936, @des wrote:

In D55424#1327922, @dteske wrote:

I obviously do not want to see dpv, libdpv, and libfigpar removed from base. I would like to update them.

To what end? Nothing in base uses them and they've been disconnected since 2023. They're gone from 14 and 15 already. We offered to move them to ports months ago.

to be reconnected to:

1. bsdinstall for dist-based installations
2. bsdconfig packages
3. for utilization by humans that use the operating system

[des]

In D55424#1327941, @dteske wrote:

In D55424#1327936, @des wrote:

To what end?

1. bsdinstall for dist-based installations
2. bsdconfig packages
3. for utilization by humans that use the operating system

They've already been replaced in bsdinstall and bsdconfig, and “humans that use the operating system“ can install them from ports after we convert them.

[dteske]

In D55424#1327945, @des wrote:

In D55424#1327941, @dteske wrote:

In D55424#1327936, @des wrote:

To what end?

1. bsdinstall for dist-based installations
2. bsdconfig packages
3. for utilization by humans that use the operating system

They've already been replaced in bsdinstall and bsdconfig, and “humans that use the operating system“ can install them from ports after we convert them.

In bsdconfig, you would be referring to:

https://cgit.freebsd.org/src/commit/usr.sbin/bsdconfig/share/packages/index.subr?id=6d3c0798ccbe7f6d3b677eed31206efc1227be50

In bsdinstall, you would be referring to:

https://cgit.freebsd.org/src/commit/usr.sbin/bsdinstall/distextract/distextract.c?id=18f20d5d967ae790f121963e1fcee68d729a529e

I used the term "reconnect" and you used the term "replaced", so let's look at what "replaced" means:

1. In bsdconfig, a gauge was replaced with no gauge
2. In bsdinstall, a gauge was replaced with what? a different gauge? Is it better? Is it worse? I don't know at this time.

The review for the latter change in bsdinstall is very low signal:

https://reviews.freebsd.org/D33581

There is an ASCII drawing on the above review, but it doesn't explain whether that is a picture of dpv or a picture of bsddialog.

To say that the entire process was tersely described is an understatement.

I would be willing to support moving dpv to ports if these commits/reviews actually had a conversation that talked about value-add versus value-loss.

My main goal here is to avoid value-loss.

Does bsddialog have an interface that is similar to dpv?

[des]

In D55424#1327980, @dteske wrote:

Does bsddialog have an interface that is similar to dpv?

Yes, bsddialog_progressview(). It's undocumented but it was developed specifically to replace libdpv in the installer.

[dteske]

In D55424#1327998, @des wrote:

In D55424#1327980, @dteske wrote:

Does bsddialog have an interface that is similar to dpv?

Yes, bsddialog_progressview(). It's undocumented but it was developed specifically to replace libdpv in the installer.

That's great to hear that it exists. Not great to hear it is undocumented.

The missing element here is that "bsdconfig packages" which formerly used dpv for showing progress on index loading was eventually going to integrate libdpv with libpkg to display progress on packages. Furthermore, it was going to be a pet project of mine to develop the World's first parallel package installer.

I have not evaluated the ability of bsddialog to fill that possibility. However, if the maintainer of bsddialog is OK with me adding missing functionality to bridge that gap when we get to it, then I am fine moving dpv to ports.

[des]

In D55424#1328007, @dteske wrote:

I have not evaluated the ability of bsddialog to fill that possibility. However, if the maintainer of bsddialog is OK with me adding missing functionality to bridge that gap when we get to it, then I am fine moving dpv to ports.

I very much doubt @asiciliano will mind

[dteske]

So, the question begs to be asked:

How does one construct a port to install software that has no home? Does that fall on me to rip it out an old base and port it to GitHub without the benefit of the FreeBSD build infrastructure?

And then I guess it also will fall on me to write the port?

[des]

In D55424#1328036, @dteske wrote:

How does one construct a port to install software that has no home? Does that fall on me to rip it out an old base and port it to GitHub without the benefit of the FreeBSD build infrastructure?

We extract the source with complete history into a separate repository under the FreeBSD organization either on GitHub or on Codeberg, unless you prefer to host it somewhere else. We'll have to replace some of the missing build glue, but that's easily done. See for instance OPIE.

And then I guess it also will fall on me to write the port?

I can write the port (or ports, if you want separate ports for each component) for you.

[dteske]

In D55424#1328049, @des wrote:

In D55424#1328036, @dteske wrote:

How does one construct a port to install software that has no home? Does that fall on me to rip it out an old base and port it to GitHub without the benefit of the FreeBSD build infrastructure?

We extract the source with complete history into a separate repository under the FreeBSD organization either on GitHub or on Codeberg, unless you prefer to host it somewhere else. We'll have to replace some of the missing build glue, but that's easily done. See for instance OPIE.

That sounds wonderful!

And then I guess it also will fall on me to write the port?

I can write the port (or ports, if you want separate ports for each component) for you.

I would greatly appreciate that.

With that all planned-out, I think that's all my concerns. With that, I'm happy to approve this review.

[dteske]

In D55424#1328049, @des wrote:

or ports, if you want separate ports for each component

Ideally, 3 ports: libfigpar, libdpv, dpv

[des]

Do you prefer GitHub or Codeberg, and what is your login there? Or would you prefer to host the repository yourself somewhere else?

[dteske]

In D55424#1328125, @des wrote:

Do you prefer GitHub or Codeberg, and what is your login there? Or would you prefer to host the repository yourself somewhere else?

Let's go with GitHub -- my login is freebsdfrau there.

[des]

In D55424#1328131, @dteske wrote:

In D55424#1328125, @des wrote:

Do you prefer GitHub or Codeberg, and what is your login there? Or would you prefer to host the repository yourself somewhere else?

Let's go with GitHub -- my login is freebsdfrau there.

@bapt can you please create freebsd/dpv on GitHub with write access for freebsdfrau and dag-erling

[asiciliano]

Hello,

My goal 4-5 years ago, when I started implementing bsddialog, was to provide a command-line interface that was almost fully compatible with LGPL dialog. The main reason was that most of the existing components already use dialog(1), so in many cases I could simply replace dialog with bsddialog. The work was quite hard and tedious, as anyone can see just by looking at the dialog(1) command-line options in the manual page. Replacing dialog with bsddialog in bsdconfig alone required several months, and that was by far the most difficult task.

For the library, the situation is different. I do not think it is feasible for me to implement the dialog(3) API; libbsddialog is fundamentally different from libdialog. Instead, I completely rewrote the front-end of sade and the other utilities (which are less complex than the partitioner). I do not have a strong opinion about whether BASE should contain dialog or bsddialog. If someone needs the dialog(3) API, that is perfectly fine with me: bsddialog could be removed and dialog re-enabled, and I would be happy to help with the reverts. I wrote a message about this on a mailing list a few years ago.

Regarding dpv(3), I found it simpler to copy the code from bsddialog_mixedgauge() in libbsddialog and add support for colors, a callback function, and extra text displayed below the screen. The resulting function is bsddialog_progressview(). It is intentionally undocumented because it is too specific for the general purpose of the library and is only used by bsdinstall. I have always been undecided about whether to keep that function or remove it and instead add color support to the regular mixedgauge through bsddialog_theme.h, as is done for the other dialogs. I will probably choose the second option, mainly for accessibility reasons and to better support users with visual impairments.

In any case, if dpv(1)/(3) is considered important for BASE, that is also fine with me. I can try to write compatibility wrappers that emulate the necessary dialog(3) behavior in order to restore dpv support.
A refactor and partial rewrite of bsddialog is already planned, and I mentioned this in an email some time ago. At this point, bsddialog is no longer easy to maintain or extend, and the TODO list of requested features has grown considerably over the years. The refactoring is expected to start in a few months and will likely take several months to complete. A significant part of the implementation will be rewritten in Lua, and, in addition to the existing C API, a native Lua API will also be provided. The goal is to make the project much easier to maintain and to enable new features to be added more quickly in the future.
Of course, I will also add an item to the TODO list to try to reintroduce dpv into BASE in the future.

Again, if I ultimately cannot provide the required compatibility, I am completely fine with removing bsddialog and re-enabling the original LGPL dialog in BASE.

[des]

In D55424#1328149, @asiciliano wrote:

Again, if I ultimately cannot provide the required compatibility, I am completely fine with removing bsddialog and re-enabling the original LGPL dialog in BASE.

No, Alfonso, you have completely misread the situation.

[dteske]

@asiciliano I don't think anyone wants to see LGPL re-infect the base.

dpv(1)/dpv(3) unfortunately got labeled as "only this" or "only that" despite the many years it took to develop to be a general purpose solution that was bred for more than it was tasked-with in base.

Few probably know about dpv(1)'s abilities below:

1. It supports Xdialog
2. It supports indefinite progress (e.g., "yes | dpv -l -" == display how many lines are coming off of "yes")
3. It supports progress-bar paging system (1000s of progress bars, showing only N-at-a-time)
4. It supports reading FIFO's or command-output

You should have a look at the EXAMPLES section of the man-page:

https://man.freebsd.org/cgi/man.cgi?query=dpv&apropos=0&sektion=0&manpath=FreeBSD+12.4-RELEASE&format=html

I absolutely loved using it for lots of things. I even built a tool called "unfurl" that unpacks anything with a progress bar. Link below:

https://github.com/freebsdfrau/FrauBSD/blob/master/unfurl/unfurl

Link to handled file types (bzip2, gzip, iso, tar, tar/bzip2, tar/gzip, tar/xz, xz):

https://github.com/freebsdfrau/FrauBSD/blob/e9db0fb244afd9a681e5648e6cde3e04418517e7/unfurl/unfurl#L479-L487

Basically any time an I/O operation took longer than 10s, I would naturally wrap it with dpv (either on-the-fly or in a script) so that I didn't have to suffer not-knowing (a) whether it was progressing or (b) how fast it was progressing.

The nature of dpv was -- like sysutils/pv -- not just to slap a progress bar in the TUI (or GUI with Xdialog / Zenity) but to also provide throughput information.

But to understand why dpv was written is to understand that I was the last maintainer of sysinstall.

ASIDE: I endeavored to make sure that when bsdinstall dropped all of sysinstall's configuration abilities, that we had something (anything) that could provide people with the missing configuration abilities of sysinstall while while bsdinstall focused largely on just the install bits -- decoupling install from configuration.

However, when bsdinstall had finally landed, it was not quite up to my standards with respect to distextract because sysinstall displayed throughput metrics on how fast distributions were being written to disk, and so naturally, I put dpv to task in distextract so that we did not lose what was deemed critical data (we ultimately wanted to retain the responsive feature of displaying how fast bytes were being slung onto the formatted system).

I haven't yet had a chance to tease apart how well bsddialog's gauge actually fills the needs that dpv solved.

I would really like to see dpv back in base sometime. It should be noted that the throughput ticker at the bottom of the screen was not rendered by dialog but by dpv. So that would not require modification of bsddialog, and arguably is not the correct place for that.

It is rather unfortunate that the licensing issue of LGPL being scuttled from base forced the removal of dpv.

EDIT: With respect to pkgbase ... dpv -- which was actually written to integrate with pkg -- dpv would be the answer to displaying progress on package installation of pkgbase too. However, at this time, I think it's better to move it aside as a port and then bring it back in once it is properly working with bsddialog. I think we should coordinate, so that we are not working against each other on a moving target since you mention that there is an upcoming refactor. At this time, I don't actually see any proper replacement or substitute for dpv in base from 13.x onward since we threw LGPL out of the base (and we are NOT about to bring dialog back into base since it is the absolute last piece of GPL software in base holding us back from being wholly BSD-licensed in base)

[des]

Here's a preview of the conversion: https://git.des.dev/des/dpv

As soon as you give me the go-ahead, I will push this to its intended location.

[dteske]

In D55424#1328418, @des wrote:

Here's a preview of the conversion: https://git.des.dev/des/dpv

As soon as you give me the go-ahead, I will push this to its intended location.

Looks good. I appreciate that the history is maintained