Paths

Table of Contentst

-
Mk/
7/7
bsd.port.mk
7/7
bsd.port.subdir.mk

Differential D42131

Mk/bsd.port.{subdir}.mk: create describe-json
ClosedPublic
Actions

Authored by fernape on Oct 9 2023, 12:49 PM.

Details

Reviewers

bapt

Group Reviewers

portmgr

Commits

R11:3d9f300e01f5: Mk/bsd.port.{subdir}.mk: create describe-json

Summary

Create describe-json target to get a JSON-compliant representation of the
ports tree.

It can be invoked from a single port directory, a category directory or from the
ports tree top directory.

It supports FLAVORS. E.g. It is possible to execute the following:

cd math/qalculate-qt && make describe-json-qt6

Performance-wise it is a bit slower than a simple make describe although it
offers a more complete vision of the port in an easily parseable format.

Test Plan

Apply patch and run "make describe-json", possibly redirecting to a file. A sample of such a file generated from my ports tree can be found here

Some examples assuming the output is in the describe.json file.

Show all ports in the "accessibility" category:

jq '.accessibility'

Show ports in the "accessibility" category with a USES=iconv dependency:

jq '.accessibility.[] | select(.uses | index("iconv")) | .pkgorigin' describe.json

Show ports in the tree that Uses=kmod:

jq '.[].[] | select(.uses | index("kmod")) | .pkgorigin' describe.json

Show ports having a LIB_DEPENDS on devel/binutils:

jq '.[].[] | select(.lib_depends | map(test("devel/binutils")) | any) | .pkgorigin' describe.json

Show ports in the "database" category which has no maintainer and are marked as deprecated:

jq '.databases.[] | select(.maintainer=="ports@FreeBSD.org" and .deprecated!="") | .pkgorigin' describe.json

Diff Detail

Repository

R11 FreeBSD ports repository

Lint

No Lint Coverage

Unit

No Test Coverage

Build Status

Buildable 53899
Build 50790: arc lint + arc unit

Event Timeline

fernape requested review of this revision.Oct 9 2023, 12:49 PM

fernape created this revision.

Harbormaster completed remote builds in B53899: Diff 128450.Oct 9 2023, 12:49 PM

fernape edited the summary of this revision. (Show Details)Oct 9 2023, 1:32 PM

fernape edited the test plan for this revision. (Show Details)

fernape added reviewers: bapt, portmgr.

am I missing something, I do not see the definition of SED_ARGS ?

Remove unused SED_ARGS

Harbormaster completed remote builds in B53911: Diff 128490.Oct 10 2023, 8:00 AM

In D42131#961439, @bapt wrote:

am I missing something, I do not see the definition of SED_ARGS ?

Damn, sorry for that. I moved all substitutions from sed(1) to make(1)'s variable modifiers and embarrassingly forgot to remove the command altogether.

bapt added inline comments.Oct 13 2023, 2:06 PM

Mk/bsd.port.mk
4412	each of those line will spawn a new sh process, so this won't be efficient, you should probably inline all of this, with ; \ at the end of each line, otherwise it will ve very slow to make at the root of the ports tree a make describe-json

Address bapt@ feedback

Don't spawn a new sh process for every ECHO_CMD

While here:

Prettyfy code
Prefix private variables with underscore
Fix flavors

Harbormaster completed remote builds in B54028: Diff 128899.Oct 17 2023, 8:41 AM

fernape marked an inline comment as done.Oct 17 2023, 8:43 AM

fernape added inline comments.

Mk/bsd.port.mk
4412	One `sh` with multiple `echo` commands, thanks! While here, make the code more readable, prefix private variables with underscore and fix flavors. `make describe-json` creates JSON-compliant output from the top of the tree, a category directory or an individual port directory.

It lacks informations about the licenses ;), but the technical part is ok, and license can be added later

This revision is now accepted and ready to land.Oct 18 2023, 7:18 AM

In D42131#964377, @bapt wrote:

It lacks informations about the licenses ;), but the technical part is ok, and license can be added later

Yup! you're right. That's important information.
I just added it as a list, since some ports have multiple licenses.
This is an example for audio/picard-plugins.

...
  "categories": [
    "audio",
    "python"
  ],
  "license": [
    "GPLv2+",
    "GPLv3+",
    "MIT",
    "WTFPL"
  ],
  "deprecated": " ",
  "broken": "",
  "distversion": "2.0.20230915",
...

It generates the whole ports tree.
If you don't say otherwise, I'll go with that version.

diff --git a/Mk/bsd.port.mk b/Mk/bsd.port.mk
index fc02b52821ac..6b3522d559a0 100644
--- a/Mk/bsd.port.mk
+++ b/Mk/bsd.port.mk
@@ -4428,6 +4428,7 @@ describe-json:
        ${ECHO_CMD} \"pkg_depends\":[\"${PKG_DEPENDS:ts,:Q:S/,/\",\"/g}\"], ;\
        ${ECHO_CMD} \"complete_options_list\":[\"${COMPLETE_OPTIONS_LIST:ts,:S/,/\",\"/g}\"], ;\
        ${ECHO_CMD} \"categories\":[\"${CATEGORIES:ts,:S/,/\",\"/g}\"], ;\
+       ${ECHO_CMD} \"license\":[\"${LICENSE:ts,:S/,/\",\"/g}\"], ;\
        ${ECHO_CMD} \"deprecated\":\""${DEPRECATED:S/"/\\\"/g:S/\\\\*/*/g:S/\\\'/'/g}" \", ;\
        ${ECHO_CMD} \"broken\":\"${BROKEN:Q:S/"/\\\"/g:S/\\\\*/*/g:S/\\\'/'/g}\", ;\
        ${ECHO_CMD} \"distversion\":\"${DISTVERSION}\", ;\

Closed by commit R11:3d9f300e01f5: Mk/bsd.port.{subdir}.mk: create describe-json (authored by fernape). · Explain WhyOct 18 2023, 11:17 AM

This revision was automatically updated to reflect the committed changes.

fernape added a commit: R11:3d9f300e01f5: Mk/bsd.port.{subdir}.mk: create describe-json.

Committed with the license info added.

Thanks for the review!

Revision Contents
Changeset List

Path

Size

Mk/

bsd.port.mk

31 lines

bsd.port.subdir.mk

20 lines

Diff 128450

View Options

Mk/bsd.port.mk

Show First 20 Lines • Show All 47 Lines • ▼ Show 20 Lines

.Fa fd

is a descriptor to be operated on by

.Fa cmd

as described below.

Depending on the value of

.Fa cmd ,

.Fn fcntl

can take an additional third argument

.Fa "long arg" .

.Fa arg .

jhbUnsubmitted

Done

Maybe just say the type of the 3rd arg depends on the operation (and document the type for each operation that takes the third arg) instead of trying to assume it's a single type? In CheriBSD we have a wrapper in libc to deal with this and have fcntl() as a truly var-args function. On existing platforms var-args is close enough to work for passing syscall arguments, but not all ABIs require this. (var-args are passed as a block in a bounded pointer on CHERI platforms and not just a continuation of register arguments).

jhb: Maybe just say the type of the 3rd arg depends on the operation (and document the type for each…

kibAuthorUnsubmitted

Done

I pushed everything but this change, the review is updated with the man page changes.

kib: I pushed everything but this change, the review is updated with the man page changes.

Unless otherwise noted below for a specific operation,

emasteUnsubmitted

Done

maybe "with type dependent on the specific operation"

or maybe

can take an additional third argument
.Fa arg.
Unless otherwise noted below for a specific operation,
.Fa arg
has type
.Vt int .

emaste: maybe "with type dependent on the specific operation" or maybe ``` can take an additional…

.Fa arg

has type

.Vt int .

.Bl -tag -width F_DUP2FD_CLOEXEC

.It Dv F_DUPFD

Return a new descriptor as follows:

.Pp

.Bl -bullet -compact -offset 4n

.It

Lowest numbered available descriptor greater than or equal to

.Fa arg .

▲ Show 20 Lines • Show All 121 Lines • ▼ Show 20 Lines

.It Dv F_GET_SEALS

Get seals associated with the file, if the underlying filesystem supports seals.

.It Dv F_ISUNIONSTACK

Check if the vnode is part of a union stack (either the "union" flag from

.Xr mount 2

or unionfs).

This is a hack not intended to be used outside of libc.

.It Dv F_KINFO

Fills a

markjUnsubmitted

Done

.It Dv F_KINFO

- Fills

+ Fills a

.Vt struct kinfo_file

markj:

.Vt struct kinfo_file

for the file referenced by the specified file descriptor.

The

.Fa arg

argument should point to the storage for

.Vt struct kinfo_file .

The

.Va kf_structsize

▲ Show 20 Lines • Show All 78 Lines • ▼ Show 20 Lines

Seals are on a per-inode basis and require support by the underlying filesystem.

If the underlying filesystem does not support seals,

.Dv F_ADD_SEALS

and

.Dv F_GET_SEALS

will fail and return

.Dv EINVAL .

.Pp

Several commands are available for doing advisory file locking;

Several operations are available for doing advisory file locking;

emasteUnsubmitted

Done

Is "command" or "operation" the better term, in general? I think we should be consistent between this sentence and the added text above/below

emaste: Is "command" or "operation" the better term, in general? I think we should be consistent…

they all operate on the following structure:

.Bd -literal

struct flock {

off_t l_start; /* starting offset */

off_t l_len; /* len = 0 means until end of file */

pid_t l_pid; /* lock owner */

short l_type; /* lock type: read/write, etc. */

short l_whence; /* type of l_start */

int l_sysid; /* remote system id or zero for local */

};

.Ed

These advisory file locking operations take a pointer to

emasteUnsubmitted

Done

These commands take a pointer to struct flock as the third argument arg.
or
These advisory file locking commands take a pointer to struct flock as the third argument arg.

emaste: These commands take a pointer to struct flock as the third argument arg. or These advisory file…

emasteUnsubmitted

Done

Maybe "operations" here too for consistency?
But from the context this one is clear so I don't mind either way.

emaste: Maybe "operations" here too for consistency? But from the context this one is clear so I don't…

.Vt struct flock

as the third argument

.Fa arg .

The commands available for advisory record locking are as follows:

.Bl -tag -width F_SETLKWX

.It Dv F_GETLK

Get the first lock that blocks the lock description pointed to by the

third argument,

.Fa arg ,

taken as a pointer to a

.Fa "struct flock"

▲ Show 20 Lines • Show All 477 Lines • Show Last 20 Lines

View Options

Mk/bsd.port.subdir.mk

Show First 20 Lines • Show All 47 Lines • ▼ Show 20 Lines

.Fa fd

is a descriptor to be operated on by

.Fa cmd

as described below.

Depending on the value of

.Fa cmd ,

.Fn fcntl

can take an additional third argument

.Fa "long arg" .

.Fa arg .

jhbUnsubmitted

Done

jhb: Maybe just say the type of the 3rd arg depends on the operation (and document the type for each…

kibAuthorUnsubmitted

Done

I pushed everything but this change, the review is updated with the man page changes.

kib: I pushed everything but this change, the review is updated with the man page changes.

Unless otherwise noted below for a specific operation,

emasteUnsubmitted

Done

maybe "with type dependent on the specific operation"

or maybe

can take an additional third argument
.Fa arg.
Unless otherwise noted below for a specific operation,
.Fa arg
has type
.Vt int .

emaste: maybe "with type dependent on the specific operation" or maybe ``` can take an additional…

.Fa arg

has type

.Vt int .

.Bl -tag -width F_DUP2FD_CLOEXEC

.It Dv F_DUPFD

Return a new descriptor as follows:

.Pp

.Bl -bullet -compact -offset 4n

.It

Lowest numbered available descriptor greater than or equal to

.Fa arg .

▲ Show 20 Lines • Show All 121 Lines • ▼ Show 20 Lines

.It Dv F_GET_SEALS

Get seals associated with the file, if the underlying filesystem supports seals.

.It Dv F_ISUNIONSTACK

Check if the vnode is part of a union stack (either the "union" flag from

.Xr mount 2

or unionfs).

This is a hack not intended to be used outside of libc.

.It Dv F_KINFO

Fills a

markjUnsubmitted

Done

.It Dv F_KINFO

- Fills

+ Fills a

.Vt struct kinfo_file

markj:

.Vt struct kinfo_file

for the file referenced by the specified file descriptor.

The

.Fa arg

argument should point to the storage for

.Vt struct kinfo_file .

The

.Va kf_structsize

▲ Show 20 Lines • Show All 78 Lines • ▼ Show 20 Lines

Seals are on a per-inode basis and require support by the underlying filesystem.

If the underlying filesystem does not support seals,

.Dv F_ADD_SEALS

and

.Dv F_GET_SEALS

will fail and return

.Dv EINVAL .

.Pp

Several commands are available for doing advisory file locking;

Several operations are available for doing advisory file locking;

emasteUnsubmitted

Done

Is "command" or "operation" the better term, in general? I think we should be consistent between this sentence and the added text above/below

emaste: Is "command" or "operation" the better term, in general? I think we should be consistent…

they all operate on the following structure:

.Bd -literal

struct flock {

off_t l_start; /* starting offset */

off_t l_len; /* len = 0 means until end of file */

pid_t l_pid; /* lock owner */

short l_type; /* lock type: read/write, etc. */

short l_whence; /* type of l_start */

int l_sysid; /* remote system id or zero for local */

};

.Ed

These advisory file locking operations take a pointer to

emasteUnsubmitted

Done

These commands take a pointer to struct flock as the third argument arg.
or
These advisory file locking commands take a pointer to struct flock as the third argument arg.

emaste: These commands take a pointer to struct flock as the third argument arg. or These advisory file…

emasteUnsubmitted

Done

Maybe "operations" here too for consistency?
But from the context this one is clear so I don't mind either way.

emaste: Maybe "operations" here too for consistency? But from the context this one is clear so I don't…

.Vt struct flock

as the third argument

.Fa arg .

The commands available for advisory record locking are as follows:

.Bl -tag -width F_SETLKWX

.It Dv F_GETLK

Get the first lock that blocks the lock description pointed to by the

third argument,

.Fa arg ,

taken as a pointer to a

.Fa "struct flock"

▲ Show 20 Lines • Show All 477 Lines • Show Last 20 Lines

Mk/bsd.port.{subdir}.mk: create describe-jsonClosedPublicActions

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 128450

Mk/bsd.port.mk

Mk/bsd.port.subdir.mk

Mk/bsd.port.{subdir}.mk: create describe-json
ClosedPublic
Actions

Revision Contents
Changeset List