This looks pretty well thought out. Give me some time to review it and get back to you. Thank you for your efforts.

In D16132#711222, @pstef wrote:

Looks good to me.

I'm playing with this utility on my stable/13 and I like the idea very much. It feels unixy to me in that the tool is abstract enough for me to apply it to various problems which may be difficult to define beforehand. But I can already see myself using this instead of generating permutations with recursive CTEs in SQL like I sometimes do.

Also, although I'm not sure anyone will find this meaningful, cmb reminds me of a problem I was solving recently at job, feeding a Cassandra table random data:

The table and its keyspace were defined as follows:
CREATE KEYSPACE test WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
CREATE TABLE test.base(part text, ord text, fill text, PRIMARY KEY((part), ord));

We feed the table like this:
# base32 -w 2000 < /dev/urandom | cut -b1-3,4-8,9- --output-delimiter=, | cqlsh -u $CASS_USER -e 'COPY test.base FROM STDIN'

Cutting base32 output into columns of arbitrary length gives us control over cardinality of each column; specifically we want to know the upper bound on the number of partitions and the number of rows in each partition. Given a set of k (32 in base32) characters, the number of n-tuples (n characters in a column) is k^n. For example, taking the first 3 letters of each line to use in the partitioning column gives us the upper bound of 32^3 partitions, which is 32768.

Forgive the verbosity and being Ubuntu-specific (base32, cut --output-delimiter, /dev/urandom), but I hope you get the idea: I tend to use unix tools when interacting with other tools or services, databases especially.

By the way, the util and the library seem small enough:

-r-xr-xr-x  1 root  wheel    46K Aug  5 14:17 /usr/bin/indent
-r--r--r--  1 root  wheel    21K Aug 15 17:49 /usr/lib/libcmb.so.0
-r-xr-xr-x  1 root  wheel    31K Aug 15 17:50 /usr/bin/cmb

In D16132#626980, @gbe wrote:

LGTM for the man page related things.

In D16132#565399, @markj wrote:

In D16132#564913, @dteske wrote:

Hi markj,

Yes, it was suggested that I make a port first.

However, everybody seems to be ignoring what I am saying regarding the utility of this tool being equitable to seq, jot, and friends.

Just yesterday, I did the following:

cmb -d / {a..z} | xargs stat 2>&1 | dpv -l -

The purpose was to generate millions of stat errors so that I could work on some filesystem patches to improve the performance of stat in nfsd.

Could I have used jot? Maybe, but it would have been more difficult.
Could I have used seq? Maybe, but same problems as jot. I needed discretely unique test matter.

I have been ignoring the suggestion to put it into ports because not one single person has even addressed the use-cases for base.

How would one make an argument for jot or seq being in base? If you pretend that they are not in base, what arguments would you use to introduce them?

cmb is kind of like that -- it has a million uses. Asking why we should have it in base is kind of like asking why we should have bc in base.

cmb is a math tool. A very powerful tool that fills a myriad requirements, not to mention the following use-case which I will once-again reiterate that has once again been ignored ...

I don't care about the build option survey. It was an example.

I care about using this for combinatoric combination of ports given various options.

Let's talk about dialog4ports -- a tool that lives in ports that is required by ports. So many times this has caused me headaches and I really don't want to go down that road. It is very frustrating when you get into a situation where ports needs X from ports but you can't compile it because your base and ports frameworks have diverged so you then have to devolve into first updating your base shared Mk files relied-on by ports. It's real shit-show.

Putting cmb in base will allow it to:
a. solve math problems
b. allow me to work on base enhancements with only base (think Filesystem debugging, filesystem optimizations, memory testing, scheduler profiling, etc.)
c. be usable by ports without being stuck in the quagmire-catch-22 described above

Regarding the prior discussion, I just see that the original thread on -announce appears to end with you saying that you would make a port. That didn't happen, and since there was some objection to putting cmb and libcmb straight into the base system, we should make sure that those objections won't be raised again after a commit. The right way to do that is to follow up on the lists, like -hackers and -arch, since only a small handful of developers are subscribed to this review.

I understand that cmb is intended to be a general-purpose utility, but that alone is not sufficient for putting it in the base system. jot(1) and seq(1) make for an interesting comparison since seq's functionality is a subset of jot's; seq was added to the base system relatively recently, specifically to make us more compatible with other unix environments. jot predates FreeBSD and comes from a time where a batteries-included approach to shipping an OS was more important than it is today.

I personally think that cmb would be an interesting addition to the base system, but there should be some concrete justification for it to be there. The build option survey is not really a compelling example. A ports target that builds a port with all possible combinations of options seems useful, but is there currently some work in progress to implement that? I understand that having the ports framework depend on a port is painful, but unless the absence of cmb from the base system is currently blocking a useful project, I don't see why this is a strong argument for not making a cmb port. You've written a number of large sh-based components of the base system - could any of them make use of cmb? My point is just that I believe that you could successfully argue for cmb's inclusion in the base system, but the idea first needs to be socialized more fully. A port is a reasonable alternative in the meantime and will make it easy for others to try using cmb.

In D25711#616156, @dteske wrote:

Just a simple bogon in the src bits, after which I will approve

Just a simple bogon in the src bits, after which I will approve

Thank for submitting. Give me a chance to review, but I do believe this looks like a good candidate for commit. Thank you

Sorry for the delay, had a disk failure take out my operations for a full 4 weeks, but finally got all the data recovered and back up and running

Hi markj,

I've only a couple changes I'll make before committing with reference to OP. Instead of "doclose" I will use "opened" which makes the code easier to read.

Take

I'm going to set an ETIMEOUT of 30 days. If there is no objection, I will import to base HEAD.

Indentation fixes. Fix whitespace for consistency. Fix >80c lines.

Pre-update commandeer

LGTM

@crees oh, didn't know @debdrup wasn't a committer. You know how you get so used to seeing someone's name in your TL that you just assume they are in-the-fold ;D lol

Thank you!

Returning to this, as I still want to add it to base.

In D23641#525823, @emaste wrote:

Ping?

The purpose of the awk is to trim a single leading-tab from the input.

In D16132#498554, @hselasky wrote:

In pkg we have a SAT solver. Can you explain how your CMB compares to a SAT solver?

I've had time to reflect, and I sincerely apologize for my unreasonable reaction to a reasonable inquiry.

Remove PORTVERSION

Remove PORTVERSION

In D22681#496190, @jhb wrote:

Erm, there's this bit in bsd.port.Mk:

.if defined(DISTVERSION)
DEV_ERROR+=     "Defining both PORTVERSION and DISTVERSION is wrong, only set one, if necessary, set DISTNAME"
.endif

I suspect DEV_ERROR means it only errors if you have DEVELOPER=yes set in /etc/make.conf as the porter's handbook recommends (see the start of chapter 3 at https://www.freebsd.org/doc/en_US.ISO8859-1/books/porters-handbook/book.html). I ran into this error the other day myself when I had a bug in one of my ports (was including bsd.port.mk again after bsd.port.pre.mk).

So maybe before you blast someone with "please test before submitting reviews" maybe *you* should test

You know that field in Phabricator and arc that says "Test Plan"?

Remove extra newline

Remove extra newline as-per mat's instruction

Remove extra newline as-per mat

Remove extra newline as-per mat

Remove extra newline per-mat

Remove extra newline

Drop Makefile change from review (done by addport)

Drop Makefile change from review (done by addport)

Drop Makefile change from review (done by addport)

Drop Makefile change from review (done by addport)

Drop Makefile change from review (done by addport)

Update LICENSE in Makefile

In D22628#495001, @mat wrote:

That is only one line of code worth. What problem are you trying to solve?

Remove noninja args to USES=cmake