books: Updating ports count statistics in dev-model

[bofh]

Summary

I have created a script that does the following:

• Calculates ports count yearly
• Calculates doc, ports, src, total unique committer numbers

This script is available at:
https://people.freebsd.org/~bofh/dropzone/dev-model.sh

And the graph generated is using this script. I have also removed the tables because it has been more than 28 years and putting this data in the tables is not feasible at all. Additionally the tables of each 1000 milestone is difficult to achieve now hence has got ridden of it too. After all a picture talks a thousand words.

Additionally I am working on a patch which will generate this graph automagically. But I will most probably need some help from @carlavilla . I will send him a patch and with some of his help I believe we can work it out.

Test Plan

http://fdp.bofh.network/books/dev-model/index.html#sub-project-ports

[bofh]

Vague hope for @pauamma

[pauamma_gundo.com]

documentation/content/en/books/dev-model/_index.adoc

1076

Can we compromise on each multiple of... I guess 5000 or 10000? Not having that table at all is horrible for screenreader friendliness (the reason I put it here).

1136

Likewise, condense it to every 5 or 10 years?

[bofh]

documentation/content/en/books/dev-model/_index.adoc

1076

This is a TOO manual process and I believe is not worth the time.

1136

Yes this is possible. Currently we can even go for 2 years which will give us roughly 14 entries. 10-15 rows of data looks reasonable to me; but more than that is hard to analyze and none checks through those.

[bcr]

I like this initiative. It does make the page much better than before and with the update script, we'll keep it up-to-date.
I'd say we commit the base bits now and refine it in a follow-up commit (i.e. when the script is running with help from Sergio).

[bofh]

Committed for now while I create another diff for the tables mentioned by @pauamma

The actual value of the table is as following:

YEAR	COUNT
1995	192
1996	485
1997	917
1998	1608
1999	2546
2000	3746
2001	5446
2002	7787
2003	9275
2004	11492
2005	13534
2006	15225
2007	17411
2008	18864
2009	20390
2010	21619
2011	22932
2012	23290
2013	24153
2014	24643
2015	24524
2016	25670
2017	26709
2018	27866
2019	28649
2020	28827
2021	28951
2022	30148

@pauamma Give me a table of your choice reducing number of rows

[grahamperrin]

Thanks, should there be a change to the date of copyright?

If so, someone might roll it into 267033 – A project model for the FreeBSD Project: STABLE is not the production release

[pauamma_gundo.com]

In D36972#839878, @bofh wrote:

Committed for now while I create another diff for the tables mentioned by @pauamma
@pauamma Give me a table of your choice reducing number of rows

Going for every other year for now. We can switch to every 5th in 5 years or so.

YEAR	COUNT
1995	192
1997	917
1999	2546
2001	5446
2003	9275
2005	13534
2007	17411
2009	20390
2011	22932
2013	24153
2015	24524
2017	26709
2019	28649
2021	28951

documentation/content/en/books/dev-model/_index.adoc

1076

This is a TOO manual process and I believe is not worth the time.

Contrariwise, I believe it is well worth my time to keep this document accessible to screenreader users, with (as I mentioned) trimmed down versions of both tables, as I believe both are needed to get an equivalent of the graph. I volunteer to put in work to that effect, as I put in work earlier so those tables (and other changes needed for screenreader friendliness) would be there in the first place.

As for being manual... I think this can be easily automated as well. For each calendar period (year, 2 years, or however long) during which it crosses a 1000, 5000, 10000, or whatever boundary, use linear interpolation between values at the start and end of period (the other table) to approximate the crossing date and output the month and year (30.5 days/month) and perhaps early/mid/late. Again, I'm willing to try adding that to the script, using perl if available or awk otherwise. I can get in touch with @carlavilla if need be.

[pauamma_gundo.com]

In D36972#839912, @grahamperrin wrote:

Thanks, should there be a change to the date of copyright?

I believe I bumped it when I made a bunch of changes in 2020, so I guess yes to be consistent.

[bofh]

In D36972#839912, @grahamperrin wrote:

Thanks, should there be a change to the date of copyright?

If so, someone might roll it into 267033 – A project model for the FreeBSD Project: STABLE is not the production release

My knowledge on COPYRIGHTS is very limited. If we need to change we have to add all the authors who have changed it though so per my little knowledge. Enlighten me please.

[bofh]

documentation/content/en/books/dev-model/_index.adoc

1076

Can you share me a formula or more like an example from the table I have shared before?

[pauamma_gundo.com]

Answer delayed due to 2-3 weeks of markedly above-average anxiety.

documentation/content/en/books/dev-model/_index.adoc

1076

Formula. Well, short algorithm anyway. C syntax not guaranteed correct, my C is very dusty.

/* Input: */
/* first_year is the first year for which we have a number of ports recorded */
/* latest_year is the latest year for which we have a number of ports recorded */
/* year is a year number no less than first_year and no more than latest_year */
/* num_ports is a table of the number of ports by year, indexed by */
/* year - first_year. */
/* granularity is the change in number of ports by year we consider significant */
/* (every 1000 ports, every 5000 ports, or whatever) */
/* Output: */
/* month is the 0-based month number during which we estimate the number of ports */
/* crossed a multiple of granularity going up between year and year+1, or -1 if */
/* either year or year+1 is outside first_year..latest_year or the number of ports */
/* didn't cross such a boundary going up. */
/* All are signed integers of appropriate length. */

if (granularity <= 0 || year < first_year || year >= latest_year) month = -1;
else if (num_ports[year - first_year] / granularity >= num_ports[year + 1 - first_year] / granularity) month = -1;
else month = (12 * ((num_ports[year - first_year] / granularity + 1) * granularity /* next multiple of granularity */ - num_ports[year - first_year]) / (num_ports[year + 1 - first_year] - num_ports[year - first_year]);

Examples run by hand using 5000 for granularity (first_year being 1995 and latest_year being 2008).
Table fragments:

numports[2001 - 1995] = 4300
numports[2002 - 1995] = 6200
month = (12 * ((num_ports[year - first_year] / granularity + 1) * granularity - num_ports[year - first_year]) / (num_ports[year + 1 - first_year] - num_ports[year - first_year]) = (12 * ((4300 / 5000 + 1) * 5000 - 4300) / (6200 - 4300) = (12 * 700) / 1900 = 4 (May) vs eyeballed mid 2001

numports[2003 - 1995] = 8100
numports[2004 - 1995] = 10050
month = (12 * ((num_ports[year - first_year] / granularity + 1) * granularity - num_ports[year - first_year]) / (num_ports[year + 1 - first_year] - num_ports[year - first_year]) = (12 * ((8100 / 5000 + 1) * 5000 - 8100) / (10050 - 8100) = (12 * 1900) / 1950 = 11 (December) vs eyeballed end of 2003.

numports[2006 - 1995] = 14000
numports[2007 - 1995] = 16200
month = (12 * ((num_ports[year - first_year] / granularity + 1) * granularity - num_ports[year - first_year]) / (num_ports[year + 1 - first_year] - num_ports[year - first_year]) = (12 * ((14000 / 5000 + 1) * 5000 - 14000) / (16200 - 14000) = (12 * 1000) / 2200) = 5 (June) vs eyeballed mid 2006

So far so good. Can you run it (by hand or not) against your table to see how correct it looks on more data?

[bofh]

I will need to check. But now I have more burning issues at hand. Have to work on Phabricator itself. So will be busy for a couple of days before jumping into docs again.