devel/buildbot*: Update to 0.9.11, Add

[• rodrigc]

Summary

devel/buildbot*: Update to 0.9.11, Add buildbot-grid-view

* Update to buildbot 0.9.11
* Add required devel/py-buildbot-grid-view dependency
* Clean up stuff from individual Makefiles that was copy/pasted from the main buildbot Makefile
* buildbot-www, buildbot-grid-view, buildbot-console-view, buildbot-waterfall-view cannot be tested outside
   of buildbot, so remove stuff from their Makefiles to simplify.

Reviewed_by: koobs, ???
Approved by:
Differential_Revision: D12479

Test Plan

• portlint: OK (all ports)
• testport: OK (poudriere: 10.3-RELEASE-amd64, Python 2.7)
• maketest: OK (4 tests failing, 5060 passing for buildbot, 0 tests failing, 229 passing for buildbot-worker)

[asomers]

It looks like devel/buildbot/files/patch-buildbot-secrets is out of date. It still applies, but the result is that buildbot/secrets/providers/{base.py, file.py, vault.py} get duped, and there may be similar errors in other files too. Running "make test" in devel/buildbot detects that error. You should investigate. We can probably delete the entire patch.

devel/buildbot-grid-view/Makefile

14

MY_DEPENDS is unconventional, and portlint complains about it. The more typical thing to do would be:

BUILD_DEPENDS=        buildbot-pkg>=${PORTVERSION}:devel/buildbot-pkg
RUN_DEPENDS=         ${BUILD_DEPENDS}

devel/buildbot-grid-view/pkg-descr

3

Trailing period, please.

devel/buildbot/Makefile

28

You should add buildbot-www, otherwise dozens of tests get skipped.

[• rodrigc]

devel/buildbot-grid-view/Makefile

14

Portlint complains about what you wrote:

RUN_DEPENDS should not be set to ${BUILD_DEPENDS} as ${BUILD_DEPENDS} includes other implicit dependencies.  Instead, copy the explicit dependencies from BUILD_DEPENDS to RUN_DEPENDS.  See http://www.freebsd.org/doc/en_US.ISO8859-1/books/porters-handbook/makefile-depend.html#AEN2154 for more details.

[koobs]

Could you please add change of maintainer to MAINTAINER=koobs@f.o (per Bugzilla #220564 Comment 10)

devel/buildbot-grid-view/pkg-descr

7

Use https URL (where/when supported)

[koobs]

test target (for Python packages) should invoke a specific version of Python, by whatever method, rather than LOCALBASE scripts, which can/may point to any (and incorrect) Python versions at any time.

Examples:

• ${PYTHON_CMD} -m {trial|trial.whatever}
• ${PYTHON_CMD} ${PYDISTUTILS_SETUP} test (if supported)

Also, please confirm QA (ive added portlint, maketest lines to TEST PLAN section)

[koobs]

Lastly, and this has been coming a long-time, but buildbot* ports have been non-compliance with Python packages since their birth:

• naming, not including a py- prefix, AND
• not using PYTHON_PKGNAMEPREFIX

With buildbot now supporting Python 2/3 this is effectively a blocker as multiple installed versions will conflict (and there's no better time than now, less ideally, after this commit).

This also means that buildbot ports will need to become USE_PYTHON=concurrent safe.

Alternatively, I'm happy to defer this to a separate logically isolated commit, to land immediately after this, as long as the relaxing of the USES=python version is removed, and lands with that next change. This removes a window within which multiple concurrent versions 'can' be attempted, but wont (but should) work due to conflicts

Edit:

Having had time for it to soak in a bit more, I definately prefer to see the rename (move), prefixing and concurrent enabling changes in a separate commit, ideally prior to this, but if that's too annoying, after this change is OK

[• rodrigc]

In D12479#258626, @asomers wrote:

It looks like devel/buildbot/files/patch-buildbot-secrets is out of date. It still applies, but the result is that buildbot/secrets/providers/{base.py, file.py, vault.py} get duped, and there may be similar errors in other files too. Running "make test" in devel/buildbot detects that error. You should investigate. We can probably delete the entire patch.

This patch is no longer needed since this issue was fixed: https://github.com/buildbot/buildbot/issues/3086

[• rodrigc]

• Add missing periods.
• Remove patch which is no longer needed.
• Remove MY_DEPENDS

[• rodrigc]

• Use https://
• Use ${PYTHON_CMD} -m twisted.trial to invoke test utility

[koobs]

Also, can gridview be added separately (after landing 0.9.11), or does 0.9.11 require it?

[• rodrigc]

• Remove concurrent
• Add back python:-2.7

[koobs]

@rodrigc Thanks for the detail, though logs aren't explicitly sought (at least by me :])

Putting in "OK's" (or summaries) is fine for TEST PLAN.

[koobs]

This looks OK to land, unless the gridview port addition can be made in a separate commit (preferred), either before this commit if it supports the current port version, or after if it requires 0.9.11.

If this change requires gridview to be added (a new compulsory module, etc), then looks good to go (pending maintainer approval)

[koobs]

Missed this due to log output:

How does make test go?

[• rodrigc]

In D12479#258670, @koobs wrote:

This looks OK to land, unless the gridview port addition can be made in a separate commit (preferred), either before this commit if it supports the current port version, or after if it requires 0.9.11.

If this change requires gridview to be added (a new compulsory module, etc), then looks good to go (pending maintainer approval)

buildbot-gridview needs to go in.

[koobs]

If @grembo is happy with (solely) my review and doesn't have time, he can just accept, in which case, I'm happy for this to land pending:

• Confirmation that maketest passes
• Updating of relevant Property: lines in commit log message (SUMMARY section)

Note: Don't forget to remove underscores in those Property_lines on commit otherwise it wont auto-close :)

[asomers]

Apart from buildbot-worker's patch file change, it all looks good to me. FWIW, I also tested the WWW server and worker with the example pyflakes project.

devel/buildbot-worker/files/patch-setup.py

3 ↗

(On Diff #33369)

Was it really necessary to recreate this patch file? I usually don't recreate patch files that still I apply; I figure it just adds line noise to the diff.

[• rodrigc]

portlint complained about the patch file, so I regenerated it.

[koobs]

If the 4 failing look like false positives, non-trivial or non-concerning, feel free to defer (Disclaimer: I am not MAINTAINER yet)

[• rodrigc]

Fix test targets. Cannot run inside WRKSRC, must run against installed versions.
Put trial temporary files in a separate tmp directory.

[• rodrigc]

The 4 failing tests looks pretty minor and not worth blocking progress on this.

[grembo]

@koobs Just glanced at it, trusting your judgement.

One more thing the ports inherited and that I wanted to change, but didn't get to is in buildbot-www/Makefile:

post-patch:
        @${FIND} ${WRKSRC} -type f | ${XARGS} -n 10 ${REINPLACE_CMD} -e \
                's|/usr/bin/python|${PYTHON_CMD}|g'

This adds newlines at the end of files which don't have it, which includes files in buildbot_www/static/fonts - as a result those are corrupted (fonts still load, but cause errors on the browser console).

[sunpoet]

LGTM with the minor changes. Thanks!

devel/buildbot-console-view/Makefile

15

You could use RUN_DEPENDS:=${BUILD_DEPENDS} instead.

17

Please use USES=python:2.7.

devel/buildbot-pkg/Makefile

14

Please use USES=python:2.7.

devel/buildbot-waterfall-view/Makefile

15

You could use RUN_DEPENDS:=${BUILD_DEPENDS} instead.

17

Please use USES=python:2.7.

devel/buildbot-worker/Makefile

21

Please use USES=python:2.7.

devel/buildbot-www/Makefile

22

Please use USES=python:2.7.

devel/buildbot/Makefile

34–35

Please use USES=python:2.7.

42–44

Please add

REINPLACE_ARGS=-i ''

in USES section to avoid creating .bak files.

devel/py-buildbot-grid-view/Makefile

16 ↗

(On Diff #33370)

Please use USES=python:2.7.

[mat]

devel/buildbot-console-view/Makefile

15

Or even better:

BUILD_DEPENDS=${RUN_DEPENDS}

and you would not have to use :=

devel/buildbot-waterfall-view/Makefile

15

or:

BUILD_DEPENDS= ${RUN_DEPENDS}

devel/buildbot-worker/Makefile

25

Why += ?

devel/buildbot-worker/files/patch-setup.py

3 ↗

(On Diff #33369)

When the line numbers change, it is usually better to update the patches, so that things do not drift too much appart.

devel/buildbot/Makefile

42–44

Or USES=shebangfix

[• rodrigc]

@koobs why wasn't this review updated after I committed rP450642 ?

[koobs]

In D12479#259604, @rodrigc wrote:

@koobs why wasn't this review updated after I committed rP450642 ?

Did not remove underscore in Differential_Revision: line in copy/pasted SUMMARY on on commit

Unfortunately, Phabricator doesn't allow 'Differential Revision' in fields, even in code blocks.

The post commit hook could have its regex syntax relaxed to allow (match) it.

[• rodrigc]

Landed in rP450642