[NEW PORT] devel/staf: Software Testing Automation Framework

[skozlov]

Summary

A port of STAF (Software Testing Automation Framework)
Bugzilla: https://bugs.freebsd.org/203728

This port:

• Unlike https://bugs.freebsd.org/94286 fully respects hier(7).
• Has full development process log which can be found at https://github.com/mutalisk404/staf-port/commits/master
• Implements Python 2.7-3.4 support (2.7, 3.4 tested)
• Implements OpenSSL support (base, port, LibreSSL tested)
• Implements optional IPv6 support (locally tested)

Known limitations:

• Does not implement support for many other STAF bindings like Java, Perl etc. Work in progress.

Questionable:

• stafproc daemon runs as root by default. See https://github.com/mutalisk404/staf-port/commit/eb0490c59b4b151b48486fb29c935b7375d7ba82

Test Plan

• Tested basic functionality as described in http://staf.sourceforge.net/current/STAFDiag.htm#d0e633 using local ipc, IPv4 and IPv6
• Tested Python bindings by importing all installed modules and running random informational functions
• Tested that debug/non-debug build produce not-stripped/stripped binaries and libs

Portlint output:

WARN: Makefile: possible use of absolute pathname "/var/db/STAF".
WARN: /poudriere/data/.m/102x64-default-debug/ref/usr/ports/devel/staf/pkg-message: possible use of absolute pathname "/etc/hosts:".

Explained:

• Absolute path to /var used for definition of db directory, nothing to substitute with from ports' framework.
• Absolute path to /etc can only be found in pkg-message

Poudriere:
Default options: https://bugs.freebsd.org/bugzilla/attachment.cgi?id=161961
Debug option only: https://bugs.freebsd.org/bugzilla/attachment.cgi?id=161962
Python 2.7: https://bugs.freebsd.org/bugzilla/attachment.cgi?id=161963
Python 3.4: https://bugs.freebsd.org/bugzilla/attachment.cgi?id=161964
LibreSSL: https://bugs.freebsd.org/bugzilla/attachment.cgi?id=161965

[koobs]

head/devel/staf/Makefile

8

Is there a https MASTER_SITE available? If so please add that as the primary with http:// as the secondary

14

Does a license file exist in WRKSRC? If so, add LICENSE_FILE=/path/to/file

49

I prefer to see OPTIONS_* blocks higher up, either before or after USE{S}_* section

56

Can you explicitly disable this if option is _OFF?

58

Can you explicitly disable this if option is _OFF?

63

This says to me that it supports Python 2 and Python 3. Is > 3.4 *explicitly* not supported?

Depending on the response, this may be better off as PYTHON_USES=python

64

This will use py3kplist even for Python 2.x builds? Why is that needed, or am I missing something?

head/devel/staf/pkg-message

4

• Typo: "requieres"
• Remove "Note that"

6

"If you haven't already"

9

Are you sure you want to suggest this in all scenarios or user configurations?

Perhaps an entry like:

<IP Address of machine> <hostname>

[koobs]

By the way, this is a GREAT review, well done! :)

[skozlov]

head/devel/staf/Makefile

8

Yes, it has https, will fix

14

It has only html version

49

Will do

56

Yes, but if IPv6 is off then I need to set staf_use_ipv6 to <blank> as noted in the documentation, should I?

58

The same as for IPv6, documentation here

63

The software includes code to build modules specially for different versions of Python and supported versions are 2.2-3.4. For 3.5 I need to build 3.4 modules and put them into 3.5 site-packages for what I need to write a lot of if-fy code. Personally I would like to wait for STAF developers to officially support python 3.5. Documentation on Python support here

64

Indeed it will enable py3kplist for Python 2.x too, but it has no effect on Python 2.x cause the code related to py3kplist first checks if the python version is >3.2 (code here). It can be moved to Python 3.x specific part between lines 76 and 80, but the move will have no effect on the process.

head/devel/staf/pkg-message

4

Will fix.

"Note that" taken from python27: https://svnweb.freebsd.org/ports/head/lang/python27/pkg-message?revision=363790&view=markup#l3

6

Will fix, copypaste from openjdk7
https://svnweb.freebsd.org/ports/head/java/openjdk7/pkg-message?revision=357236&view=markup#l6

9

It may be a better solution, I just wanted to make it easier for inexperienced users. Experienced users will know what they have to do after the 1st line.

[skozlov]

Fix obvious mistakes reported by Koobs

[skozlov]

Explicitly disable IPV6 and OPENSSL options

If the environment which "make" is ran in has one of STAF_USE_[IPV6|OPENSSL]=1
set it can affect the building process of cause the building process to fail

[skozlov]

head/devel/staf/Makefile

57

I realized why that's needed, please see the comment to new diff

59

I realized why that's needed, please see the comment to new diff

[koobs]

head/devel/staf/Makefile

14

I suppose HTML is better than nothing, feel free to add it to LICENSE_FILE, and ask them for a plaintext one in an issue upstream ;)

56

Does it work?

Failing empty working, have you tried ='' ?

57

Not sure I understand?

59

Not sure I understand?

63

If you're saying there's not just two versions of the code (one for 2.x and one for 3.x), that is really annoying.

But fine for now if it's too much work or more clarity is required.

64

I'll take your word for it :)

head/devel/staf/pkg-message

4

Feel free to provide a patch to fix ;)

9

I'll leave the choice with you.

The goal should only to make this port work, without impacting other configurations. The closer we can get to that the better.

[skozlov]

head/devel/staf/Makefile

57

Disregard that, this is related to your comment for the first diff about explicitly switching off IPV6 and OPENSSL.
I'm just making my mistakes and learn how to use phabricator.

59

Disregard that, this is related to your comment for the first diff about explicitly switching off IPV6 and OPENSSL.
I'm just making my mistakes and learn how to use phabricator.

[skozlov]

• Added LICENSE_FILE
• Make setting IPV6 and OPENSSL off even more explicit

[skozlov]

Forgot to add devel/Makefile to previous diff

[skozlov]

head/devel/staf/Makefile

56

It does, but at the last diff I've made disabling the option as explicit as possible.