Add new port textproc/scancode-toolkit

[miwi]

Summary

This diff adds the port textproc/scancode-toolkit to the ports tree. It's also listed on the WantedPorts wiki page.

ScanCode is a suite of command line utilities to reliably scan a codebase for license, copyright, package manifests and direct dependencies and other interesting origin and licensing information discovered in source and binary code files.

• Builds fine on 11.2-RELEASE, 12.0-BETA2, 13.0-CURRENT@ r339730 for each amd64 + i386 architecture

The upstream package comes with pre-compiled binaries (file, p7zip, libmagic) for non-BSD platforms that comes in form of separate Python packages (which are also available on PyPi). Those binaries are stripped and replaced by relative symlinks in the build process.

PR # 230114

[pfg]

Cool thanks! Please try to upstream the changes when you can.

[pfg]

I think you still have to fix the shebang on "configure".

I created a new issue on github to track the fixes:

https://github.com/nexB/scancode-toolkit/issues/1147

[kai]

In D16484#350055, @pfg wrote:

I think you still have to fix the shebang on "configure".

I created a new issue on github to track the fixes:

https://github.com/nexB/scancode-toolkit/issues/1147

Thanks for your feedback and suggestions! I have to admit that I did not look at the configure script closely because the configure script is intended to create scancode for virtual environments.

If you like I can update the diff with the required tweaks for FreeBSD in conjunction with the configure script. Three more files (configure, etc/configure.py and etc/conf/base.py) would then be patched but will not be used otherwise in the whole build process.

[mat]

textproc/scancode-toolkit/Makefile

85

ARCH is not defined before including bsd.port.options.mk.

[kai]

Addressed the hint from mat@:

• added bsd.port.options.mk before referencing the ARCH variable

[kai]

textproc/scancode-toolkit/Makefile

85

Thanks for pointing that out.

[kai]

Addressing a suggestion miwi@ gave me on IRC:

It would be better when scancode-toolkit uses libarchive and file from the ports tree instead of the base system. This makes the port more independent from the different FreeBSD releases and one would use always the same set of libarchive and file regardless on which platform scancode-toolkit will be run.

Thus the following changes were made:

• added file and libarchive to RUN_DEPENDS. (I've tried to add libarchive by usingc LIB_DEPENDS first but then the build process falls back using /usr/lib/libarchive.so.)
• minor changes to the comments
• changed ${LN} commands in the post-patch target

[mat]

textproc/scancode-toolkit/Makefile

87

missing blank line after include.

102–104

RM can take more than one argument, you know.

108–110

MKDIR can take more than one argument.

[kai]

Addressing the remarks given by mat@:

• add missing blank after ".include <bsd.options.mk>"
• optimize the usage of ${MKDIR} and ${RM}

[mat]

textproc/scancode-toolkit/Makefile

89–96

Thinking about this, it would probably be more readable to do something like this:

(list taken from the output of make targets on my 11.1 box.)

.for _a in arm armeb armv6 i386 mips mipsel mipsn32 powerpc
ARCHDIR_${_a}= freebsd-32
.endfor

ARCHDIR=    ${ARCHDIR_${ARCH}:Ufreebsd-64}

And you could drop the .include altogether.

124–129

All those links are somewhat bogus. Links inside PREFIX/LOCALBASE should be relative, not absolute.

[kai]

Addressing given hints and suggestions from mat@:

• made test for 32-/64-bit architecture more readable
• changed ${LN} occurrences into ${RLN} to create relative links inside ${LOCALBASE}/${PREFIX}
• removed ".include <bsd.options.mk>" and ARCHBITS variable - both are no longer required

[kai]

textproc/scancode-toolkit/Makefile

89–96

Thanks, it's now much clearer to read and to revise when changes in the future are required. Sometimes I really miss the forest for the trees...

[mat]

textproc/scancode-toolkit/Makefile

119–123

RLN only works correctly within STAGEDIR, you cannot use it to references files outside of it.

It will work, but it will generate very long relative paths looking like ../../../../../../../../usr/local/blah, which only work because at one point, you reach the root directory and .. = ..

[kai]

The updated diff contains a revision of the install target regarding the ${RLN} operations as pointed out by mat@:

• The target creates now bogus directories/files for LOCALBASE in the stage directory. Those files are used temporarily as sources for the relative symbolic links that will be created in the later step. After the creation of the symbolic links the bogus files are removed.
• added BOGUSLNKS and BOGUSDIR_ variables
• added/revised comments

[kai]

textproc/scancode-toolkit/Makefile

124–129

I have to admit that I was not completely sure about it whether to use absolute/relative links. I took beside consulting the PHB also a quick search in the ports tree for some relevant examples but I have found so far only occurrences like ${LN} -sf ${LOCALBASE}/source/dir ${STAGEDIR}/target/dir .

Thus I hope that the approach in this diff is not too complicated but maybe there's still the one or other better way to accomplish this.

[kai]

Upstream released 2.9.7 some days ago which introduced some major changes regarding the pre-compiled binaries. The pre-compiled binaries are now outsourced into separate Python packages and are already available on PyPi:

• extractcode-7z
• extractcode-libarchive
• typecode-libmagic

Thus the textproc/scancode-toolkit port now can be set as architecture-neutral. On the other hand the platform-specific Python packages are not required on FreeBSD but were added as new dependencies in the setup.py file and thus are required for runtime.

In the perspective of FreeBSD only the directory structure and the Python code of each platform-specific Package is required. When building the package, following actions happen:

• build scancode-toolkit
• build the Python packages extractcode-7z, extractcode-libarchive and typecode-libmagic
• create bogus, relative links for 7z, 7z.so, libarchive.so, magic.mgc and libmagic.so into the according Python packages

[kai]

Removed BUILD_DEPENDS from previous diff, otherwise no changes.

Please note, that the following ports in the ports tree must be updated first to build the new revision of scancode-toolkit successfully:

• devel/py-boolean.py (see also PR 232881)
• textproc/py-license-expression (see also PR 232880)

[miwi]

Committed already