Page MenuHomeFreeBSD

science/tensorflow-core: Work-in-progress port for TensorFlow core library.
Needs ReviewPublic

Authored by arrowd on Jun 19 2018, 12:27 PM.

Details

Reviewers
0mp
Group Reviewers
Contributor Reviewers (ports)
Summary

At this moment the port doesn't even build due to strange compiler/linker bug. The Diff is created to make it easier to reproduce that bug.
I'll proceed shaping the port after it is fixed.

Diff Detail

Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 19906
Build 19427: arc lint + arc unit

Event Timeline

arrowd created this revision.Jun 19 2018, 12:27 PM
tcberner added inline comments.
science/tensorflow-core/Makefile
6

^ wrong order, ...PREFIX should come before ...VERSION

14

^ it seems this is already needed during fetch as it is now?

19

^ this is the default, you can drop it

53

^ this sounds like you want HAS_CONFIGURE

linimon retitled this revision from Work-in-progress port for TensorFlow core library. to science/tensorflow-core: Work-in-progress port for TensorFlow core library..Jul 18 2018, 1:21 PM
arrowd marked 4 inline comments as done.Oct 2 2018, 9:21 AM
arrowd updated this revision to Diff 48645.
  • Update to 1.11.0 and address comments.
0mp requested changes to this revision.Oct 2 2018, 9:24 AM
0mp added a subscriber: 0mp.
0mp added inline comments.
science/tensorflow-core/Makefile
11

Missing LICENSE_FILE=${WRKSRC}/LICENSE.

This revision now requires changes to proceed.Oct 2 2018, 9:24 AM
mat added a subscriber: mat.Oct 3 2018, 8:36 AM
mat added inline comments.
science/tensorflow-core/Makefile
14

A port can never depend on lang/python. (or lang/python2 or lang/python3.)

43

Why the include?

57

This is empty, remove.

arrowd marked an inline comment as done.Oct 3 2018, 10:55 AM
arrowd added inline comments.
science/tensorflow-core/Makefile
14

What should be done then? There is no USES=python:fetch.

mat added inline comments.Oct 3 2018, 12:46 PM
science/tensorflow-core/Makefile
14

Mmm, right, in any way, not lang/python, so python${PYTHON_DEFAULT}:lang/python${PYTHON_DEFAULT:S/.//} would probably be the right one.

48–51

You don't have access to WRKDIR during the fetch phase. (and you cannot access the network during the extract phase.)

arrowd marked an inline comment as done and an inline comment as not done.Oct 3 2018, 1:49 PM
arrowd added inline comments.
science/tensorflow-core/Makefile
48–51

You don't have access to WRKDIR during the fetch phase. (and you cannot access the network during the extract phase.)

I'm doomed, then? For build systems like Bazel or Haskell Stack it is extremely difficult to prepare all required distfiles, because they tend to download them themselves. I can try going hardcore way by examining what and where to gets downloaded and then make up distinfo from that, but maybe there is another way?

I was suspecting I can't access network after fetch, but I'm really surprised that I can't touch WRKDIR during fetch. Why is that?

mat added inline comments.Oct 3 2018, 1:55 PM
science/tensorflow-core/Makefile
48–51

Because it is created during the extract phase.

I think the correct way to go is to create a USES=bazel that works the same way as USES=cargo, in two steps, one for the maintainer to generate a list of stuff to download, and add to the Makefile, and then for the build system to use that to download stuff.

arrowd added inline comments.Oct 3 2018, 8:54 PM
science/tensorflow-core/Makefile
48–51

Unlike Cargo, Bazel fetches distfiles from various sources (like, https://github.com/grpc/grpc/archive/v1.13.0.tar.gz and https://curl.haxx.se/download/curl-7.60.0.tar.gz). We'll end up with huge MASTER_SITES list and huge distinfo. Should we really follow this path?

If we can't unpack to WRKDIR, maybe some other temporary dir will do? DISTDIR even?

mat added inline comments.Oct 4 2018, 8:53 AM
science/tensorflow-core/Makefile
48–51

I'm not sure having a huge MASTER_SITES is a problem, as it would be automatically generated.

arrowd added inline comments.Oct 4 2018, 9:40 AM
science/tensorflow-core/Makefile
48–51

As I said, automatic generation isn't easily implemetable for Bazel's BUILD files. For instance, TensorFlow has clauses like

tf_http_archive(
        name = "grpc",
        sha256 = "50db9cf2221354485eb7c3bd55a4c27190caef7048a2a1a15fbe60a498f98b44",
        strip_prefix = "grpc-1.13.0",
        system_build_file = clean_dep("//third_party/systemlibs:grpc.BUILD"),
        urls = [
            "https://mirror.bazel.build/github.com/grpc/grpc/archive/v1.13.0.tar.gz",
            "https://github.com/grpc/grpc/archive/v1.13.0.tar.gz",
        ],
)

scattered,possibly, over several .bzl files. We can parse it out, of course, but this would be TensorFlow-specific anyways. And since it is me who will be doing all the heavy-lifting, I'd like to hear if extracting into some temporary dir during fetch step is an absolute "no-no", before going this gross (in my opinion) way.

arrowd updated this revision to Diff 56719.Apr 26 2019, 5:26 PM

Take repackaging route, like wine-i386.

I was able to produce libtensorflow.so and this port installs repackaged binary. This approach is also working for Python bindings, but this are currently failing due to external deps.

arrowd marked 7 inline comments as done.Apr 26 2019, 5:30 PM

Mark old comments as done.

mat added inline comments.Apr 30 2019, 3:25 PM
science/tensorflow-core/Makefile
30

The 3.6 here should probably not be hardcoded.

46–47

This can be done in one step:

git clone -b v${DISTVERSION} https://github.com/tensorflow/tensorflow tensorflow-${DISTVERSION}

Also, if you don't care about the history, you could add a --depth 1 to speed up the download (goes from 350MB down to 45MB).

arrowd marked 2 inline comments as done.May 1 2019, 7:30 AM
arrowd updated this revision to Diff 56902.
  • Add ONLY_FOR_ARCHS, address comments.