Page MenuHomeFreeBSD

Add USES=node[:args] for specifying a dependency on Node.js
Needs ReviewPublic

Authored by bhughes on Aug 4 2017, 12:30 PM.

Details

Reviewers
luca.pizzamiglio_gmail.com
sunpoet
Group Reviewers
O5: Ports Framework(Owns No Changed Paths)
portmgr
Summary

This is based lightly on Mk/Uses/python.mk. Add build, run, test
dependencies based on the arguments received. A port can specify which
major version it needs. If the port does not explicitly set a version,
use a default which can be overridden by the user. Current possible
values are 4, 6, 8, and 9, which correspond to the active LTS and
Current releases from upstream.

Diff Detail

Repository
rP FreeBSD ports repository
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 15135
Build 15224: arc lint + arc unit

Event Timeline

jrm created this revision.Aug 4 2017, 12:30 PM
jrm updated this revision to Diff 31573.Aug 4 2017, 12:32 PM

Missed a node 7 that should have been 8

swills added a subscriber: swills.EditedAug 4 2017, 3:16 PM

Looks pretty good to me.

It looks good to me.

jrm updated this revision to Diff 31625.Aug 5 2017, 1:17 PM
jrm edited edge metadata.

Remove run-time dependencies on devel/gmake as suggested by adamw.

sunpoet edited edge metadata.Aug 6 2017, 5:11 PM

It still needs more discussion.

  1. npm removed all options since r414303. Besides, each npm depends on different node versions to allow package users (of www/node*) to have an official npm package respectively.
  2. To me, the node.mk is incomplete. At least a port maintainer is not able to mark the port requiring node 6.x and above (e.g. USES=node:6+ or USES=node:6-).
  3. The runtime dependency on gmake was added with PR 204547.

I do not think it's a good idea to combine node.mk with npm/yarn change.

Mk/bsd.default-versions.mk
17–18

It will exceed 80 characters in this line. Please move PERL5 to the next line.

www/npm/Makefile
20 ↗(On Diff #31625)

It's more precise to use USES=node:run.

And www/npm was supposed to run with node 8 (see r414303).

jrm added a comment.Aug 6 2017, 5:30 PM
  1. npm removed all options since r414303. Besides, each npm depends on different node versions to allow package users (of www/node*) to have an official npm package respectively.

This is not what upstream recommends.
npm is a separate project from Node.js, and tends to update more frequently. As a result, even if you've just downloaded Node.js (and therefore npm), you'll probably need to update your npm.

  1. The runtime dependency on gmake was added with PR 204547.

The argument put forth (by others) here was that npm does not require devel/gmake. Some node modules require it to build, but some modules have other dependecies to build and we wouldn't add all those depedencies to www/npm or www/yarn as well.

bhughes added inline comments.Aug 6 2017, 5:44 PM
Mk/Uses/node.mk
26

bhughes@FreeBSD.org now ;)

jrm updated this revision to Diff 31677.Aug 6 2017, 6:44 PM
jrm marked 3 inline comments as done.
  • change node.mk maintainer address to bhughes@FreeBSD.org
  • wrap line that exceeded 80 characters (newline was at 81)
  • switch USES=node to USES=node:run in www/npm and www/yarn
jrm edited the summary of this revision. (Show Details)Aug 6 2017, 6:48 PM

How does node-gyp fit into this picture? This is the component that should pull in gmake dependency.

I would love to switch my custom ports tree used to build node-sass binaries (https://github.com/saper/ports-exp/) to a more sane generic node framework.

jrm added a comment.Aug 7 2017, 12:36 AM

node-gyp isn't in the ports tree, so it's out of our control whether it pulls in gmake. It would be nice if it were smart enough to recognize that it was building on FreeBSD and pull in a local copy of GNU make.

We have to be very careful about creating ports for node modules; it will be a substantial amount of both initial and continuous work to essentially reinvent the npm/yarn wheel. Yes, our wheel would be nicer and customized to our needs, but is the cost to benefit ratio positive? Before we even consider having node modules in the tree, I think someone has to step up to say they are willing to take the reigns for such an investment of time and effort.

node-gyp is brought by npm, at least in some version (often a newer version of node-gyp is required by the modules).

If we drop gmake dependency from npm, we should provide a separate node-gyp port to install it globally. This way FreeBSD's node-gyp can be more up to date than npm's and therefore there will be no need to pull a local version in the package tree.

node-gyp might need some FreeBSD specific patches from time to time (I think it does not support unbundled libuv very well today).

I am also not convinced that polluting FreeBSD ports tree with npm modules is a good idea. But we need sound infrastructure to support binary modules and that
includes seamlessly working node-gyp and node-pre-gyp.

Probably some most problematic binary node extensions (like node-git, node-sass and maybe others could be made to ports. Especially those, that do not provide binary packages for download for FreeBSD. Using pkg here would be a big advantage to the users. Unfortunately, there is currently no way to tell npm - "Don't npm install that module, use pkg".

jrm added a comment.Aug 7 2017, 11:47 AM

If we drop gmake dependency from npm, we should provide a separate node-gyp port to install it globally. This way FreeBSD's node-gyp can be more up to date than npm's and therefore there will be no need to pull a local version in the package tree.

It would be nice to have ports for node-gyp and node modules that require building. If this were the case, ports like net-im/mastodon could then depend on those ports, and then a mastodon-node port could be created, so users wouldn't have to do yarn install at all. But, we would be approaching that slippery slope of replacing npm and yarn. For example, node-gyp depends on 12 node modules. Do each of these dependencies have dependencies?

Is upstream receptive to FreeBSD-specific fixes? If FreeBSD-specific issues were addressed upstream, would that address your concerns?

mat added inline comments.Aug 7 2017, 1:02 PM
Mk/Uses/node.mk
26

Yeah, please don't "gift" this to portmgr :-p

We're also working to make FreeBSD more nodejs friendly. Our goal is to bring electron.io and Atom to FreeBSD.

Mixing packages from the portstree and npm is, in general, not a good idea, because different projects could require different version of the same package. Our approach is to bring in the portstree only the built/native components and somehow tell npm to use those, instead of rebuild, to minimize potential conflicts.

bhughes updated this revision to Diff 35653.Nov 23 2017, 11:02 AM

Commandeer this change and updated it to support the current Node.js
versions in the ports tree: 4, 6, 8, and 9.

bhughes updated this revision to Diff 35654.Nov 23 2017, 11:03 AM

Set MAINTAINER to my FreeBSD.org address

bhughes retitled this revision from Add bhughes' USES=node framework and switch www/npm and www/yarn to use it. to Add USES=node[:args] for specifying a dependency on Node.js.Nov 23 2017, 11:05 AM
bhughes edited the summary of this revision. (Show Details)
bhughes edited the test plan for this revision. (Show Details)
bhughes added a reviewer: robak.

To me, the node.mk is incomplete. At least a port maintainer is not able to mark the port requiring node 6.x and above (e.g. USES=node:6+ or USES=node:6-).

I agree with this.

I do not think it's a good idea to combine node.mk with npm/yarn change.

To me, the node.mk is incomplete. At least a port maintainer is not able to mark the port requiring node 6.x and above (e.g. USES=node:6+ or USES=node:6-).

I agree. Unfortunately my Makefile-fu isn't strong enough at the moment to make that possible. If anyone knows how or can point me at examples of how to do it, I would be very appreciative.

I do not think it's a good idea to combine node.mk with npm/yarn change.

I agree. When commandeering the review, I've removed the npm and yarn changes. I would like to do those after USES=node passes review.

To me, the node.mk is incomplete. At least a port maintainer is not able to mark the port requiring node 6.x and above (e.g. USES=node:6+ or USES=node:6-).

I agree. Unfortunately my Makefile-fu isn't strong enough at the moment to make that possible. If anyone knows how or can point me at examples of how to do it, I would be very appreciative.

python.mk can be a good reference for that. I'm willing to help, if you need and I'm able to

ygy added a subscriber: ygy.Jan 28 2018, 6:09 PM
bhughes updated this revision to Diff 39475.Feb 18 2018, 7:00 PM

Keeping the change up-to-date

bhughes commandeered this revision.Feb 18 2018, 7:01 PM
bhughes edited reviewers, added: jrm; removed: bhughes.

Forgot to actually commandeer the review... done now.

jrm removed a reviewer: jrm.Feb 27 2018, 2:53 PM
robak removed a reviewer: robak.Mar 7 2018, 4:43 PM
mat added a comment.Mar 8 2018, 10:39 AM

Now that we have flavors, it would probably be better to have flavors for this, so that stuff that works with node get built with all the versions they support, no?

saper_saper.info added a comment.EditedMar 11 2018, 9:42 PM

We're also working to make FreeBSD more nodejs friendly. Our goal is to bring electron.io and Atom to FreeBSD.

Mixing packages from the portstree and npm is, in general, not a good idea, because different projects could require different version of the same package. Our approach is to bring in the portstree only the built/native components and somehow tell npm to use those, instead of rebuild, to minimize potential conflicts.

@pizzamig

I am collecting all attempts so far to make FreeBSD nodejs friendly. Do you have any material I could link to?

So far I have found three attempts to do that, I link relevant material in Pinboard under

https://pinboard.in/t:nodejs/t:porting/t:FreeBSD/