Native modules without bindings.gyp

[konsumer]

Hi, great & useful package.

I work on node-raylib and I am trying to help users make standalone games with it. I am not sure how to inline the native dep. It doesn't use bindings.gyp (node-raylib uses cmakejs.) Is it possible?

I might just be misunderstanding how addons works, I am not sure.

This works fine:

boxednode -s main.js -t hello

and produces hello which requires node_modules/raylib/build/Release/node-raylib.node in same dir to work. I'd like to inline it, for a single executable file (with no separate native module needed) so I tried this:

compileJSFileAsBinary({
  nodeVersionRange: process.versions.node,
  sourceFile: 'main.js',
  targetFile: 'hello'
  addons: [
    {
      path: 'node_modules/raylib',
      requireRegexp: /build\/Release\/node-raylib\.node/
    }
  ]
})

And I get this:

→  Preparing addon at node_modules/raylib ...
  ✖  Failed: Cannot read /var/folders/47/smf4dj454fd4mw881t380dz80000gs/T/boxednode/main/node-v20.0.0/deps/9752c49f4d546974ac7d079cc418da10/binding.gyp: ENOENT: no such file or directory, open '/var/folders/47/smf4dj454fd4mw881t380dz80000gs/T/boxednode/main/node-v20.0.0/deps/9752c49f4d546974ac7d079cc418da10/binding.gyp'
/Users/konsumer/Desktop/testbuild/node_modules/boxednode/lib/native-addons.js:18
        throw new Error(`Cannot read ${filename}: ${err.message}`);

Error: Cannot read /var/folders/47/smf4dj454fd4mw881t380dz80000gs/T/boxednode/main/node-v20.0.0/deps/9752c49f4d546974ac7d079cc418da10/binding.gyp: ENOENT: no such file or directory, open '/var/folders/47/smf4dj454fd4mw881t380dz80000gs/T/boxednode/main/node-v20.0.0/deps/9752c49f4d546974ac7d079cc418da10/binding.gyp'
    at loadGYPConfig (/Users/konsumer/Desktop/testbuild/node_modules/boxednode/lib/native-addons.js:18:15)
    at async modifyAddonGyp (/Users/konsumer/Desktop/testbuild/node_modules/boxednode/lib/native-addons.js:96:20)
    at async compileJSFileAsBinaryImpl (/Users/konsumer/Desktop/testbuild/node_modules/boxednode/lib/index.js:246:33)
    at async compileJSFileAsBinary (/Users/konsumer/Desktop/testbuild/node_modules/boxednode/lib/index.js:371:9)

Node.js v20.0.0

/var/folders/47/smf4dj454fd4mw881t380dz80000gs/T/boxednode/main/node-v20.0.0/deps/9752c49f4d546974ac7d079cc418da10 seems to point to raylib, so I guess the error is essentially saying "node-raylib does not use bindings.gyp" which is true:

bin/
build/
drm/
man/
node_modules/
src/
tools/
CMakeLists.txt
LICENSE.md
README.md
index.js
package-lock.json
package.json

[addaleax]

Hi @konsumer! The quick summary is that yes, currently, we're relying on the ability to modify binding.gyp files during the compilation process in order so that we can make some specific adjustments to it in order to be able to consume it as a static library, rather than a dynamic one.

Some parts of this could probably be fairly easily translated to CMake, but one particularly tricky bit is that the linking to the core generated binary happens by adding the addon's binding.gyp as a GYP dependency to the main binary's .gyp file. It's probably possible to work around that, but that's part of what makes letting go of gyp as a system tricky.

Ideally we'd want the ability to include addons generated from other build systems (also for our own company purposes), but until we get to it (which may be a long time) or somebody else picks this up, the way to consume non-.gyp addons would be to build and ship them separately, unfortunately.

(Note that this trickiness is part of why the experimental Node.js built-in single-executable work also didn't include native addons in its scope.)

[konsumer]

Yeh, that makes sense. thanks for the reply!

I experimented a bit with built-in single-execetable stuff here and managed to get it working pretty well with mac & linux (but not windows, not sure why.)

The upside of that method is no rebuilding of the exe, you just package it up as a bundle and inject it into the already-compiled exe.

I used bpkg to bundle the native lib in the js (it extracts them at runtime.) On windows it seems to extract it wrong, which I think is maybe a bpkg bug.

[addaleax]

@konsumer Yeah, extracting at runtime is definitely a possibility here, and that's okay, it's just not something that boxednode supports right now (i.e. it's aiming to be a "true" single-executable creator). It's probably possible to work out some operating-system-specific hacks to get this to work, though.

[konsumer]

bpkg might still help with that, as it inlines the native modules directly, I am not sure. The original purpose was to make a fully-inlined single JS file, which is what I feed into SEA worfkflow.