Node-pre-gyp makes it easy to publish and install nodejs c addons from binaries


Usually this is no problem - but things get complicated with native addons are used. When installing an addon on a machine without a compiler, we are out of luck.

Sounds magical, and in some respects it is! Probably the first thing to know about node-pre-gyp is that it is simply a tool to automate what would normally be a tedious manual set of steps to properly download a pre-built binary.

It just makes doing the following easier:. Yes… I know Node. The eventual build files will select the appropriate one for the intended platform see below. Thanks to Nadeau Software for the code that I adapted to make this example! The contents of index. At this point, we need to add a package. These files will end up changing a bit when we add node-pre-gyp support. Executing the project will result in the realization that the Node.

The goal of this post is now to turn this addon into a pre-built executable or set of executables that can be deployed to different platforms with the same ease-of-use, without the build step requirement. As described above, node-pre-gyp is a tool that makes it easy to deploy platform-specific binaries to a host i.

This means that as long as you pre-build and deploy your native addon for all of your supported platforms OS, architecture, Node. Next, we need to drop a new section into this same file to tell node-pre-gyp how to name the binaries that it will create. These are all automatically determined by node-pre-gyp when it build the binary packages to be deployed.

They go as follows:. The preinstall entry is just telling npm to install node-pre-gyp before doing anything else. The --fallback-to-build flag tells node-pre-gyp to do the full build if it cannot locate the required binaries the specified remote host.

Until we actually start deploying, this will always be the case, and the addon will build using the end-user local compiler as it has done before. Lastly, but perhaps most importantly, we need to modify how a program require ing this addon finds the main entry point. Recall our original package. This no longer holds though, now the binary addon will hopefully be found, pre-built, on a remote host.

This bit of code loads node-pre-gypshows it where the package. Code require ing this addon still works the same way - but now node-pre-gyp is locating the binary. The binary entry in package. Check out the binding. Your final file structure should look like this, with the only change being the new index. Do an npm install again, and you should see some new messages printing to the screen….

In the printout, you should see the node-pre-gyp install --fallback-to-build command being executed. Tried to download and node-pre-gyp ERR! Pre-built binaries not found. If all goes well, you can again run the addon with node index. Your first decision is what sort of host you want to use. You have two basic choices - an Amazon S3 bucket, or anything else. Github is a popular choice, but really any web host is fine etc. There is a module to automate a lot of the process when using github.

The advantage of using S3 buckets is that node-pre-gyp can handle the deployment publishing process for you, entirely. The basic steps are as follows:. You can keep all the default properties set. Login to IAM, and create a new policy.

Ensure all the necessary permissions are set, and be sure to set the resource to contain your new S3 bucket. Now attach the policy to either an existing or new user account.

I named this file node-pre-gyp. I node-pre-gyp makes it easy to publish and install nodejs c addons from binaries it to my. In order to do the publishing, we need to add aws-sdk to our dependencies. Now, we can publish by using node-pre-gyp directly. If you installed node-pre-gyp globally, you can use it by typing node-pre-gyp at the command line.

The npm install builds the package, the node-pre-gyp command packages makes the zip file and publishes it to S3 - you should see a message at the end giving you the URL where the package was published. Running this on a Mac gave me the following URL - https: Now, in the example directory, do a fresh npm install. The next step is to re-run the npm install and. It also means you likely need to do this with various versions of Node.

You may even go as far as different CPU architectures. This is outside the scope of this article, but is the next logical step. You can find the full source code for this in the nodecpp-demo repository, this example is found in the prebuilt directory.

Check out some of my eariler posts and my book for more help on that part. Sign up to for Node Addons newsletter so you find out when new articles are published right away. Cross-platform addons with node-pre-gyp 07 Feb Node. It just makes doing the following easier: During build development Automatically names built addon executables based on the current platform OSarchitecture i. Packages the executable into zipped payloads. Optionally automatically publishes the payload to an Amazon S3 bucket.

You can also manually upload them elsewhere. Addon setup Create a package. Setting up node-pre-gyp As described above, node-pre-gyp is a tool that makes it node-pre-gyp makes it easy to publish and install nodejs c addons from binaries to deploy platform-specific binaries to a host i.

They go as follows: This is where the binary build output will be placed locally, when we do an npm install on the addon before deployment. The name of the binary build output - both locally and remote. The variables being used are as follows: Release or Debug - you can pass —debug to set to Debug during build platform: Basically boils down to the OS - darwinlinuxor win This is pulled from process.

The version of your addon - this is derived from the package. This is derived from the Node. Node-pre-gyp makes it easy to publish and install nodejs c addons from binaries developed for different Node. This is automatically detected for you. The basic steps are as follows: Now publish like this: You should be able to download the zip node-pre-gyp makes it easy to publish and install nodejs c addons from binaries.

For a hello world example of a module packaged with node-pre-gyp see https: Both --build-from-source and --fallback-to-build can be passed alone or they can provide values. For a full example see node-addon-examples's package. But this does not behave predictably across all npm versions - see https: So we do not recommend using preinstall to install node-pre-gyp. Instead we recommend bundling. More history on this at https: The location your native module is placed after a build.

This should be an empty directory without other Javascript files. This entire directory will be packaged in the binary tarball. When installing from a remote package this directory will be overwritten with the contents of the node-pre-gyp makes it easy to publish and install nodejs c addons from binaries. A url to the remote location where you've published tarball binaries must be https not http.

Why then not require S3? Because while some applications using node-pre-gyp need to distribute binaries as large as MB, others might have very small binaries and might wish to store them in a GitHub repo. This is not recommended, but if an author really wants to host in a non-s3 location then it should be possible. It should also be mentioned that there is an optional and entirely separate npm module called node-pre-gyp-github which is intended to complement node-pre-gyp and be installed along with it.

It provides the ability to store and publish your binaries within your repositories GitHub Releases if you would rather not use S3 directly. Installation and usage instructions can be found herebut the basic premise is that instead of using the node-pre-gyp publish command you would node-pre-gyp makes it easy to publish and install nodejs c addons from binaries node-pre-gyp-github publish. It is recommended that you customize this property.

This is an extra path to use for publishing and finding remote tarballs. It is recommended to provide a value like. This is the versioned name of the remote tarball containing the binary. Then your remote tarball will be looked up at, for example, https: In this case you can just copy binaries to the new version behind the scenes like:. A new target must be added to binding. For a full example see node-addon-example's binding. For a full example see node-addon-example's index.

The --build-from-source tells node-pre-gyp to not look for a remote package and instead dispatch to node-gyp to build. Now node-pre-gyp should now also be installed as a local dependency so the command line tool it offers can be found at.

Now you need to publish builds for all the platforms and node versions you wish to support. This is best automated. Now publish your module to the npm registry. Users will now be able to install your module from a binary.

If a a binary was not available for a given platform and --fallback-to-build was used then node-gyp rebuild will be called to try to source compile the module.

N-API is Node runtime engine agnostic and guarantees modules created today will continue to run, without changes, into the future. Using node-pre-gyp with N-API projects requires a handful of additional configuration values and imposes some additional requirements.

Therefore, an N-API module must declare in its package. In addition, since multiple builds may be required for a single module, path and file names must be specified in way that avoids naming conflicts. An N-API modules must declare in its package. This is accomplished by including an napi-versions array property in the binary object. How this version number is communicated is described next.

Since node-pre-gyp fires off multiple operations for each request, it is essential that path and file names be created in such a way as to avoid collisions. This is accomplished by imposing additional path and file naming requirements. No problem if it's in both. For those who need them in legacy projects, two additional configuration values are available for all builds.

If N-API is not supported, this value is an empty string. These values are present for use in the binding. You can host wherever you choose but S3 is cheap, node-pre-gyp publish expects it, and S3 can be integrated well with Travis. Here is an approach to do this:. And have your key and secret key ready for writing to the bucket. It is recommended to create a IAM user with a policy that only gives permissions to the specific bucket you plan to publish to.

This can be done in the IAM console by: It should generate a policy like:. You can use any of the methods described at http: Or pass options in any way supported by RC. You may also need to specify the region if it is not explicit in the host value you use. The bucket can also be specified but it is optional because node-pre-gyp will detect it from the host value.

Appveyor can build binaries and publish the results per commit and supports:. For an example of doing this see node-sqlite3's appveyor.

Once you have committed an appveyor. Encrypt your S3 AWS keys by going to https: Just put node-pre-gyp package publish in your appveyor. You might wish to publish binaries only on a specific commit. To do this you could borrow from the Travis CI idea of commit keywords and add special handling for commit messages with [publish binary]:. If your commit message contains special characters e. An alternative is to use PowerShell, which gives you additional possibilities, like ignoring case by using ToLower:.

Remember this publishing is not the same as npm publish. We're just talking about the binary module here and not your entire npm package. Travis can push to S3 after a successful build and supports both:.

For an example of doing this node-pre-gyp makes it easy to publish and install nodejs c addons from binaries node-add-example's. See the node-sqlite3 scripts for an example of doing this.

More details on Travis encryption at http: Just put node-pre-gyp package publish in your. If you node-pre-gyp makes it easy to publish and install nodejs c addons from binaries binaries for OS X in addition to linux you can enable multi-os for Travis. Also create platform specific sections for any deps that node-pre-gyp makes it easy to publish and install nodejs c addons from binaries install. For example if you need libpng:.

For detailed multi-OS examples see node-mapnik and node-sqlite3. To do so you will need to:. Second, the OS X machines do not support using a matrix for installing different Node. So you need to bootstrap the installation of Node. You only want to run the publish node-pre-gyp makes it easy to publish and install nodejs c addons from binaries the push commit. To automate the publishing of your entire package to npm on Travis see http: The strings are evaluated by node-pre-gyp depending on your system and any custom build flags you passed.

The options are visible in the code at https: Using the npm config argument: Install v8-profiler from npm. Follow us at github. A variety of developer targeted commands for packaging, testing, and publishing binaries. A JavaScript module that can dynamically require your installed binary: Pass the target arch and override the host arch. Valid values are 'ia32','x64', or arm. Pass the target platform and override the host platform.

Valid values are linuxdarwinwin32sunosfreebsdopenbsdand aix. This is useful if: The larger app also depends on other modules installed with node-pre-gyp You only want to trigger a source compile for myapp and the other modules. Configuring This is a guide to configuring your module to use node-pre-gyp. DeleteObject "" s3: GetObject "" s3: GetObjectAcl "" s3:

For a hello world example of a module packaged with node-pre-gyp see https: See the Frequently Ask Questions. Both --build-from-source and --fallback-to-build can be passed alone or they can provide values.

For a full example see node-addon-examples's package. The location your native module is placed after a build. This should be an empty directory without other Javascript files. This entire directory will be packaged in the binary tarball. When installing from a remote package this directory will be overwritten with the contents of the tarball.

A url to the remote location where you've published tarball binaries must be https not http. Why then not require S3? Because while some applications using node-pre-gyp need to distribute binaries as large as MB, others might have very small binaries and might wish to store them in a GitHub repo.

This is not recommended, but if an author really wants to host in a non-s3 location then it should be possible. It should also be mentioned that there is an optional and entirely separate npm module called node-pre-gyp-github which is intended to complement node-pre-gyp and be installed along with it.

It provides the ability to store and publish your binaries within your repositories GitHub Releases if you would rather not use S3 directly.

Installation and usage instructions can be found herebut the basic premise is that instead of using the node-pre-gyp publish command you would use node-pre-gyp-github publish. It is recommended that you customize this property. This is an extra path to use for publishing and finding remote tarballs. It is recommended to provide a value like.

This is the versioned name of the remote tarball containing the binary. Then your remote tarball will be looked up at, for example, https: In this case you can just copy binaries to the new version behind the scenes like:.

A new target must be added to binding. For a full example see node-addon-example's binding. For a full example see node-addon-example's index. The --build-from-source tells node-pre-gyp to not look for a remote package and instead dispatch to node-gyp to build.

Now node-pre-gyp should now also be installed as a local dependency so the command line tool it offers can be found at. Now you need to publish builds for all the platforms and node versions you wish to support. This is best automated. Now publish your module to the npm registry. Users will now be able to install your module from a binary.

If a a binary was not available for a given platform and --fallback-to-build was used then node-gyp rebuild will be called to try to source compile the module. You can host wherever you choose but S3 is cheap, node-pre-gyp publish expects it, and S3 can be integrated well with Travis.

Here is an approach to do this:. And have your key and secret key ready for writing to the bucket. It is recommended to create a IAM user with a policy that only gives permissions to the specific bucket you plan to publish to.

This can be done in the IAM console by: It should generate a policy like:. You can use any of the methods described at http: Or pass options in any way supported by RC. You may also need to specify the region if it is not explicit in the host value you use. The bucket can also be specified but it is optional because node-pre-gyp will detect it from the host value. Appveyor can build binaries and publish the results per commit and supports:. For an example of doing this see node-sqlite3's appveyor.

Once you have committed an appveyor. Encrypt your S3 AWS keys by going to https: Just put node-pre-gyp package publish in your appveyor. You might wish to publish binaries only on a specific commit.

To do this you could borrow from the Travis CI idea of commit keywords and add special handling for commit messages with [publish binary]:. If your commit message contains special characters e. An alternative is to use PowerShell, which gives you additional possibilities, like ignoring case by using ToLower:. Remember this publishing is not the same as npm publish. We're just talking about the binary module here and not your entire npm package.

Travis can push to S3 after a successful build and supports both:. For an example of doing this see node-add-example's. See the node-sqlite3 scripts for an example of doing this. More details on Travis encryption at http: Just put node-pre-gyp package publish in your. If you want binaries for OS X in addition to linux you can enable multi-os for Travis. Also create platform specific sections for any deps that need install. For example if you need libpng:. For detailed multi-OS examples see node-mapnik and node-sqlite3.

To do so you will need to:. Second, the OS X machines do not support using a matrix for installing different Node. So you need to bootstrap the installation of Node. You only want to run the publish on the push commit. To automate the publishing of your entire package to npm on Travis see http: The strings are evaluated by node-pre-gyp depending on your system and any custom build flags you passed.

The options are visible in the code at https: Using the npm config argument: Install v8-profiler from npm. Name Last Update Last Commit. A variety of developer targeted commands for packaging, testing, and publishing binaries.

A JavaScript module that can dynamically require your installed binary: Pass the target arch and override the host arch. Valid values are 'ia32','x64', or arm. Pass the target platform and override the host platform. Valid values are linuxdarwinwin32sunosfreebsdopenbsdand aix.

This is useful if: The larger app also depends on other modules installed with node-pre-gyp You only want to trigger a source compile for myapp and the other modules. Configuring This is a guide to configuring your module to use node-pre-gyp. This property supports variables based on Versioning. It is highly recommended that you use Amazon S3.

Various node-pre-gyp commands like publish and info only work with an S3 host. S3 is a very solid hosting platform for distributing large files. We provide detail documentation for using S3 hosting with node-pre-gyp.

In this case you can just copy binaries to the new version behind the scenes like: Add a target like this at the end of your targets list: You have installed aws-sdk with npm install aws-sdk You have created a bucket already.

The host points to an S3 http or https endpoint. You have configured node-pre-gyp to read your S3 credentials see S3 hosting for details. You can also host your binaries elsewhere. To do this requires: You manually publish the binary created by the package command to an https endpoint Ensure that the host value points to your custom https endpoint. See Appveyor Automation for how to auto-publish builds on Windows. What will happen is this: S3 Hosting You can host wherever you choose but S3 is cheap, node-pre-gyp publish expects it, and S3 can be integrated well with Travis.

Here is an approach to do this: First, get setup locally and test the workflow: It should generate a policy like: