The issue described above is no news in the Node.js world. More
specifically is an issue in Node's most common package manager, npm.
If the package manager can't warrant determinism, different developers
may be developing under different environments. This is also a problem
for continuous integration tools. And this is why Facebook developed
their own package manager as a replacement for npm, Yarn. Also, this
is the reason of important changes in latest npm release, v5
has seen light a few days ago.
I set my final goal to: have one source entry per Node dependency in
the flatpak manifest. The tarballs for each Node package are
available either in https://registry.npmjs.org for npm and
https://registry.yarnpkg.com for Yarn.
So I decided to take a look at both, Yarn and npm v5. This blog post
was a good start to understand their different takes at determinism.
Both are similar in that they translate the
package.json with the
project requirements to a
yarn.lock (Yarn) or
(npm v5). This lock file has a list of each dependency, a flattened
version of the tree of dependencies. For example here is the head of
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
As you can see, there is a URL to each tarball, plus a checksum.
First I tried npm v5 and the offline install worked. But the offline
mode works using a cache directory, not with the tarballs like I
Yarn on the other hand does store the tarballs in a directory, and
uses that directory to skip downloads if instructed to do so. They
call this directory the "Offline mirror". Note Yarn also has a
caching mechanism but the offline install will still work if the cache
With that, I was able to integrate Yarn with Flatpak as follows:
- I converted the flatpak manifest into a template.
- Created a script that parses the
yarn.lock looking for URLs to
tarballs and adds them as sources into the template, creating the
final flatpak manifest.
- Because Yarn uses SHA1 and flatpak uses SHA256, the script also
validates the downloaded files with the SHA1 and calculates their
SHA256, which is also added to the manifest.
The resulting manifest has a long list of files. The following is an
extract that corresponds with the piece of lock file above:
With the Yarn offline mirror recreated in the build directory, the
application can now be installed paassing
--offline in the Makefile:
cd electron && node yarn.js install --non-interactive --offline --verbose
cd electron && node yarn.js run pack --non-interactive --offline
Actually this build will hang at
yarn install --non-interactive,