Reproducible install-less execution of package executables (pnpm dlx)

[jonathanmorley]

Context

Execution Environments

We have CI pipelines (GitHub Actions) in which we want to execute an executable from some npm package (e.g. eslint). These pipelines will be executed against a large number of pnpm-compatible repositories.

Reproducibility

Given the security and compatibility risks of running unpinned packages, we prefer all dependencies to be 'locked' somehow.

User Control

Repositories may or may not want to exert control over the versions of the packages used.

For example, a repository may have some eslint v8 configs, and want to signal that eslint@^8 should be used when linting the repository. Similarly, a repo with v9-compatible configs wouls want to signal that eslint@>9 should be used.

On the other hand, a repository may use the defaults, not have any configuration, and may want to explicitly say that it will defer the choice of eslint version to whatever the pipeline decides.

Current Situation

We currently enforce that all repositories declare a devDependency on all tools invoked by the pipeline. This gives us reproducibility, as we can use the repository's lockfile to pin the versions of the tool and its dependencies. It gives us partial user control (repositories can choose the version they need).

What we do not have with this is:

• Repositories cannot defer the choice of version to the pipeline (they must have concrete versions expressed in a lockfile).
• We have to do a full pnpm install every time we want to run a simple tool like eslint. In a monorepo, this can get time-expensive. Additionally, there are additional setup complexities; while eslint is available on npmjs.org, other packages installed in the repository may not be (packages hosted by our internal package registery). To make these available, we set up networking routes and credentials. These at the very least feel like overkill for a single tool call.

Using pnpm dlx package@version as it currently is, would solve for the need to pnpm install every time, but, as I understand it, would not result in a reproducible installation. While package would be pinned at version, any dependencies of package would be installed at the highest version within the range that brings them in. It also does not allow repositories to choose the version they want.

Potential Solutions

I will mention https://github.com/orgs/pnpm/discussions/7447 here, as I think its desires overlap with mine, but I will propose a different solution, which I acknowledge has limitations, but can help illustrate an interface that I think would work.

Two extensions to the pnpm dlx command that:

1. Add a --respect-lockfile argument
2. Add a --hashlock argument

--respect-lockfile would mean that when pnpm dlx executes, it would check if it is in a workspace with a lockfile. If so, it would check whether the packages being invoked are in that lockfile. If so, it would use that (pinned) version of the package, and its dependencies.

--hashlock is a bit more complicated. Conceptually, I want some way to have a 'portable' lockfile that can be included in the command execution. I have a proof of concept repository at https://github.com/jonathanmorley/hashlock (full disclosure, vibecoded in about 20m) that can:

1. Generate all possible trees of 'lockfiles' for a package/version.
2. Create a hash for each of those lockfiles.
3. Look up a lockfile by hash.
   While this has obvious scaling and performance issues (this has a crazy big-O), I hope it can help show what I am thinking of by 'portable lockfiles'.

By using pnpm dlx, this solution might run into those same plugin issues that https://github.com/orgs/pnpm/discussions/7447 talks about. I don't know the details there, and haven't considered a solution that would address that.