Replacing Lerna with npm builtins

I've used Lerna for years at work and in open source projects. It's a great tool for managing monorepos.

It also does a lot of things, and an npm install --save-dev installs 614 packages. The last commit in the Lerna GitHub repo (at the time of writing) was June 2021. It's unclear whether it's still maintained.

As part of an effort to apply the same standards that I apply to the source code of PRPL to the tooling in the monorepo, I decided to replace Lerna.

In this note I'll share a bit about what went into this project.

Identifying current behavior

I went through the exact same process I went through to replace Cypress. The first step is to identify what exactly Lerna does for me today:

  1. PRPL makes use of Lerna's bootstrap and exec commands to install and hoist package node modules.

The npm script looks like:

lerna bootstrap && lerna exec npm install
  1. PRPL makes use of Lerna's publish command to bump package versions, publish to npm, and generate changelogs automatically.

The npm script looks like:

npm run build && lerna publish

Replacing bootstrap

See this PR for the feature parity replacement.

Solving for the first problem was straightforward.

In the time since Lerna released its first major version six years ago, the major package managers of the JavaScript ecosystem have introduced a feature called workspaces that solve the same problems that lerna bootstrap does.

I like using the builtin tools that platforms offer where possible, so naturally I reached for npm workspaces.

This is all I needed to add in the root package.json:

"workspaces": [
  "packages/*"
  "packages/*",
  "examples/*",
  "tests",
]

After defining the workspaces, running npm install in the root of the project installs all dependencies of workspace modules and hoists them at the project root. That's it!

Replacing publish

Replacing lerna publish was less straightforward and can be broken down into three sub-problems: versioning, publishing to npm, and changelog generation.

Versioning

In the case of versioning, npm has a builtin version command that does that, but it's much less smart than what Lerna does.

Lerna will automagically decide for you based on your changes what the next version of a package should be, while npm requires you tell it that explicitly.

Well, unlike some much larger monorepos out there, PRPL has only a dozen or so modules to deal with and probably not many more on the horizon.

I decided that this was an acceptable solution, and wrote a bash script to make it just a little more elegant for cases when I needed to bump all packages at the same time:

# Bump all package versions.

# See https://docs.npmjs.com/cli/v8/commands/npm-version#synopsis

# Example usage:
# `npm run version patch`

bump=$1 # e.g. major, minor, patch

for pkg in packages/*; do
  cd $pkg
  npm version $bump
  cd ../..
done

Publishing

The same is true of publishing, npm has a less smart builtin publish command.

Lerna would publish all modules for you in a single command, but I couldn't figure out a way to do that with npm. Back to writing bash:

# Publish packages.

# Example usage:
#  - `npm run publish [OTP]`
#  - `npm run publish dry-run`
#  - `npm run publish [OTP] core server`
#  - `npm run publish dry-run core server`

# Does not automatically bump versions or write changelogs, do this prior to running this script.

pkgs=$@
run_state=""
otp=""

if [ $# -eq 0 ]; then
   echo "Publish command should have either 'dry-run' or an OTP as a first positional argument, exiting."
   exit 0
fi

if [ "$1" == "dry-run" ]; then
   pkgs="${pkgs/"$1"/}"
   run_state="--$1"
else
   pkgs="${pkgs/"$1"/}"
   otp="--otp=$1"
fi

if [ "$pkgs" == "" ]; then
   for pkg in packages/*; do
      npm publish $run_state $otp --workspace=$pkg
   done
   exit 0
fi

for pkg in $pkgs; do
   npm publish $run_state $otp --workspace "packages/${pkg}"
done

I didn't bother making the arguments non-positional, and I also didn't solve the potential scenario of the OTP expiring before all modules are published. This is good enough.

Changelog generation

After a detour through the conventional changelog repository, which has many modules for solving the individual problems of generating changelogs automatically from conventional commits, I was tired.

I decided that PRPL is small and stable enough that I can deal with writing them manually. Maybe someday I'll automate this again, but not today.

One interesting find is that Lerna's default changelog generation creates files that say this at the top:

All notable changes to this project will be documented in this file.

Then it goes on to flood the file with messages like this for every new patch:

Note: Version bump only for package @prpl/plugin-sitemap

So, I deleted all those messages. When I write them manually, I'll only include the meaningful refactors, fixes and features.

A fun side effect of using npm workspaces is that I no longer have to use npm link to test out packages. Well, only when testing in another workspace in the monorepo at least.

Here's what my dev script looked like before (by now you've realised I like bash):

# Develop a package - e.g. `npm run dev -- core server plugin-rss`

pkgs=$@
comma_separated_pkgs=${pkgs// /,}

trap cleanup EXIT

function cleanup() {
  for pkg in $pkgs
  do  
    cd packages/$pkg && npm unlink -g @prpl/$pkg --silent && cd ../..
  done
  printf "\n\nGlobally linked modules:\n\n"
  npm ls -g --depth=0 --link=true
}

for pkg in $pkgs
do  
  cd packages/$pkg && npm link --silent && cd ../..
done

npx rollup -w -c rollup.config.js --scope=$comma_separated_pkgs

It would run npm link and rollup in watch mode for the packages you specify, and through the use of a trap unlink the modules when you killed the process.

After moving to workspaces, I reduced it to:

# Develop packages.

# Example usage:
#  - `npm run dev core`
#  - `npm run dev core server`

pkgs=$@
comma_separated_pkgs=${pkgs// /,}

npx rollup -w -c rollup.config.js --scope=$comma_separated_pkgs

And that's it! Now only the sturdy rollup module and its watch mode is required to rebuild packages when there are changes.

Bonus x2: Documentation

After all this I decided to create two new documents so that I don't forget everything:

I used to have this info in the main README, but if we're honest that file is mostly for people considering using PRPL, not contributing to it.

Retrospective

And there we have it! Now if you git clone the PRPL monorepo and npm install, you'll be downloading 614 fewer modules to your file system.

Thanks for joining me on this esoteric journey about monorepo tooling. Until next time, farewell!

Thanks for reading! Go home for more notes.