Replacing Cypress with uvu, linkedom and cssom

I recently replaced Cypress with a few smaller tools, and the test runtime for an open source monorepo I work on went from 35 seconds to 4 seconds. I'll expand a bit here on the thought process and how the migration went.

Reevaluating Cypress

If you pop open the package.json of any prominent project in the JavaScript ecosystem today, it would be no surprise to see Cypress listed as a dev dependency.

There's a reason for that. You get a solid developer experience and a huge API surface that very likely includes whatever use case you require.

But, you have to accept these drawbacks:

So for PRPL, a monorepo with a half dozen modules (including a local development server), a couple dozen test cases, and (probably) no users other than me, I asked myself:

Do the tradeoffs Cypress present still make sense given my goal of fuss-free long term maintenance?

Defining current behavior

The first step was to figure out exactly what Cypress was solving for me. This is what I came up with:

Breaking it down like this made it clear that I may not need the (admittedly convenient) kitchen sink that Cypress offers.

Well scoped replacements

I had two requirements for any new tools that could replace Cypress:

First and foremost, I needed a new test runner. Then, in order to assert against output files reliably, I needed to be able to parse HTML and CSS strings into DOM and CSSOM documents. These are the modules I settled on:

The authors are diligent about what dependencies (if any) they adopt, and opinionated about what is and isn't a concern of the module. I'm a fan!

Migration

See this PR for the feature parity replacement.

Let's look at a few tests before and after replacement.

Testing page interpolation

This is what the Cypress test looked like for PRPL page interpolation:

const content = {
  a: {
    id: 'a',
    title: 'A',
    slug: 'notes/a',
    date: '2020-01-01',
    description: 'A description',
    body: 'A body'
  },
  b: {
    id: 'b',
    title: 'B',
    slug: 'notes/b',
    date: '2020-01-02',
    description: 'B description',
    body: 'B body'
  }
};

const { a, b } = content;

describe('PRPL pages', () => {
  it('should interpolate templates', () => {
    cy.visit(a.slug);

    cy.get('[data-cy=page-meta-title]').should('have.text', a.title);
    cy.get('[data-cy=page-title]').should('have.text', a.title);
    cy.get('[data-cy=page-date]').should('have.text', a.date);
    cy.get('[data-cy=page-body]').should('include.text', a.body);

    cy.visit(b.slug);

    cy.get('[data-cy=page-meta-title]').should('have.text', b.title);
    cy.get('[data-cy=page-title]').should('have.text', b.title);
    cy.get('[data-cy=page-date]').should('have.text', b.date);
    cy.get('[data-cy=page-body]').should('include.text', b.body);
  });
});

There are no imports or setup/teardown hooks whatsoever. The tests use Cypress' utilities to visit a page, grab a DOM node and run an assertion against it.

Now, the equivalent using uvu and linkedom:

import { test } from 'uvu';
import * as assert from 'uvu/assert';
import { constructDOM } from '../../utils/construct-dom.js';

const page = {
  a: {
    id: 'a',
    title: 'A',
    date: '2020-01-01',
    body: 'A body',
    other: 'Misc'
  },
  b: {
    id: 'b',
    title: 'B',
    date: '2020-01-02',
    body: 'B body',
    other: 'Misc'
  }
};

const document = {};

test.before(async () => {
  document.a = await constructDOM('notes/a.html');
  document.b = await constructDOM('notes/b.html');
});

test('should interpolate templates', () => {
  assert.is(document.a.querySelector('[data-cy=page-meta-title]').textContent, page.a.title);
  assert.is(document.a.querySelector('[data-cy=page-title]').textContent, page.a.title);
  assert.is(document.a.querySelector('[data-cy=page-date]').textContent, page.a.date);
  assert.is(document.a.querySelector('[data-cy=page-body] > p').textContent, page.a.body);

  assert.is(document.b.querySelector('[data-cy=page-meta-title]').textContent, page.b.title);
  assert.is(document.b.querySelector('[data-cy=page-title]').textContent, page.b.title);
  assert.is(document.b.querySelector('[data-cy=page-date]').textContent, page.b.date);
  assert.is(document.b.querySelector('[data-cy=page-body] > p').textContent, page.b.body);
});

test.run();

The interesting piece here is the import of the constructDOM utility, which uses linkedom and looks like this (it's changed since the PR but the concept is the same):

import path from 'path';
import { readFile } from 'fs/promises';
import { DOMParser } from 'linkedom';

async function constructDOM(filePath, mimeType = 'text/html') {
  try {
    const buffer = await readFile(path.resolve(`../test-site/dist/${filePath}`));
    const string = buffer.toString();
    const parser = new DOMParser();
    return parser.parseFromString(string, mimeType);
  } catch (error) {
    console.error(error);
  }
}

export { constructDOM };

We use linkedom to create a DOM from the built HTML and assert against it, pretty neat!

Testing the CSS imports plugin

PRPL has a plugin that resolves CSS imports at build time. The Cypress version of the test for it looked like this:

const pages = {
  testCSSImports: {
    route: 'plugin-css-imports',
    CSSFileName: 'plugin-css-imports.css'
  }
};

const { testCSSImports } = pages;

const CSSFetchRequestAlias = 'CSSFileRequest';

describe('CSS imports plugin', () => {
  // Force all requests to not return from cache
  before(() => {
    cy.intercept(`/${testCSSImports.CSSFileName}`, { middleware: true }, (req) => {
      req.on('before:response', (res) => {
        res.headers['cache-control'] = 'no-store';
      });
    });
  });

  it('should interpolate CSS imports', () => {
    cy.intercept('GET', `/${testCSSImports.CSSFileName}`).as(CSSFetchRequestAlias);

    cy.visit(testCSSImports.route);

    cy.wait(`@${CSSFetchRequestAlias}`)
      .its('response.body')
      .should('not.include', '@import')
      .and('include', 'h1');
  });
});

Again no imports or setup/teardown, but it does make clever use of Cypress' ability to wait for particular network requests and assert against them. It required a bit of a hack to disable browser caching as well.

Here's the version using uvu and cssom:

import { test } from 'uvu';
import * as assert from 'uvu/assert';
import { constructCSSOM } from '../../utils/construct-cssom.js';

let cssom;

test.before(async () => {
  cssom = await constructCSSOM('plugin-css-imports.css');
});

test('should interpolate CSS imports', () => {
  const [firstRule, secondRule] = cssom.cssRules;
  assert.is(firstRule.selectorText, 'h1');
  assert.is(secondRule.selectorText, '.plugin-css-imports');
});

test.run();

Given the stretch I had to make to test this sort of thing with Cypress, this implementation is much more straightforward by comparison.

Instead of trying to make sure the browser doesn't make extra requests for CSS imports at runtime, we check instead that the output CSS file has all CSS imports resolved into the respective file.

For completeness, here's what the constructCSSOM utility looks like:

import path from 'path';
import { readFile } from 'fs/promises';
import { parse as parseCSS } from 'cssom';

async function constructCSSOM(filePath) {
  try {
    const buffer = await readFile(path.resolve(`../test-site/dist/${filePath}`));
    const css = buffer.toString();
    return parseCSS(css);
  } catch (error) {
    console.error(error);
  }
}

export { constructCSSOM };

Testing the server

The Cypress test suite didn't yet have tests written for PRPL's server module, so there's nothing to compare against. In a later PR I did add these tests using the new tools. I thought I'd include it here because the implementation is somewhat interesting.

Let's look at just the test case modified HTML source files:

import { test } from 'uvu';
import * as assert from 'uvu/assert';
import { constructDOM } from '../../utils/construct-dom.js';
import { constructCSSOM } from '../../utils/construct-cssom.js';
import { fetch } from '../../utils/fetch.js';
import { listenForChange } from '../../utils/listen-for-change.js';
import { writeSiteFile } from '../../utils/write-site-file.js';
import { readFile, writeFile } from 'fs/promises';
import { resolve } from 'path';

let currentModified;

test.before(async () => {
  const { lastModified } = await fetch('/');
  currentModified = lastModified;
});

const files = { 'index.html': null, 'index.css': null, 'index.js': null };

test.before.each(async () => {
  for (const file in files) {
    files[file] = await readFile(resolve(`sites/server/src/${file}`));
  }
});

test.after.each(async () => {
  for (const file in files) {
    await writeFile(resolve(`sites/server/src/${file}`), files[file]);
    files[file] = null;
  }
});

const edited = 'I was edited and reloaded by PRPL server';

test('should update if a source HTML file is changed', async () => {
  const file = 'index.html';

  const { document: srcDom } = await constructDOM({ src: `server/src/${file}` });
  srcDom.querySelector('h1').textContent = edited;
  await writeSiteFile({ target: `server/src/${file}`, om: srcDom });

  const { changed, data: html } = await listenForChange('/', currentModified);
  assert.ok(changed);

  const { document: serverDom } = await constructDOM({ src: html, type: 'string' });
  assert.equal(serverDom.querySelector('h1').textContent, edited);
});

test.run();

On a high level, what happens in the setup/teardown hooks is:

And during each test (while the server is running), we:

Arguably the most interesting part of this is the polling approach, listenForChange, which looks like:

import { fetch } from './fetch.js';
import { wait } from './wait.js';

async function listenForChange(filePath, currentModified) {
  let changedAt;
  let changed = false;
  let data;

  for (let i = 0; i < 30; i++) {
    await wait(100);

    const { data: fetchedData, lastModified } = await fetch(filePath);

    if (currentModified !== lastModified) {
      changedAt = lastModified;
      changed = true;
      data = fetchedData;
      break;
    }
  }

  return { changed, changedAt, data };
}

export { listenForChange };

In a nutshell, the function makes a request every 100 milliseconds and checks the last-modified header to see if the server has modified the file. If it did, send the data back so it can be asserted against.

Brief retrospective

So, we've walked through the approach to this replacement project and also taken a tour of some of the more interesting parts of the test migration.

Looking back on it, you could argue that the amount of time spent re-solving problems that Cypress solves out of the box isn't worth the gain of a faster test runtime.

In a corporate environment, I'd say you're probably right. But for PRPL, a small open source project that I maintain, I had fun putting in the hours. Whether a solution like this is right for your project, well, as always, it depends.

Thank you so much for joining me for the ride if you made it this far, hopefully I'll see you in the next note!

Thanks for reading! Go home for more notes.