Thoughts on APIs stability

2023-02-19 | 6.5 min reading

I saw people in terror in front of an npm update. Developers spending countless hours trying to figure out why the project they trying to setup didn’t work.

Did they use npm install instead of npm ci? Did someone forget to commit package-lock.json?

Most of this fear is rooted in API breakage which is killing the trust developers have in npm and the ecosystem.

How comes we’ve got so many fragile APIs and still, we fail to improve the situation?

I suspect the issue is cultural.

While it’s true that we work in a challenging environment — new js APIs, new css properties, browser fragmentation — we can not deny that the negative effects of breaking APIs are minimised while new versions and features grab the attention of the whole community.

Stable — instead of being welcome — is synonymous with dead.

This substrate is breeding ephemeral fragmented packages with constant breaking APIs.

Consequences

Direct impact

The main issue? Instead of writing new features, you’ll have to fight just to maintain the current functionalities — see Eduardo Ferro Aldama, Basal Cost of software.

Worst, the risk of assuming the application is in a working state while in reality, you have subtle errors in place.

Joel Spolsky enlightened very well the situation in his all-time classic Fire And Motion (not only the FE world suffers from the APIs instability plague).

Think of the history of data access strategies to come out of Microsoft. ODBC, RDO, DAO, ADO, OLEDB, now ADO.NET – All New! Are these technological imperatives? The result of an incompetent design group that needs to reinvent data access every goddamn year? (That’s probably it, actually.) But the end result is just cover fire. The competition has no choice but to spend all their time porting and keeping up, time that they can’t spend writing new features.
— Joel Spolsky

Externalities

In general, breaking APIs degrade the ecosystem.

Plugins and libraries get affected by the breaking changes. Some will be able to get along, others will die.

Tutorials, articles and questions on StackOverflow will be obsolete.

Long-time developers may find their mental model out of fashion and feel it’s time to migrate to a new library or piece of tech.

It’s important to notice how a community around a project enrich the quality of the project. Every developer lost, potentially is a developer who could have contributed to the cause.

What makes a breaking change

If we want to investigate the problem, we should start analysing what makes a breaking change.

At least, we’ve got 3 factors:

file path or file name change
function implementation change
function removal
configuration files change

file path or file name change

If you import a function directly from a specific file and then the filename change or the function move to a different file your code may break.

Let’s say you have a function manageAcmi in @acmi/acmi.js which is part of the @acmi package.

// in @acmi/acmi.js
export function manageAcmi() {
  .
  .
  .
}

// and then in your code you may have
import {manageAcmi} from @acmi/acmi

If now, manageAcmi gets moved to @acmi/manage.js for example, your code will break.

A bundler may spot this easily. It can be fixed at build time.

function implementation change

A function gets rewritten or changed.

Because of the change, given the same input, the output is now different.

Or maybe the function’s arguments change. The order can be inverted, extra arguments can be added.

This is probably the most insidious break.

A bundler may not immediately throw an error with the application appearing to work while having potential issues at runtime.

function removal

A function gets deleted. As per file path or file name change a bundler may spot this when using import.

The fix will present the challenge to re-create an equivalent of the deleted function.

configuration files change

Mostly driven by a newer version of node, many tools require a rewrite of the configuration files (e.g.: require / import, module.export / export).

As this type of change rarely is introduced in isolation, maintaining the current working status gets quite challenging.

With breaking plugins and alien extra syntax, the developer experience turns into a cortisol explosion.

After thought

The promise of semver is that you can let tooling _automatically_ update all dependencies that haven't bumped a major version. Thereby getting bug fixes, security patches (important!), and even new features, without having to manually review changelogs for hundreds/thousands of dependencies. I think that's basically it's whole point, the context of automatic tooling.

Is it perfect? No. Because some dependencies don't use semver; some maintainers make mistakes and there are 'bugs' in their release numbers; _and_ sometimes bugfixes and even security patches are available _only_ in a major release, so your automatic updates might miss them.
— jrochkind1

If you feel keen to jump in the semver rabbit hole read Why Semantic Versioning Isn't | Hacker News and Why Semantic Versioning Isn't (2015) | Hacker News

Now, I heard some devs saying: Hey, we do have semver to point out if we have a breaking change. They say this as semver was the solution of the problem.

Let me state it: semver is not a solution, is just contingency.

The most important issue with semver is the way a developers decide the bump of MAJOR, MINOR and PATCH.

The process is totally subjective and endless are the cases of a MINOR version introducing breaking changes.

Part of the problem could be mitigated with an Enforced Semantic Versioning, with code deciding the next version (still, we got the issue of “how” we would enforce it, as javascript is not strongly typed).

In addition, most of the packages stop supporting older versions after they release a major upgrade. You’re basically forced to adopt the new version or be on your own.

Very few packages are able to provide a decent experience on version migration.

jquery.migrate

A luxurious study case is jquery.migrate which patch the deprecated function and warns when they get used.

The library facilitates a gradual migration from jQuery 1.12 to 3.6 helping the developers to avoid nasty surprises.

Ideally, it would be nice having developers to think ahead about the lifecycle of the library and have respect for the APIs from the beginning.

So, what are the possibles alternative if you have to break an APIs?

using function suffixes instead of function changes (function duplication, this would actually prevent breaking the current API),
providing migration libraries (as per jquery.migrate),
writing migration guides and tutorials (CHANGELOG.md, README.md),

It’s inspiring to see projects like Clojure which are deeply aware of the problem that with a slow but solid approach avoids at any cost breaking APIs.

Rich Hickey made a similar (but more clear) case against semantic versioning in his Spec-ulation keynote https://www.youtube.com/watch?v=oyLBGkS5ICk
He advocates talking about "breakage" and "accretion" instead of the vague term "change".

His most controversial point, as far as I remember, was that any breaking change (i.e. a major version bump in semver) might as well be considered another product/package/lib and should better choose another name instead of pretending to be the same thing with a bumped number. For me that is taking it too far in most cases (e.g. when the intent of the software remains the same), since it would also have very disruptive effects on the community, documentation, reputation, ... of the package.

Another thought about semver: a bugfix requires only a minor version bump, but IMHO this could also be breaking if people were relying on the buggy behavior. I see the value of semver, but I guess it will always be a too-neat abstraction over something that is inherently too complex to communicate in a simple version string.
— vincentdm

Avoiding APIs breaks to bite you

If you’re trying to setup an old codebase without success, it may be worth using npm ci instead of npm install.

While npm install tries to install the latest compatible version of the package (real code may disagree with the semver number), npm ci will use package-lock.json to install the exact working version (as far as package-lock.json has been committed diligently).

Another strategy is to use nvm and have a .nvmrc file to specify the node version used.

Summary

The FE world is immersed in a constantly changing landscape: new js APIs, new css properties and browser fragmentation.

In addition, a culture that pushes for novelty and sees the stable as boring and dead is breading ephemeral fragmented packages with brittle APIs.

This generates an insecure ecosystem to build your software upon.

Innovation is necessary but all the possible attempts of having stable APIs must be pursued if we care about software quality and long-term value.

While tempting to name and shame, we focused instead on possible causes, solutions, ideas and consequences. In particular: Basal Cost of software, Enforced Semantic Versioning, using function suffixes, providing migration libraries and writing migration tutorials.

In the meanwhile, npm ci and nvm may help you in the long run to survive APIs break.

Last thought: wider the number of features, files and loc, higher the risk of having a breaking change. Think about it next time you feel the urge to npm install a package.

Evaluating the project spirit, core team and long-term vision may spare you from some cortisol in the long run.