Pure ESM package
source link: https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Pure ESM package
The package linked to from here is now pure ESM. It cannot be require()'d from CommonJS.
This means you have the following choices:
- Use ESM yourself. (preferred)
Useimport foo from 'foo'instead ofconst foo = require('foo')to import the package. You also need to put"type": "module"in your package.json and more. Follow the below guide. - If the package is used in an async context, you could use
await import(…)from CommonJS instead ofrequire(…). - Stay on the existing version of the package until you can move to ESM.
You also need to make sure you're on the latest minor version of Node.js. At minimum Node.js 12.20, 14.14, or 16.0.
I would strongly recommend moving to ESM. ESM can still import CommonJS packages, but CommonJS packages cannot import ESM packages synchronously.
ESM is natively supported by Node.js 12 and later.
You can read more about my ESM plans.
My repos are not the place to ask ESM/TypeScript/Webpack/Jest/ts-node/CRA support questions.
How can I move my CommonJS project to ESM?
- Add
"type": "module"to your package.json. - Replace
"main": "index.js"with"exports": "./index.js"in your package.json. - Update the
"engines"field in package.json to Node.js 12:"node": "^12.20.0 || ^14.13.1 || >=16.0.0". - Remove
'use strict';from all JavaScript files. - Replace all
require()/module.exportwithimport/export. - Use only full relative file paths for imports:
import x from '.';→import x from './index.js';. - If you have a TypeScript type definition (for example,
index.d.ts), update it to use ESM imports/exports. - Optional but recommended, use the
node:protocol for imports.
Sidenote: If you're looking for guidance on how to add types to your JavaScript package, check out my guide.
Can I import ESM packages in my TypeScript project?
Yes, but you need to convert your project to output ESM. See below.
How can I make my TypeScript project output ESM?
- Add
"type": "module"to your package.json. - Replace
"main": "index.js"with"exports": "./index.js"in your package.json. - Update the
"engines"field in package.json to Node.js 12:"node": "^12.20.0 || ^14.13.1 || >=16.0.0". - Add
"module": "ES2020"to your tsconfig.json. - Use only full relative file paths for imports:
import x from '.';→import x from './index.js';. - Remove
namespaceusage and useexportinstead. - Optional but recommended, use the
node:protocol for imports. - You must use a
.jsextension in relative imports even though you're importing.tsfiles.
If you use ts-node, follow this guide.
How can I import ESM in Electron?
Electron doesn't yet support ESM natively.
You have the following options:
- Stay on the previous version of the package in question.
- Bundle your dependencies with Webpack into a CommonJS bundle.
- Use the
esmpackage.
I'm having problems with ESM and Webpack
The problem is either Webpack or your Webpack configuration. First, ensure you are on the latest version of Webpack. Please don't open an issue on my repo. Try asking on Stack Overflow or open an issue the Webpack repo.
I'm having problems with ESM and Next.js
You must enable the experimental support for ESM.
I'm having problems with ESM and Jest
Read this first. The problem is either Jest (#9771) or your Jest configuration. First, ensure you are on the latest version of Jest. Please don't open an issue on my repo. Try asking on Stack Overflow or open an issue the Jest repo.
I'm having problems with ESM and TypeScript
If you have decided to make your project ESM ("type": "module" in your package.json), make sure you have "module": "ES2020" in your tsconfig.json and that all your import statements to local files use the .js extension, not .ts or no extension.
I'm having problems with ESM and ts-node
Follow this guide and ensure you are on the latest version of ts-node.
I'm having problems with ESM and Create React App
Create React App doesn't yet fully support ESM. I would recommend opening an issue on their repo with the problem you have encountered. One known issue is #10933.
How can I use TypeScript with AVA for an ESM project?
Follow this guide.
How can I make sure I don't accidentally use CommonJS-specific conventions?
We got you covered with this ESLint rule. You should also use this rule.
What do I use instead of __dirname and __filename?
import {fileURLToPath} from 'node:url';
import path from 'node:path';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(fileURLToPath(import.meta.url));However, in most cases, this is better:
import {fileURLToPath} from 'node:url';
const foo = fileURLToPath(new URL('foo.js', import.meta.url));And many Node.js APIs accept URL directly, so you can just do this:
const foo = new URL('foo.js', import.meta.url);How can I import a module and bypass the cache for testing?
There's no good way to do this yet. Not until we get ESM loader hooks. For now, this snippet can be useful:
const importFresh = async modulePath => import(`${modulePath}?x=${new Date()}`);
const chalk = (await importFresh('chalk')).default;Note: This will cause memory leaks, so only use it for testing, not in production. Also, it will only reload the imported module, not its dependencies.
How can I import JSON?
JavaScript Modules will eventually get native support for JSON, but for now, you can do this:
import {promises as fs} from 'node:fs';
const packageJson = JSON.parse(await fs.readFile('package.json'));If you target Node.js 14 or later, you can import it using import fs from 'node:fs/promises'; instead.
When should I use a default export or named exports?
My general rule is that if something exports a single main thing, it should be a default export.
Keep in mind that you can combine a default export with named exports when it makes sense:
import readJson, {JSONError} from 'read-json';Here, we had exported the main thing readJson, but we also exported an error as a named export.
Asynchronous and synchronous API
If your package has both an asynchronous and synchronous main API, I would recommend using named exports:
import {readJson, readJsonSync} from 'read-json';This makes it clear to the reader that the package exports multiple main APIs. We also follow the Node.js convention of suffixing the synchronous API with Sync.
Readable named exports
I have noticed a bad pattern of packages using overly generic names for named exports:
import {parse} from 'parse-json';This forces the consumer to either accept the ambiguous name (which might cause naming conflicts) or rename it:
import {parse as parseJson} from 'parse-json';Instead, make it easy for the user:
import {parseJson} from 'parse-json';Examples
With ESM, I now prefer descriptive named exports more often than a namespace default export:
CommonJS (before):
const isStream = require('is-stream');
isStream.writable(…);ESM (now):
import {isWritableStream} from 'is-stream';
isWritableStream(…);@jimmywarting just to clarify my phrasing, I didn’t mean top-level await, I meant the app that is ultimately at the top of the dependency tree, the consumer of these libraries, such as servers or really just the kind of thing that isn’t depended on itself, but it depends on smaller libraries, such as the ones Sindre makes.
@RDIL Indeed, you are correct. And herein lies the problem. I have such an app. It has 529 JS files. All are Common JS. I use dynamic require (eg: for (device in deviceList) { deviceModule[device.name] = require(./devices/${device.name}) - that kinda thing). I have the occasional require in a code block at the top level (eg: if (process.env.TEST) require('foo') - that kinda thing).
We can moan about all this as much as we like. @sindresorhus isn't going to change their approach. Others will follow. It is, as they say, what it is. So the question is not "why are you doing this", it's "how do we cope?" (ignoring the obvious starting answer of "use other modules" because that will work today but become increasingly difficult over time - and does nothing to solve the fundamental problem of not migrating).
I'm in a corporate environment. Nobody is going to stop product development to let us take a month to modify everything, retest, etc. Every file needs to be edited. Several fundamental ways we do things have to change - eg I have stuff like: require('routes/common/feature')('Group') and then require('routes/common/feature')('User') which return classes generated by the module that are 99% similar.
So I'm left with one solution: Do it a bit at a time. Treat it as debt as update things as we touch them. Except that's not possible. Or, at least, nobody has given me any ideas on how to achieve that. If I want to upgrade chalk@4 to chalk@5, any file that uses it needs to be ESM. It's my logger. EVERY file uses it. So mjs/js tricks probably won't cut it. That means a single, one-time cutover from require to import and import(). ie a month of work.
Unless somebody here has any ideas on how to do this a bit at a time. I thought maybe I could work out my entire require hierarchy and then start at the top and rename it to .mjs and change every require to an import. And then completely refactor things like: require(config-${process.env.NODE_ENV}) somehow. Then I work down a little bit at a time. Eventually, in several months' time, I can switch package.json to module and rename every file back to .js and then (and only then) will I be able to use chalk@5. I think. It's mindbending but I think that is supposed to work. MJS can import CJS so start at the top and work down. And just deal with each disaster as it comes???
Does anybody here have any practical experience in actually doing this. I'm not talking about little modules loading other little modules. I'm talking about a single application with hundreds of files. Looking for a sane, one step at a time plan for migrating.
Does anybody here have any practical experience in actually doing this.
It might be easier than you think to come together as a team and migrate the whole project from CJS to ESM in a day or so. I've migrated projects from CJS to ESM that have hundreds of modules. It's boring and unpleasant but doable, and pretty satisfying once it's done.
Several fundamental ways we do things have to change
Yes, probably it's easiest to refactor those fundamental problems one at a time while the project is still CJS. When approaches are compatible with ESM, push the button and migrate all the CJS modules to ESM.
So I'm left with one solution: Do it a bit at a time.
Nah, get it done as fast as possible. It's the sort of thing that within a single project, is hard to do incrementally and is most efficient when doing stuff in bulk. Freeze PRs until the migration is complete.
Avoid attempting to use Babel or some other transpiler like a codemod to transform CJS syntax to ESM; it can add undesirable helpers, mess with formatting, move comments around, etc.
With clever regex in VS Code's find and replace to swap require and module.exports = for ESM syntax you can get the majority of the grunt work done in a few clicks, although I recommend mashing the replace button one at a time to make sure you didn't somehow mess up with the regex. I use macOS Finder app to search for .js files and mass rename them to .mjs in only a couple of clicks. There are ESLint plugins that automatically fix missing .mjs file extensions in import paths. Then spend a day or so refactoring things to fit the way ESM works regarding default vs named imports, dynamic imports, or dependencies that need updating/swapping.
Migration is a lot easier if you already understand ESM and the way things should be so you don't waste time floundering while ironing out the kinks. We've all had years to get used to understand ESM, Node.js ESM/CJS interop, how the package exports field works, etc. This stuff is central to our tradecraft and we've known all along the day would come we have to migrate.
I don't work for any company, but I would think that freezing the entire project for a day or 2 to transition to an entirely new module system just to update one dependency... I really don't think that's going to happen. Even with all migration guides, even with modern tooling, I just don't think they will do it, at least not any time soon.
I don't know exactly which company @KrayzeeKev works for, but I would guess that their thought process will be something similar to what I just laid out. I feel like the idea of ESM can't catch on until large scale projects adopt it, which will take time. It would be cool if it didn't, but that just doesn't seem to be the case.
TL;DR: it may be doable, but companies likely won't want to take the step of doing it.
If you work at a sweatshop that's so severely lacking in engineer leadership where a day or two spent on a valuable migration is unacceptable, then it's probably easier to just quit and work somewhere better. I'm serious; there are heaps of remote employment opportunities for engineers right now.
99% of the time when you hear a dev say "management would never go for that" they just lack confidence to state what needs to happen, like people in other departments do. Management regularly sends people off on week-long teamwork or certification courses, "business trips", and other bullshit time spends. Massive companies like Panasonic are rolling out 4 day working weeks, and lots of tech companies have mandatory "innovation days" each month where employees get to do whatever code they like. I just don't buy the whole "we don't have the time" thing for something so important. And it is important; it's the module system that glues all your code together and allows you to integrate packages from the wider ecosystem. Do nothing about this and your org will be cutoff from the best packages on npm.
I don't understand why some people aren't happy and excited about ESM; writing code all day is what we do, it's our purpose, and it just got better! Pure ESM packages are opening up whole new worlds of possibility… I've been working on a SSR web app framework that is zero build, has full type safety via JSDoc comments and TS, and uses pure ESM with import maps to do it. Rich GraphQL powered web apps with SSR with only 6 or so dependencies, instead of thousands you see in old-school node_modules, when you inspect the source in the browser that's the source code in your repo! Only the immediately useful code is downloaded and cached on demand; there is no waste over the network in bundles. This stuff is seriously fun. Working around CJS that will never work in the browser is super not fun.
People that think they are only migrating to ESM "just to update one dependency" don't understand the big picture about how overwhelmingly positive this is for the ecosystem.
I don't think us grumbling in this discussion are unhappy about the improvements ESM brings. I'm personally far from happy and excited about it but I understand that it's better for browsers in the long term. I think we would have hoped for a bit easier transition. The tooling wasn't ready a few months ago, it's getting better but it's still not perfect and difficult and cumbersome to setup.
I personally left my previous company that will not migrate to ESM anytime soon, and I started to write ESM projects at my new job. I left for many reasons, the lack of ESM migrations was not of them. A migration of an existing project to ESM is hard to justify from an engineering point of view in my opinion. We all have limited time at work, and migrating to ESM is not fun to everyone and breaks many things. You can have a great company with great coworkers who decided together that a migration was not worth the hassle.
It's probably easier faster to fork ESM-only packages and add CJS exports than it is to update existing project's to ESM. At least until the greater ecosystem supports ESM so... maybe 2 or 3 more years from now
It's probably easier faster to fork ESM-only packages and add CJS exports
With that mindset then cjs is only going to stick around for longer time period. Think it's just better to rip the bandage off
With that mindset then cjs is only going to stick around for longer time period. Think it's just better to rip the bandage off
If the esm-only packages kept exporting cjs too then there wouldn't be a problem in the first place.
What percentage of projects benefit from packages without any cjs? (none? seeing as you can export both) What percentage benefit from cjs compatible packages? (most?) There's no point except to bully people into wasting time re-architecting their projects to fit with a half finished esm ecosystem, half of which will revert back when they find out it's not ready. In my opition esm is the future but forcing people to migrate for no reason is weird and very annoying.
As someone also stuck with electron's lack of ESM support I made a tool to auto-convert ESM packages (and their dependencies recursively) to CommonJS and publish under a custom npm scope: https://github.com/mifi/commonify
Probably doesn't work on all ESM packages, but I tested on a few like lowdb and execa and it successfully converts the latest version!
It feels a bit wrong to have to do this, but I can't find any other good option :|
I'm thankful for Sindre's work over the years, quite amazing really but kinda reminds me of Lodash in that there's so many great alternatives out there that don't require me to rethink my entire application.
Ah well.
I am a huge fan of your work, @sindresorhus, but this just makes me sad.
If you’re a Node.js package maintainer, please consider setting aside some time in May to move your packages to ESM. It will benefit the whole ecosystem!
We're a "start-up" (non-VC) and trying to get our wheels off the ground and we have 40+ projects starting from node 6, a few still in node 10 but the most parts in node 12. Redoing everything to use ESM is simply not an option as that would be many, many hours of work (1000'ds probably with testing) and we simply can't afford that...
I can't find any indication that CJS would disappear any time soon (5+ years at least) and I would think that the node community needs to find a better approach at migrating CJS to ESM when that happens as there is going to be a lot more, and certainly bigger companies, than us in the same boat... The move to forces ESM feels forced and premature and reading a lot about it the only benefit I can agree on is the browser support (which we don't use).
I know you've made your choice and feel proud of that but please take into account that you might be causing a lot of issues for struggling companies continuing to drive the hard bargain to migrate to ESM... some might just not be able to do that...
We have to move on and find other options which is a shame, and will also cost us, but I wish you all the best and I hope we can come back when ESM is actually a thing and not some wild dream... :(
@QAnders Dual packages have their own problems. The alternative would have been to not move to ESM yet, but a lot of users were pushing me to move to ESM for better browser and tree-shaking support. There's really no way to please everyone. We also don't want to end up in a Python 3 situation where it takes more than decade to move.
@sindresorhus I've had no choice but to use dual packages due to lack of esm support in tooling (Eg jest). Jest was the deal breaker for me. I do agree with this shift to esm but I just wish the big players (electron, jest) would actually support it.
Redoing everything to use ESM is simply not an option as that would be many, many hours of work (1000'ds probably with testing) and we simply can't afford that...
It's not like you have to completely rewrite the apps. A lot of the ESM conversion can even be automated. Most of it is just converting require to import, exports, and handling __dirname. There are some blockers, like TypeScript and Jest, but they will soon have full ESM support too. And no one is forcing you to upgrade dependencies. You can just hold off for now. In my packages, I'm also backporting security fixes.
Many also don't realize you can use ESM from CommonJS by using await import(). The import then becomes async, but for Node.js projects, most things are async already, so it should not matter much.
I've had no choice but to use dual packages due to lack of esm support in tooling (Eg jest). Jest was the deal breaker for me. I do agree with this shift to esm but I just wish the big players (electron, jest) would actually support it.
Go complain to such tools (Electron, TypeScript, Jest, etc) rather than maintainers of minor open source packages. ESM should not come as a surprise. It's been in development for over 10 years. These tools had plenty of time to get ready, but "chose" not to (yes, I'm oversimplifying it).
@sindresorhus, not complaining, just stating an observation. I'll emphasise again that I agree with this change.
Sorry to kick the hornet's nest, @sindresorhus, and thanks for your response!
I do understand your standpoint and do respect that but IMO it's too soon to start pushing hard for ESM and getting developers riled up to start doing everything in ESM as there's (obviously) a lot of stuff not up to speed for it so I was merely commenting on your statement to other maintainers, not your own decision... :)
It puts a lot of us in a difficult position on where to turn and what to do as we're only half way there and there's no clear path so regardless of what we, "users" of maintained packages, do we'll have to cut through some jungle and hope we'll find the correct road on the other side...
EDIT:
We run most of our code in AWS, Lambda's mostly, and this was posted just Jan 6th: https://aws.amazon.com/about-aws/whats-new/2022/01/aws-lambda-es-modules-top-level-await-node-js-14/
So, yes, a lot of the big dragons are playing catch-up but we're a long way from being close to have ESM fully operational...
Copy link
joebowbeer commented on Feb 1
Compare with jose@4 approach?
https://github.com/panva/jose#documentation
An early attempt was made to push the envelope but was eventually abandoned
Dual packages can only have problems when a package is stateful or relies on identity. The vast majority of your packages do not qualify, so no such problems can possibly exist for those.
Tree-shaking also only provides a benefit when a module exports more than one thing. A package that exports one function, for example, needs no tree-shaking in any module system - there’s nothing to shake.
Copy link
KrayzeeKev commented on Feb 3
@sindresorhus "Many also don't realize you can use ESM from CommonJS by using await import(). The import then becomes async, but for Node.js projects, most things are async already, so it should not matter much."
Most things are async with the one MAJOR exception: require. Module loading is not async. You can't replace:
const User = require('./models/User') at the top of your file with: const User = await import('./models/User.js') because you can't use top-level await in CommonJS. Chicken and egg.
OR you have to add an async init() function to every module and then every module that loads it has to call the init during startup. So a complete rearchitecture of the app - clearly silly and only stated to demonstrate that.
When it comes down to it - as soon as you have an ESM module in your code EVERYTHING up the dependency hierarchy (assuming it's even a hierarchy and not really a graph) has to be ESM. And we use "Chalk" in our logger. And EVERYTHING uses the logger.
While I agree with what you're trying to drive, I agree that the ecosystem needs a little more time. And we need that ecosystem to be given a push at the same time. But it's NOT an easy move. It requires a significant amount of time to be invested which is, in many corporate environments, hard to get.
Also, while I believe you said you'd continue backporting security fixes into previous versions and thank you very much for that, it will not stop code scanning tools reporting the module as out of date. And corporate security will require it to be updated. And if our excuse is we don't have time to shut down our project for days to refactor 20 different projects (common dependencies) they'll immediately decide that all 20 of our projects are clearly out-of-date and require refactoring. (Although they won't come with time or money :-) )
I've spent quite some time experimenting with this and the only solution I've found is a complete refactor in one go. I tried to start at the top and work down (which can technically) work but you run into dependency hell very quickly.
While I agree with what you're trying to drive, I agree that the ecosystem needs a little more time. And we need that ecosystem to be given a push at the same time.
The way i see things also are that new packages pops up all the time with the commonjs as the default when running npm init or some similar. if nobody sets the foot down and says it's a ESM only module that can only be imported using es module syntax then there will only be more new commonjs created all the time.
Typescripts default setup have most often be to compile anything to commonjs as the target. It's ironic that typescript have mostly been in the form of "non real" esm with extension less paths and have never bother to deal with resolving that missing .js extension automatically for you when you compile to javascript and totally ignored this issue with the methodology that they don't want to rewrite any valid js code that you write (ref) it have been like this for years and they don't want to do anything to change this and they say that you should just add that .js at the end anyway in every file as it should be in the runtime, if you don't then this is your own fault. yet they rewrite perfectly valid dynamic import(x) to require so the statment don't hold true hold true that they don't touch or change any valid js code... And this is not how Deno works, there you have to add the .ts extension at the end. and microsoft typescript version says to use .js so it contradicts each other a bit. ppl don't want to import a .js file that don't exist cuz everything is written in .ts
so typescript have basically overlooked the hole ESM and ignored it for years under all this time all the way from node v12.17 up to todays node version 18.x. so i think it's for both to blame that folks haven't added that extension at the end and for microsoft to not resolve this path for you... that is why typescript haven't done anything to support ESM upon till now when they are seeing pushback from ESM only packages and it would never have done any progress on this if nobody is ever switching to ESM. then we would still be using commonjs
I'd guess most of maintainers moving packages to pure ESM are barely even affected by the change.
Many of them don't use typescript, jest, or other large popular packages outside their own. I doubt many care about compatibility with a variety of environments (eg serverless, Electron) since they're getting paid to write packages. Sindre uses his own linter, testing library, etc so he's isolated from the incompatibilities he creates. He's even locked issues on repos with millions of weekly downloads at the same time he breaks everyone's code with changes. Some of the maintainers in this thread even seem outright hostile!
If you have long term support in mind it's probably time to get off these maintainers' packages before the next breaking change.
ESM is a cancer. It is a failed attempt by a committee to reinvent the wheels which have been with us since the very early days of NodeJS (and those wheels are still serving us well for Christ's sake!) I can't comprehend why so many developers are willing to take the leap of faith. Time to get rid of it before it leaves us a wasteland.
I'd guess most of maintainers moving packages to pure ESM are barely even affected by the change.
You say that like it's a bad thing. It's almost as if we sensibly aligned our tooling and code to web standards, instead of picking whatever corporate bloatware frameworks blogspam articles are shilling this month, and now we have no problems writing and using standard JavaScript. What would you prefer, that we were as horribly unprepared and terribly affected by the change as you?
This discussion thread is not productive and I never meant for this gist to be a place for everyone to went about ESM.
CONSIDER THIS THREAD LOCKED
ANY COMMENTS AFTER THIS ONE WILL BE DELETED
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK