Skip to content

Import statements are hoisted when target is CommonJs #1686

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
petkaantonov opened this issue May 12, 2021 · 15 comments
Closed

Import statements are hoisted when target is CommonJs #1686

petkaantonov opened this issue May 12, 2021 · 15 comments
Labels

Comments

@petkaantonov
Copy link

Describe the bug

Import statements are hoisted when target is CommonJS:

import A from "a;
console.log(1);
import B from "a";

Becomes

require(...);
require(...);
console.log(1);

This is incorrect, require calls are normal functions and should have preserved order. Imports should only be hoisted when they are actual es6 imports. This is incompatible with tsc and probably others, tsc keeps the order as expected when output module is commonjs.

Additional context

It was fixed but then reverted in #1457

@mischnic
Copy link
Contributor

With native esm (in Node or in browsers), the following prints "a, b, index"

// index.js
import "./a.js";
console.log("index");
import "./b.js";

// a.js
console.log("a");

// b.js
console.log("b");

And I'd argue that transpiling the code to CJS shouldn't change the behaviour of the code

@petkaantonov
Copy link
Author

petkaantonov commented May 12, 2021

  • It is incompatible with tsc
  • It sucks for commonjs users, remember we are forced to use import not because it is better syntax but because typescript types dont work with require
  • Most code was written with require in mind, so require order matters
  • If you are able to use esm, why compile code to cjs? and why expect the behavior be the same? They are incompatible module systems.

@mischnic
Copy link
Contributor

From my perspective, the real problem is that tsc and Babel/the spec don't agree

I guess swc has to choose a side here.

@kdy1
Copy link
Member

kdy1 commented May 12, 2021

This feature is useful for using some of node.js modules, which expect execution order of common js modules.
So I think this behavior should be allowed, but only with a bad-named flag, just like dangerouslySetInnerHTML.
I'll think more about the name of the flag.

@petkaantonov
Copy link
Author

Some suggestions: disableImportHoisting, noImportHoist

@devongovett
Copy link
Contributor

devongovett commented May 12, 2021

Isn't this exactly what the jest mock transform allows already? Do you have a different use case for this? IMO deviating from the spec behavior is a bad idea. TSC is simply wrong.

If you want to imports to behave like require, why not just write require?

@kdy1
Copy link
Member

kdy1 commented May 12, 2021

I missed that users can use require.
Thanks, @devongovett !

I think users should use require if the execution order of require is required.

@kdy1 kdy1 closed this as completed May 12, 2021
@devongovett
Copy link
Contributor

Also fwiw TSC is planning on changing this. microsoft/TypeScript#39764

@petkaantonov
Copy link
Author

Isn't this exactly what the jest mock transform allows already? Do you have a different use case for this? IMO deviating from the spec behavior is a bad idea. TSC is simply wrong.

If you want to imports to behave like require, why not just write require?

TypeScript types don't work with require. When I am compiling TypeScript to CommonJS, the ESM spec has nothing to do with it and its spec should be irrelevant and the commonjs spec should be relevant - which specifies no hoisting. At least a flag should be introduced so that commonjs spec is followed when compiling for commonjs.

dbjorge added a commit to microsoft/accessibility-insights-web that referenced this issue Jun 4, 2021
#### Details

Profiling our unit tests showed that the overwhelming majority of time spent was in `ts-jest` parsing typescript files. This PR is an experiment in replacing `ts-jest` with an alternative typescript transpiler, `swc`, which is less mature but much faster.

##### Motivation

On my local machine (a Surface Laptop 3 running Windows 21H1 OS Build 19043.985), comparative timings are:

| scenario | `main` (ts-jest) | this PR (swc) |
| - | - | - |
| `yarn test -- -- --coverage=false -- footer-section.test.tsx` | 12s ±1s | 4.3s ±0.2s |
| `yarn test -- -- -- footer-section.test.tsx` | 45s ±3s | 6.2s ±0.0s |
| `yarn test` | 11m54s | 3m ±14s |

On our CI build agents:

| scenario | `main` (ts-jest) | this PR (swc) |
| - | - | - |
| unit tests | [15m1s](https://dev.azure.com/accessibility-insights/accessibility-insights-web/_build/results?buildId=21734&view=logs&j=3613c2bd-8b62-5a70-8491-3e8459586450&t=d1343856-9cd9-5649-cdc6-81bc16053e6f) | [5m9s](https://dev.azure.com/accessibility-insights/accessibility-insights-web/_build/results?buildId=21738&view=logs&j=3613c2bd-8b62-5a70-8491-3e8459586450&t=d1343856-9cd9-5649-cdc6-81bc16053e6f) | 
| web-e2e tests | [10m10s](https://dev.azure.com/accessibility-insights/accessibility-insights-web/_build/results?buildId=21734&view=logs&j=952ed597-794a-5756-0328-a93bb2daa2a4&t=46864a42-71df-5c2d-c60f-5fec8d56499d) | [8m6s](https://dev.azure.com/accessibility-insights/accessibility-insights-web/_build/results?buildId=21738&view=logs&j=952ed597-794a-5756-0328-a93bb2daa2a4&t=46864a42-71df-5c2d-c60f-5fec8d56499d) |
| unified-e2e tests | [10m39s](https://dev.azure.com/accessibility-insights/accessibility-insights-web/_build/results?buildId=21734&view=logs&j=c5567cdb-c930-534d-6489-7c2dcf7ac8ee&t=fe6db478-348c-54d0-bafe-001288e28283) | [10m33s](https://dev.azure.com/accessibility-insights/accessibility-insights-web/_build/results?buildId=21738&view=logs&j=c5567cdb-c930-534d-6489-7c2dcf7ac8ee&t=fe6db478-348c-54d0-bafe-001288e28283) |
| total PR build time | [~22m](https://dev.azure.com/accessibility-insights/accessibility-insights-web/_build?definitionId=38&_a=summary) | [~14m](https://dev.azure.com/accessibility-insights/accessibility-insights-web/_build/results?buildId=21740&view=results) |

##### Context

This is not as clear cut a transition as the timings above suggest. `swc` is a relatively young project; it's not quite as firmly single-maintainer as `esbuild`, but it's still got a fairly concerning bus factor compared to `tsc` or `babel` (what `ts-jest` hands off to). Still, the timings speak for themselves and it's easy enough to transition back if that ever becomes necessary.

If this experiment goes well, we might consider a similar change to update our webpack config to use `swc-loader` instead of `ts-loader`.

One of the reasons `swc` is so much faster is that it does not perform type-checking. Right now we do type-checking as part of build, but if we were to switch webpack to use swc, we'd then need a separate build step for type-checking (comparable to the current `yarn null:check` step).

Implementation notes:
* There are a few subtly important things in the `.swcrc` file, which unfortunately doesn't support comments:
    * The `noInterop: true` flag is the equivalent of TypeScript's default `esModuleInterop: false` behavior. It is necessary; we use several modules which cannot be used correctly with interop behavior in play. See swc-project/swc#1778 for details.
    * The empty `"env": {}` property is necessary to trigger `swc` to respect the `browserslist` entry in `package.json`
    * Directing `swc` to target a reasonably recent set of browsers (via the new `browserslist` entry in `package.json`) is required; there are several tests with snapshots that change under default targeting, and there are a few components that break outright.
    * `tsx: true` is required (it tells `swc` to enable support for `tsx` files; it doesn't prevent non-tsx files from parsing)
* For now, this is using a custom Jest transformer to call `@swc/core`. The swc project maintains a `@swc/jest` transformer, but swc-project/jest#8 prevents it from being able to use the `noInterop: true` setting we require, and I can't submit a PR to patch the project until legal rubber stamps their CLA
* `collapsible-script-provider.tsx` required a hacky workaround for swc-project/swc#1165
    * even after applying the workaround, running jest with and without `--coverage=false` emitted javascript with different whitespace. `collapsible-script-provider.test.ts` needed to be updated to format the resulting js before snapshotting it.
* `src/scanner/processor.ts` used a mutable global in a module to enable its tests to inject test data. This confused swc; I replaced it with a more typical ctor injection pattern.
* `browser-adapter-factory.test.tsx` did a dance around an `import 'webextension-polyfill-ts';` statement to trick the module into thinking `window.chrome.runtime.id` is defined (it checks for this during import and refuses to load if it is not set). The dance we used is not compliant with how ES6 imports are defined to work (they are defined as hoisting import statements to the top of the file, ie, before we stub the window property). TypeScript has an outstanding PR/issue to stop supporting what we were previously doing (microsoft/TypeScript#39764) and swc intentionally doesn't support it (swc-project/swc#1686) (babel doesn't either), so I had to modify how our dance worked by moving it to a separate file and having it use a raw `require()` rather than an `import`.

#### Pull request checklist
<!-- If a checklist item is not applicable to this change, write "n/a" in the checkbox -->
- [n/a] Addresses an existing issue: #0000
- [x] Ran `yarn fastpass`
- [x] Added/updated relevant unit test(s) (and ran `yarn test`)
- [x] Verified code coverage for the changes made. Check coverage report at: `<rootDir>/test-results/unit/coverage`
- [x] PR title *AND* final merge commit title both start with a semantic tag (`fix:`, `chore:`, `feat(feature-name):`, `refactor:`). See `CONTRIBUTING.md`.
- [n/a] (UI changes only) Added screenshots/GIFs to description above
- [n/a] (UI changes only) Verified usability with NVDA/JAWS
@lagnat
Copy link

lagnat commented Jul 14, 2022

Also fwiw TSC is planning on changing this. microsoft/TypeScript#39764

Apparently not. The PR was closed and the bug is marked won't-fix.

@kdy1
Copy link
Member

kdy1 commented Jul 14, 2022

@lagnat Please see #5205

@lagnat
Copy link

lagnat commented Jul 14, 2022

We could argue if it's a tsc bug or not but we won't convince each other. What is more at issue is that some projects that use tsc cannot safely switch to swc without re-writes. You could add a compatibility flag to get more projects using swc but it seems that the swc teams is not interested in that and I'm here to voice my opinion that I find it unfortunate.

@magic-akari
Copy link
Member

@lagnat evanw's explanation is the best answer.

The conversion from ESM to CommonJS done by esbuild attempts to preserve the behavior of the original code as much as possible. What you are observing happens because in ESM, all imports are essentially "hoisted" to the top of the file since all dependencies of a given file must be evaluated before any code in that file is evaluated. This is just how the JavaScript language works. More information: evanw/esbuild#1395 (comment).

The code you wrote happens to work due to a defect in the TypeScript compiler: microsoft/TypeScript#16166 (comment). Writing such code is risky because your code will break as soon as it's run in a real ESM environment (such as esbuild, or node's ESM mode, or in a browser).

Originally posted by @evanw in evanw/esbuild#1444 (comment)

@lagnat
Copy link

lagnat commented Jul 15, 2022

It's not a defect because Microsoft closed the bug report as won't-fix and didn't take the PR. Microsoft gets to decide how tsc works and this is what they've decided. I get that you don't agree with the decision and I understand why.. in fact I totally agree with you, but tsc works the way it works and I don't see it changing. I fully acknowledge that the authors of swc can do whatever they want too but it's a strange hill to die on because it hinders a wider adoption of swc. Adding a config option such as emulateBuggyTscAndDontHoist makes your position clear and gets more code on swc. I don't use esm at the moment and I acknowledge that I'll be forced to deal with this issue when that time comes.

@swc-project swc-project locked and limited conversation to collaborators Jul 15, 2022
@kdy1
Copy link
Member

kdy1 commented Jul 15, 2022

Please use discussion thread instead

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Development

No branches or pull requests

6 participants