Skip to content

Commit 320dd0c

Browse files
Mr3zeeMr3zee
authored
Update Yarn -> Npm (#364)
1 parent 392f100 commit 320dd0c

File tree

20 files changed

+9780
-5180
lines changed

20 files changed

+9780
-5180
lines changed

‎.idea/kotlinx-rpc.iml

Lines changed: 7 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎.puppeteerrc.cjs

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,13 @@
1+
/*
2+
* Copyright 2023-2025 JetBrains s.r.o and contributors. Use of this source code is governed by the Apache 2.0 license.
3+
*/
4+
15
const {join} = require('path');
26

3-
const isCI = process.env.TEAMCITY_VERSION || process.env.GITHUB_ACTIONS;
7+
const isTC = process.env.TEAMCITY_VERSION;
8+
const isGA = process.env.GITHUB_ACTIONS;
49

5-
if (isCI) {
10+
if (isTC !== undefined || isGA !== undefined) {
611
/**
712
* @type {import("puppeteer").Configuration}
813
*/

‎compiler-plugin/gradle.properties

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,9 @@ org.gradle.configuration-cache=true
1919
org.jetbrains.dokka.experimental.gradle.pluginMode=V2Enabled
2020
org.jetbrains.dokka.experimental.gradle.pluginMode.noWarn=true
2121

22+
# https://youtrack.jetbrains.com/issue/KT-78504
23+
kotlin.js.yarn=false
24+
2225
# development mode for kotlinx.rpc gradle plugin. Uses local project paths to apply the compiler plugin
2326
kotlinx.rpc.plugin.internalDevelopment=true
2427

‎docs/environment.md

Lines changed: 70 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -363,6 +363,74 @@ Here is a 'simple' guide for solving problems:
363363
- Try different IDE version (last resort)
364364
- Docker
365365
- `Cannot connect to the Docker daemon` - open `Docker Desktop`
366+
- Kotlin/Js or Kotlin/Wasm
367+
- `kotlinUpgradePackageLock` or `kotlinWasmUpgradePackageLock` (and also `kotlinNpmInstall` or `kotlinWasmNpmInstall`)
368+
have a funny tendency to fail sometimes, and you don't know why.
369+
<br/>
370+
I'll tell you!
371+
<br/>
372+
We use proxy repos, and sometimes dependencies get downloaded from the wrong source.
373+
Make sure ALL urls in `package-lock.json` files start with `https://packages.jetbrains.team/npm/p/krpc/build-deps/`.
374+
<br/>
375+
If something doesn't work, your steps are:
376+
- Delete `package-lock.json` file
377+
- Delete `<REPO_ROOT>/build/js` / `<REPO_ROOT>/build/wasm`
378+
- Run `kotlinUpgradePackageLock` / `kotlinWasmUpgradePackageLock`
379+
- If the problem persists:
380+
- Check that `<REPO_ROOT>/build/<target>/.npmrc` AND `<REPO_ROOT>/build/<target>/.yarnrc` are present
381+
- Check that `.yarnrc` contains one line: `registry: "https://packages.jetbrains.team/npm/p/krpc/build-deps/"`
382+
- Check that `.npmrc` contains the following lines:
383+
- `registry="https://packages.jetbrains.team/npm/p/krpc/build-deps/"`
384+
- `always-auth=true`
385+
- `save-exact=true`
386+
- `//packages.jetbrains.team/npm/p/krpc/build-deps/:_authToken=<your_auth_token>`,
387+
where `<your_auth_token>` is from the [proxy-repositories.md](proxy-repositories.md) guide.
388+
- Check that `<USER_HOME>/.npmrc` / `<USER_HOME>/.yarnrc` don't interfere
389+
command to debug. Replace versions of tools if needed.
390+
- When you get the following error — `puppeteer` failed to run the installation script.
391+
Reasons vary, try updating the version to a newer one,
392+
check the [.puppeteerrc.cjs](../.puppeteerrc.cjs) and [chrome_bin.js](../karma/chrome_bin.js) files if they are valid js.
393+
394+
For (2), check out our guide on configuring puppeteer at https://pptr.dev/guides/configuration.
395+
at ChromeLauncher.resolveExecutablePath (/rpc/build/js/packages/kotlinx-rpc-utils-test/node_modules/puppeteer-core/lib/cjs/puppeteer/node/ProductLauncher.js:295:27)
396+
397+
- When the previous error is gone, you may get the next one.
398+
399+
Errors occurred during launch of browser for testing.
400+
- ChromeHeadless
401+
Please make sure that you have installed browsers.
402+
Or change it via
403+
browser {
404+
testTask {
405+
useKarma {
406+
useFirefox()
407+
useChrome()
408+
useSafari()
409+
}
410+
}
411+
}
412+
This means the `puppeteer` failed to locate Chrome.
413+
Either the cache dir is wrong (check [.puppeteerrc.cjs](../.puppeteerrc.cjs) file) or it really isn't there.
414+
<br/>
415+
Reasons again vary.
416+
When `npm` installs `puppeteer`, it should execute script to install the browser too
417+
(On CI to the `<ROOT_DIR>/.puppeteer/browsers` directory).
418+
This absence may be caused by the `--ignore-scripts` flag.
419+
Check the clean installation (`rm -rf build && ./gradlew clean cleanJsBrowserTest`) with `--debug` flag.
420+
(Something like `./gradlew jsBrowserTest --debug`).
421+
**IMPORTANT: run in docker with `TEAMCITY_VERSION` env var set, if you are chasing a CI fail**.
422+
<br/>
423+
The property is set in [npm.kt](../gradle-conventions/src/main/kotlin/util/tasks/npm.kt), see `ignoreScripts`,
424+
it should be `false`.
425+
<br/>
426+
If this is not the case, check the debug log for other `node`-related issues.
427+
Try installing browsers manually: `~/.gradle/nodejs/node-v22.0.0-linux-x64/bin/node build/js/node_modules/puppeteer/install.mjs`
428+
If this works — problem is somewhere in KGP and probably your configs.
429+
Check that your config (like ones with `ignoreScript`) are actually applied,
430+
as they use on demand execution and may target wrong plugin or extension and never be executed.
431+
<br/>
432+
**Bonus**: it may not be installed, because npm install doesn't do this.
433+
See the long comment in [npm.kt](../gradle-conventions/src/main/kotlin/util/tasks/npm.kt).
366434

367435
Something doesn't work, and you are sure it's not your fault? Report it appropriately! Don't be lazy.
368436

@@ -377,7 +445,8 @@ all included builds (not subprojects) must reflect the change.
377445
- `checkLegacyAbi` / `updateLegacyAbi` - ABI checks.
378446
See https://kotlinlang.org/docs/whatsnew22.html#binary-compatibility-validation-included-in-kotlin-gradle-plugin.
379447
Former BCV: https://github.com/Kotlin/binary-compatibility-validator
380-
- `kotlinUpgradeYarnLock` - update [kotlin-js-store](../kotlin-js-store) contents, usually after Kotlin version update.
448+
- `kotlinUpgradePackageLock` / `kotlinWasmUpgradePackageLock` - update [kotlin-js-store](../kotlin-js-store) contents,
449+
usually after Kotlin version update.
381450
- `updateDocsChangelog` - put modified [CONTRIBUTING.md](../CONTRIBUTING.md) into [topics](pages/kotlinx-rpc/topics)
382451
- `detekt` - run detekt checks.
383452
- `verifyPlatformTable` / `dumpPlatformTable` - Update [platforms.topic](pages/kotlinx-rpc/topics/platforms.topic)

‎dokka-plugin/gradle.properties

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,9 @@ org.gradle.configuration-cache=true
1919
org.jetbrains.dokka.experimental.gradle.pluginMode=V2Enabled
2020
org.jetbrains.dokka.experimental.gradle.pluginMode.noWarn=true
2121

22+
# https://youtrack.jetbrains.com/issue/KT-78504
23+
kotlin.js.yarn=false
24+
2225
# development mode for kotlinx.rpc gradle plugin. Uses local project paths to apply the compiler plugin
2326
kotlinx.rpc.plugin.internalDevelopment=true
2427

‎gradle-conventions-settings/settings.gradle.kts

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,36 @@
22
* Copyright 2023-2025 JetBrains s.r.o and contributors. Use of this source code is governed by the Apache 2.0 license.
33
*/
44

5+
plugins {
6+
// Keep it in sync with libs.versions.toml
7+
id("com.gradle.develocity") version "3.19.2"
8+
}
9+
510
rootProject.name = "gradle-conventions-settings"
611

712
// Code below is a hack because a chicken-egg problem, I can't use myself as a settings-plugin
813
apply(from = "src/main/kotlin/conventions-repositories.settings.gradle.kts")
914
apply(from = "src/main/kotlin/conventions-version-resolution.settings.gradle.kts")
1015

1116
include(":develocity")
17+
18+
// Should be in sync with ktorbuild.develocity.settings.gradle.kts
19+
develocity {
20+
server = "https://ge.jetbrains.com"
21+
}
22+
23+
val isCIRun = providers.environmentVariable("TEAMCITY_VERSION").isPresent ||
24+
providers.environmentVariable("GITHUB_ACTIONS").isPresent
25+
26+
buildCache {
27+
if (isCIRun) {
28+
local {
29+
isEnabled = false
30+
}
31+
}
32+
33+
remote(develocity.buildCache) {
34+
isPush = isCIRun
35+
isEnabled = true
36+
}
37+
}

‎gradle-conventions/src/main/kotlin/util/targets/js.kt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ fun KmpConfig.configureJs() {
2222
nodejs {
2323
testTask {
2424
useMocha {
25-
timeout = "100s"
25+
timeout = "300s"
2626
}
2727
}
2828
}

‎gradle-conventions/src/main/kotlin/util/tasks/npm.kt

Lines changed: 121 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -6,30 +6,50 @@ package util.tasks
66

77
import org.gradle.api.GradleException
88
import org.gradle.api.Project
9+
import org.gradle.api.Task
10+
import org.gradle.api.tasks.Delete
11+
import org.gradle.api.tasks.Exec
12+
import org.gradle.api.tasks.TaskProvider
913
import org.gradle.kotlin.dsl.*
1014
import org.jetbrains.kotlin.gradle.targets.js.nodejs.NodeJsEnvSpec
15+
import org.jetbrains.kotlin.gradle.targets.js.nodejs.NodeJsRootExtension
1116
import org.jetbrains.kotlin.gradle.targets.js.nodejs.NodeJsRootPlugin
12-
import org.jetbrains.kotlin.gradle.targets.js.yarn.YarnLockMismatchReport
13-
import org.jetbrains.kotlin.gradle.targets.js.yarn.YarnPlugin
14-
import org.jetbrains.kotlin.gradle.targets.js.yarn.YarnRootEnvSpec
17+
import org.jetbrains.kotlin.gradle.targets.js.npm.BaseNpmExtension
18+
import org.jetbrains.kotlin.gradle.targets.js.npm.LockFileMismatchReport
19+
import org.jetbrains.kotlin.gradle.targets.js.npm.NpmExtension
20+
import org.jetbrains.kotlin.gradle.targets.wasm.nodejs.WasmNodeJsEnvSpec
21+
import org.jetbrains.kotlin.gradle.targets.wasm.nodejs.WasmNodeJsRootExtension
22+
import org.jetbrains.kotlin.gradle.targets.wasm.nodejs.WasmNodeJsRootPlugin
23+
import org.jetbrains.kotlin.gradle.targets.wasm.npm.WasmNpmExtension
24+
import org.jetbrains.kotlin.gradle.targets.web.nodejs.BaseNodeJsEnvSpec
25+
import org.jetbrains.kotlin.gradle.targets.web.nodejs.BaseNodeJsRootExtension
26+
import org.jetbrains.kotlin.gradle.targets.web.nodejs.CommonNodeJsRootPlugin
1527
import util.other.optionalProperty
1628
import util.other.spacePassword
1729
import util.other.useProxyRepositories
1830
import java.io.File
1931

20-
fun Project.configureNpm() {
21-
val kotlinMasterBuild by optionalProperty()
22-
23-
val executeNpmLogin by tasks.registering {
32+
const val PUPPETEER_BROWSERS_DIR = ".puppeteer"
33+
34+
private inline fun <
35+
reified Plugin : CommonNodeJsRootPlugin,
36+
reified Spec : BaseNodeJsEnvSpec,
37+
reified RootExtension : BaseNodeJsRootExtension,
38+
> Project.registerCustomNpmTasks(
39+
target: String,
40+
useProxy: Boolean,
41+
) {
42+
val capitalizedTarget = target.replaceFirstChar { it.titlecase() }
43+
val login = tasks.register("execute${capitalizedTarget}NpmLogin") {
2444
if (!useProxyRepositories) {
25-
return@registering
45+
return@register
2646
}
2747

2848
val registryUrl = "https://packages.jetbrains.team/npm/p/krpc/build-deps/"
2949

3050
// To prevent leaking of credentials in VCS on dev machine use the build directory config file
31-
val buildYarnConfigFile = File(project.rootDir, "build/js/.yarnrc")
32-
val buildNpmConfigFile = File(project.rootDir, "build/js/.npmrc")
51+
val buildYarnConfigFile = File(project.rootDir, "build/$target/.yarnrc")
52+
val buildNpmConfigFile = File(project.rootDir, "build/$target/.npmrc")
3353

3454
val spacePassword: String? = spacePassword
3555

@@ -64,36 +84,107 @@ fun Project.configureNpm() {
6484
outputs.file(buildNpmConfigFile).withPropertyName("buildOutputNpmFile")
6585
}
6686

67-
val useProxy = useProxyRepositories
68-
69-
plugins.withType(NodeJsRootPlugin::class.java).configureEach {
70-
rootProject.extensions.configure<NodeJsEnvSpec> {
87+
plugins.withType<Plugin>().configureEach {
88+
extensions.configure<Spec> {
7189
download = true
90+
7291
if (useProxy) {
7392
downloadBaseUrl = "https://packages.jetbrains.team/files/p/krpc/build-deps/"
7493
}
75-
}
7694

77-
tasks.named("kotlinNpmInstall").configure {
78-
dependsOn(executeNpmLogin)
95+
val nodeExecutable = executable
96+
97+
extensions.configure<RootExtension> {
98+
/**
99+
* Long story short:
100+
* We can use Yarn because of this: https://youtrack.jetbrains.com/issue/KT-78504
101+
* So: `kotlin.js.yarn=false`
102+
*
103+
* When we use `npm` instead, it runs `npm install` task.
104+
* That should install `puppeteer` package, which should run its script `install.mjs`.
105+
* That script installs browsers for tests.
106+
*
107+
* If we pass `--ignore-scripts` to npm, the script won't be executed.
108+
* KGP does it by default.
109+
* So we set `ignoreScripts = false`.
110+
* We set it for `NpmExtension` and `WasmNpmExtension` in `NodeJsRootPlugin` and `WasmNodeJsRootPlugin`
111+
* respectively (and not their common supertype because it will not work)
112+
*
113+
* And this approach worked for Yarn.
114+
* Script was executed, browsers were installed.
115+
*
116+
* For some reason, for `npm` it didn't work.
117+
* Even with a proper flag (which I checked with a --debug flag).
118+
*
119+
* So we need to run the script manually AFTER `kotlinNpmInstall` (or `kotlinWasmNpmInstall`).
120+
* But also, BEFORE every other action that comes after `kotlinNpmInstall` (or `kotlinWasmNpmInstall`),
121+
* as otherwise there will be race in parallel tasks execution.
122+
*
123+
* Hence, all shenanigans.
124+
*/
125+
val puppeteerInstall = tasks.register<Exec>("puppeteerInstall$capitalizedTarget") {
126+
commandLine(nodeExecutable.get(), "build/$target/node_modules/puppeteer/install.mjs")
127+
workingDir = rootProject.projectDir
128+
129+
// keep in sync with <ROOT>/.puppeteer.cjs
130+
outputs.dir(rootProject.projectDir.resolve(PUPPETEER_BROWSERS_DIR).resolve("browsers"))
131+
}
132+
133+
npmInstallTaskProvider.configure {
134+
dependsOn(login)
135+
finalizedBy(puppeteerInstall)
136+
}
137+
138+
tasks.matching { task ->
139+
task.dependsOn.any { dependency ->
140+
when (dependency) {
141+
is Task -> dependency == npmInstallTaskProvider.get()
142+
is TaskProvider<*> -> dependency == npmInstallTaskProvider
143+
is String -> dependency == npmInstallTaskProvider.name
144+
else -> false
145+
}
146+
}
147+
}.configureEach {
148+
dependsOn(puppeteerInstall)
149+
}
150+
}
79151
}
80152
}
153+
}
81154

82-
// necessary for CI js tests
83-
rootProject.plugins.withType<YarnPlugin> {
84-
rootProject.extensions.configure<YarnRootEnvSpec> {
85-
ignoreScripts = false
86-
download = true
155+
@Suppress("UnusedReceiverParameter")
156+
fun BaseNpmExtension.configureNpmExtension(useProxy: Boolean, kotlinMasterBuild: Boolean) {
157+
// todo it still doesn't work for an unknown reason, see 'puppeteerInstall*' tasks above
158+
ignoreScripts = false // if true - puppeteer won't install browsers
87159

88-
yarnLockMismatchReport = if (useProxy && !kotlinMasterBuild) {
89-
YarnLockMismatchReport.FAIL
90-
} else {
91-
YarnLockMismatchReport.WARNING
92-
}
160+
packageLockMismatchReport = if (useProxy && !kotlinMasterBuild) {
161+
LockFileMismatchReport.FAIL
162+
} else {
163+
LockFileMismatchReport.WARNING
164+
}
165+
}
93166

94-
if (useProxy) {
95-
downloadBaseUrl = "https://packages.jetbrains.team/files/p/krpc/build-deps/"
96-
}
167+
fun Project.configureNpm() {
168+
val kotlinMasterBuild by optionalProperty()
169+
val useProxy = useProxyRepositories
170+
171+
registerCustomNpmTasks<NodeJsRootPlugin, NodeJsEnvSpec, NodeJsRootExtension>("js", useProxy)
172+
registerCustomNpmTasks<WasmNodeJsRootPlugin, WasmNodeJsEnvSpec, WasmNodeJsRootExtension>("wasm", useProxy)
173+
174+
// necessary for CI js tests
175+
rootProject.plugins.withType<NodeJsRootPlugin> {
176+
rootProject.extensions.configure<NpmExtension> {
177+
configureNpmExtension(useProxy, kotlinMasterBuild)
178+
}
179+
}
180+
181+
rootProject.plugins.withType<WasmNodeJsRootPlugin> {
182+
rootProject.extensions.configure<WasmNpmExtension> {
183+
configureNpmExtension(useProxy, kotlinMasterBuild)
97184
}
98185
}
186+
187+
tasks.named<Delete>("clean") {
188+
delete(project.layout.projectDirectory.dir(PUPPETEER_BROWSERS_DIR))
189+
}
99190
}

‎gradle-plugin/gradle.properties

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,9 @@ org.gradle.configuration-cache=true
1919
org.jetbrains.dokka.experimental.gradle.pluginMode=V2Enabled
2020
org.jetbrains.dokka.experimental.gradle.pluginMode.noWarn=true
2121

22+
# https://youtrack.jetbrains.com/issue/KT-78504
23+
kotlin.js.yarn=false
24+
2225
# development mode for kotlinx.rpc gradle plugin. Uses local project paths to apply the compiler plugin
2326
kotlinx.rpc.plugin.internalDevelopment=true
2427

‎gradle.properties

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,9 @@ org.gradle.configuration-cache=true
1919
org.jetbrains.dokka.experimental.gradle.pluginMode=V2Enabled
2020
org.jetbrains.dokka.experimental.gradle.pluginMode.noWarn=true
2121

22+
# https://youtrack.jetbrains.com/issue/KT-78504
23+
kotlin.js.yarn=false
24+
2225
# development mode for kotlinx.rpc gradle plugin. Uses local project paths to apply the compiler plugin
2326
kotlinx.rpc.plugin.internalDevelopment=true
2427

‎gradle/wrapper/gradle-wrapper.jar

-16.6 KB
Binary file not shown.

0 commit comments

Comments
 (0)