feat: validate crawlee versions in Actor.init

[B4nan]

The SDK depends on @crawlee/core, but if the user installs an incompatible version of crawlee, they might end up with two installations of the @crawlee/core which means multiple static registers like the one for the global configuration, which breaks the integration, since the SDK will only touch the global config it sees, but it's a different one than the explicitly installed package.

Closes #237

[vladfrangu]

Why do we not just declare crawlee as a peer dependency? Would solve this, no?

[B4nan]

You mean declare it only as a peer instead of a real dependency? In the good old times, that would require the user to install it. Nowadays I believe different package managers do different things, as NPM just decided to install them (which is IMO nonsense in the first place), but yarn wont do that and fail. We don't want people to install both if they don't want to use crawlee.

And honestly, I would like to keep this validation even if it would help. It wont do any harm if things are set up correctly or the node modules folder is not even present. But it will catch such issues if its there.

[vladfrangu]

I'm just worried about relying on fs checks for this... What if

• the users use yarn/pnpm pnp right, no node_modules folder so no check
• the users use pnpm (will it support the hardlinks? Probably)
• doesn't respect that folder structure and flattens it out (like most of them do)

Specifically I'm worried that if our checks don't cover enough we'll need to patch release a few times till we get it right™️

Peer dependencies would solve some of these issues, but because the ecosystem cannot decide what the hell it wants, it's like you said, a mess. npm now automatically installs peer deps (but if you specify a different version that matches the range it won't install twice), yarn does... either nothing or something (looking at you peerDependencyMetadata), who knows about pnpm/bun.

There is also optionalDependencies but not sure if that'll do what we want either. 😢

---

Unrelated, but please use path.join for paths, something something windows 😄

[B4nan]

doesn't respect that folder structure and flattens it out (like most of them do)

Flattening works only if the version ranges match, and then there is no problem.

I've been using similar checks for quite some time (years), and I got inspired by docusaurus doing the same (or similar) thing too. Yes, we might need to improve the logic over time, but only to make it more strict, not to fix some issues it introduces.

There is also optionalDependencies but not sure if that'll do what we want either. 😢

This is not an optional peer, its required, that's why its an actual dependency in the first place.

[vladfrangu]

This is not an optional peer, its required, that's why its an actual dependency in the first place

Optionals still get installed. They're "optional" in the sense that if it fails to install it won't throw an error and abort the whole install process

[B4nan]

Unrelated, but please use path.join for paths, something something windows 😄

used path.normalize, that should do the job too, right?

[B4nan]

Optionals still get installed. They're "optional" in the sense that if it fails to install it won't throw an error and abort the whole install process

Well, that's the part I was referring to, NPM does that nowadays, I don't think yarn would (at least it wasn't doing that some time ago and failed instead)...

edit: uh, I got confused with this again, I keep taking about peers but we are talking optional instead... Again, its not optional dependency at all.

[vladfrangu]

Unrelated, but please use path.join for paths, something something windows 😄

used path.normalize, that should do the job too, right?

I think so, yeah!

[B4nan]

2bc86c7 will help with pnpm and yarn pnp

[B4nan]

I will merge this in and release, worst case we can revert, but it looks all good locally and will help at least with the happy path.