Skip to content

feat(next): Allow specifying workflow directories#1159

Open
rovo89 wants to merge 1 commit intovercel:mainfrom
rovo89:feat/next-workflow-dirs
Open

feat(next): Allow specifying workflow directories#1159
rovo89 wants to merge 1 commit intovercel:mainfrom
rovo89:feat/next-workflow-dirs

Conversation

@rovo89
Copy link
Contributor

@rovo89 rovo89 commented Feb 21, 2026

This brings back the option introduced in #843 (and removed in #961), but with a slightly different meaning. Instead of specifying directories of pages whiuch call server actions which trigger workflows, the option is now meant to directly specify the directory with the workflow files. That removes two layers of indirection, imports way less files and therefore really improves performance as the original PR intended (~15s -> ~2.5s on a medium-sized app).

This brings back the option introduced in vercel#843 (and removed in vercel#961),
but with a slightly different meaning. Instead of specifying directories
of pages whiuch call server actions which trigger workflows, the option
is now meant to directly specify the directory with the workflow files.
That removes two layers of indirection, imports way less files and
therefore really improves performance as the original PR intended
(~15s -> ~2.5s on a medium-sized app).
@changeset-bot
Copy link

changeset-bot bot commented Feb 21, 2026

🦋 Changeset detected

Latest commit: 7e4c85b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 14 packages
Name Type
@workflow/next Patch
workflow Patch
@workflow/world-testing Patch
@workflow/core Patch
@workflow/builders Patch
@workflow/cli Patch
@workflow/nitro Patch
@workflow/web-shared Patch
@workflow/astro Patch
@workflow/nest Patch
@workflow/rollup Patch
@workflow/sveltekit Patch
@workflow/vite Patch
@workflow/nuxt Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel
Copy link
Contributor

vercel bot commented Feb 21, 2026

@rovo89 is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.

Copy link
Member

@ijjk ijjk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi have you tried upgrading to latest workflow package and Next.js >= v16.2.0-canary.48 and then enabling this flag in workflows:

export default withWorkflow(nextConfig, { workflows: { lazyDiscovery: true } });

That will no longer use esbuild for bundling steps and discovery workflows/steps so should see the perf improvement without having to worry about a config like this.

@rovo89
Copy link
Contributor Author

rovo89 commented Feb 26, 2026

Hi have you tried upgrading to latest workflow package and Next.js >= v16.2.0-canary.48 and then enabling this flag in workflows:

I have just tried it. Without any other changes, I get:

[local world] Failed to queue message {
  queueName: '__wkf_step_step//./src/workflows/chat/steps//loadChat',
  text: '"Error: Step \\"step//./src/workflows/chat/steps//loadChat\\" not found"',
  status: 500,
  ...

The "Run Created", "Run Started" and "Step Created" events exist, so it seems to have found the workflow, but not the step.

Works find with lazyDiscovery: false. Is there any documentation on this parameter, or some preview docs for the Next.js canary, especially for deferredEntries? Do I understand you correctly that no scanning at all is required with lazy discovery? It's not just a shift to a later point in time?

@ijjk
Copy link
Member

ijjk commented Feb 27, 2026

Hi, can you share exact Next.js version did you use latest canary 16.2.0-canary.64, and latest workflow package 4.1.0-beta.60 also just confirming this is with turbopack or webpack?

Also same results next dev and next build && next start or only next dev?

The lazy discovery is collecting the workflow/step usage in the workflow's loader then generating the route entries for webpack or turbopack to handle from that point. It's goal is to remove using esbuild to pass over all entries which is very expensive.

Comment on lines +76 to +80
export default withWorkflow(nextConfig, {
workflows: {
dirs: ['workflows', 'src/workflows'], // [!code highlight]
},
});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should do dirs (for other reasons - like @ijjk mentioned - the lazy approach is the correct answer here).

but if we were to do dirs or add new configs, we shouldn't be putting it in a enw config options at the end of withWorkflow I think. instead the signature of withWorkflow should become withWorkflow(NextConfig & { workflow: WorkflowConfig }) imo (including the local port/datadir options and future config options)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the lazy approach works, then sure, I'll use it. Just began testing it again.

But what would the call to withWorkflow look like then? Keep in mind that the config might be a function, so we can't easily pass {...nextConfig, workflow: {...}}. Putting the option in the "raw" object literal would require typing it as NextConfigOrFn & WorkflowConfig, which again wouldn't work for functions and would be an extra step. I don't see what's wrong with the current signature, withSentryConfig does it the same way. Maybe Next.js could give an easier way for multiple modules to add config, but then you'd also need to care about namespaces, ...

dirs: ['.'], // Different apps that use nitro have different directories
}),
buildTarget: 'next', // Placeholder, not actually used
buildTarget: 'standalone', // Placeholder, not actually used
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'standalone; means something else. this could be nitro but honestly better to leave it out of this PR since it's not used anyway

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unless this was needed for some reason?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It complained about the missing dirsAreEntrypoints, but yeah, could just pass that.


**Solution:** Use the `workflows.dirs` option to point directly at the directories containing your workflow files. By default, `withWorkflow` uses entrypoint directories (`pages`, `app`, `src/pages`, `src/app`) as starting points and traces their imports to discover workflows, which in large applications can involve crawling thousands of files.

If your workflows live in a dedicated directory, configure `dirs` to only scan that directory:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the other problem with "dirs" is that it causes split brain I think:

the client mode/webpack transformation still has to happen in the actual code (api routes/server actions) that trigger workflows. we can't skip that - and that code won't live in the workflows directory

dirs feels like it misleads you into believing you've limited the search space to specific directory when you're only limiting the search space of actual workflows/steps imo

@rovo89
Copy link
Contributor Author

rovo89 commented Feb 27, 2026

Hi, can you share exact Next.js version did you use latest canary 16.2.0-canary.64, and latest workflow package 4.1.0-beta.60 also just confirming this is with turbopack or webpack?

I just tried it again, next@16.2.0-canary.65 and workflow@4.1.0-beta.61 with turbopack.

Also same results next dev and next build && next start or only next dev?

Both. Do I have to enable any other config?

@ijjk
Copy link
Member

ijjk commented Feb 27, 2026

Any chance you could share the repo/a repro? Our E2E tests are currently running against this setup with Next.js canary so would love to track down where the setup is failing.

@masterjanic
Copy link

I can also confirm that lazyDiscovery is just not working with turbopack. The workflows are never discovered and just time out. We need a solution because current build times are massive for my app and I often receive an out of memory event.

[0] [local world] Queue operation failed: TypeError: fetch failed
[0]     at ignore-listed frames {
[0]   [cause]: Error [SocketError]: other side closed
[0]       at ignore-listed frames {
[0]     code: 'UND_ERR_SOCKET',
[0]     socket: {
[0]       localAddress: '127.0.0.1',
[0]       localPort: 42910,
[0]       remoteAddress: '127.0.0.1',
[0]       remotePort: 8888,
[0]       remoteFamily: 'IPv4',
[0]       timeout: undefined,
[0]       bytesWritten: 861,
[0]       bytesRead: 0
[0]     }
[0]   }
[0] }

@rovo89
Copy link
Contributor Author

rovo89 commented Feb 28, 2026

@ijjk Here's a very simple repro: https://github.com/rovo89/lazy-workflow. Minimal changes compared to next-create-app: rovo89/lazy-workflow@753aaed. Node.js version 22 vs 24 doesn't make a difference. With lazyDiscovery: false, it works fine.

@rovo89
Copy link
Contributor Author

rovo89 commented Feb 28, 2026

I also tried moving the directives into the functions themselves, doesn't help. I had switched to the file-level directives because Prettier kept putting parentheses around them, and it's a bit shorter.

Moving them both functions into one file (rovo89/lazy-workflow@bbd5076) solves it, but the recommendation is to keep them separate: https://useworkflow.dev/docs/foundations/workflows-and-steps#project-structure

Webpack also doesn't work.

@ijjk
Copy link
Member

ijjk commented Feb 28, 2026

@rovo89 thanks for sharing that context, I'll take a closer look this not working as expected with webpack is surprising so there's probably a higher level bug here with the lazy setup we can address

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants