-
Notifications
You must be signed in to change notification settings - Fork 204
docs: add scheduling & cron patterns to foundations #1111
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
johnlindquist
wants to merge
5
commits into
main
Choose a base branch
from
docs/scheduling-cron
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
41e0483
docs(foundations): add scheduling and cron patterns guide
johnlindquist 40da1fd
docs(foundations): fix scheduling date math and weekly dispatch
johnlindquist 80bf5ac
docs(foundations): add FatalError escape hatch and Run.cancel() to sc…
johnlindquist 5b0e8db
Merge remote-tracking branch 'origin/main' into docs/scheduling-cron
johnlindquist e376cd7
Address PR review feedback on scheduling docs
johnlindquist File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,7 +8,8 @@ | |
| "hooks", | ||
| "streaming", | ||
| "serialization", | ||
| "idempotency" | ||
| "idempotency", | ||
| "scheduling" | ||
| ], | ||
| "defaultOpen": true | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,269 @@ | ||
| --- | ||
| title: Scheduling & Cron | ||
| description: Implement recurring jobs and scheduled tasks using durable workflow patterns. | ||
| type: guide | ||
| summary: Build scheduled and recurring execution patterns with durable sleep loops. | ||
| prerequisites: | ||
| - /docs/foundations/workflows-and-steps | ||
| related: | ||
| - /docs/foundations/common-patterns | ||
| - /docs/api-reference/workflow/sleep | ||
| - /docs/foundations/hooks | ||
| --- | ||
|
|
||
| Workflows naturally support scheduling through [`sleep()`](/docs/api-reference/workflow/sleep). Unlike traditional cron systems that require external infrastructure, workflow-based scheduling is durable - if the server restarts, the schedule resumes without missing a beat. And because `sleep()` suspends the workflow without consuming compute resources, a workflow sleeping for one minute costs the same as one sleeping for a month. | ||
|
|
||
| ## Recurring Execution Patterns | ||
|
|
||
| There are three patterns for recurring jobs, each suited to different lifetimes and workloads. | ||
|
|
||
| ### Loop with Steps | ||
|
|
||
| The simplest pattern is a `while(true)` loop that runs a job and sleeps between iterations. Use this for **short-lived or bounded** work that eventually exits — polling that resolves, health checks with a failure threshold, or jobs with a known end condition. | ||
|
|
||
| ```typescript title="workflows/recurring-job.ts" lineNumbers | ||
| import { sleep } from "workflow"; | ||
| declare function runJob(): Promise<{ success: boolean }>; // @setup | ||
|
|
||
| export async function recurringJobWorkflow() { | ||
| "use workflow"; | ||
|
|
||
| while (true) { // [!code highlight] | ||
| // runJob can be a step function or an entire composed workflow | ||
| // See: /docs/foundations/common-patterns#workflow-composition | ||
| const result = await runJob(); // [!code highlight] | ||
|
|
||
| if (!result.success) { | ||
| break; | ||
| } | ||
|
|
||
| await sleep("1 hour"); // [!code highlight] | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| <Callout type="warn"> | ||
| Every iteration appends to the workflow's event log. For workflows that run indefinitely, the log grows without bound, causing replay performance to degrade over time. For true infinite cron jobs, use **daisy-chaining** instead. | ||
| </Callout> | ||
|
|
||
| ### Dispatcher Loop | ||
|
|
||
| A long-running dispatcher loop that uses a step function to call `start()` and kick off **background child workflows** without awaiting their completion. Each child workflow runs with its own event log, so the dispatcher's log stays small. | ||
|
|
||
| ```typescript title="workflows/cron-dispatcher.ts" lineNumbers | ||
| import { sleep } from "workflow"; | ||
| declare function triggerDailyReport(): Promise<string>; // @setup | ||
| declare function triggerCleanup(): Promise<string>; // @setup | ||
|
|
||
| export async function cronDispatcherWorkflow() { | ||
| "use workflow"; | ||
|
|
||
| let iteration = 0; | ||
|
|
||
| while (true) { | ||
| // Run daily report every iteration | ||
| await triggerDailyReport(); | ||
|
|
||
| // Run cleanup every 7 iterations (weekly), skipping the first run | ||
| iteration++; | ||
| if (iteration % 7 === 0) { // [!code highlight] | ||
| await triggerCleanup(); // [!code highlight] | ||
| } | ||
|
|
||
| await sleep("1 day"); | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ```typescript title="workflows/steps.ts" lineNumbers | ||
| import { start } from "workflow/api"; | ||
|
|
||
| export async function triggerDailyReport() { | ||
| "use step"; | ||
|
|
||
| // stepId is stable across retries — use it if the child workflow | ||
| // needs deduplication (e.g., as an idempotency key for external APIs) | ||
| const run = await start(dailyReportWorkflow, []); // [!code highlight] | ||
| return run.runId; | ||
| } | ||
|
|
||
| export async function triggerCleanup() { | ||
| "use step"; | ||
|
|
||
| // stepId is stable across retries — use it if the child workflow | ||
| // needs deduplication (e.g., as an idempotency key for external APIs) | ||
| const run = await start(cleanupWorkflow, []); | ||
| return run.runId; | ||
| } | ||
|
|
||
| // These are independent workflows started by the dispatcher | ||
| export async function dailyReportWorkflow() { | ||
| "use workflow"; | ||
| // Generate and send daily report | ||
| } | ||
|
|
||
| export async function cleanupWorkflow() { | ||
| "use workflow"; | ||
| // Clean up old data | ||
| } | ||
| ``` | ||
|
|
||
| Each child workflow runs independently. If a daily report fails, it does not affect the dispatcher or the cleanup workflow. You can monitor each run separately using its `runId`. | ||
|
|
||
| ### Daisy-Chaining | ||
|
|
||
| The recommended pattern for **infinite, true cron jobs**. Instead of looping, the workflow executes its logic, sleeps for the desired duration, and as its final action triggers a brand new execution of itself before exiting. This ensures the event log resets with each run, keeping replay performance constant regardless of how long the job has been running. | ||
|
|
||
| ```typescript title="workflows/daisy-chain.ts" lineNumbers | ||
| import { sleep } from "workflow"; | ||
| declare function runJob(): Promise<void>; // @setup | ||
| declare function startNextRun(): Promise<string>; // @setup | ||
|
|
||
| export async function cronWorkflow() { | ||
| "use workflow"; | ||
|
|
||
| await runJob(); | ||
| await sleep("1 hour"); | ||
|
|
||
| // Start the next execution before exiting — event log resets | ||
| await startNextRun(); // [!code highlight] | ||
| } | ||
| ``` | ||
|
|
||
| ```typescript title="workflows/steps.ts" lineNumbers | ||
| import { start } from "workflow/api"; | ||
|
|
||
| export async function startNextRun() { | ||
| "use step"; | ||
|
|
||
| const run = await start(cronWorkflow, []); | ||
| return run.runId; | ||
| } | ||
| ``` | ||
|
|
||
| <Callout type="info"> | ||
| `sleep()` consumes no compute resources while waiting. A workflow sleeping for one hour is effectively free until it wakes up. | ||
| </Callout> | ||
|
|
||
| For more on triggering workflows from within workflows, see [Workflow Composition](/docs/foundations/common-patterns#workflow-composition). | ||
|
|
||
| ## Scheduled Tasks at Specific Times | ||
|
|
||
| Pass a `Date` object to `sleep()` to wait until a specific point in time. This is useful for tasks that must run at exact times rather than fixed intervals. | ||
|
|
||
| ```typescript title="workflows/scheduled-task.ts" lineNumbers | ||
| import { sleep } from "workflow"; | ||
| declare function sendReport(period: string): Promise<void>; // @setup | ||
|
|
||
| export async function endOfMonthReportWorkflow(year: number, monthNumber: number) { | ||
| "use workflow"; | ||
|
|
||
| // monthNumber is 1-12; Date constructor uses 0-based months | ||
| // new Date(2025, 6, 1) = July 1st 2025 (month index 6 = July) | ||
| const reportDate = new Date(year, monthNumber, 1, 0, 0, 0); // [!code highlight] | ||
| await sleep(reportDate); // [!code highlight] | ||
|
|
||
| await sendReport(`${year}-${String(monthNumber).padStart(2, "0")}`); | ||
| } | ||
| ``` | ||
|
|
||
| <Callout type="info"> | ||
| `Date` constructors are deterministic inside workflow functions - the framework ensures consistent values across replays. | ||
| </Callout> | ||
|
|
||
| ## Health Checks and Keep-Alive | ||
|
|
||
| Periodic health monitoring is a natural fit for durable workflows. The workflow checks a service, takes action if something is wrong, and sleeps before checking again. | ||
|
|
||
| ```typescript title="workflows/health-check.ts" lineNumbers | ||
| import { sleep } from "workflow"; | ||
| declare function checkServiceHealth(serviceUrl: string): Promise<{ healthy: boolean; latencyMs: number }>; // @setup | ||
| declare function sendAlert(serviceUrl: string, latencyMs: number): Promise<void>; // @setup | ||
| declare function restartService(serviceUrl: string): Promise<void>; // @setup | ||
|
|
||
| export async function healthCheckWorkflow(serviceUrl: string) { | ||
| "use workflow"; | ||
|
|
||
| let consecutiveFailures = 0; | ||
|
|
||
| while (true) { | ||
| const status = await checkServiceHealth(serviceUrl); | ||
|
|
||
| if (!status.healthy) { | ||
| consecutiveFailures++; | ||
|
|
||
| await sendAlert(serviceUrl, status.latencyMs); | ||
|
|
||
| if (consecutiveFailures >= 10) { // [!code highlight] | ||
| throw new Error(`${serviceUrl} unrecoverable after 10 consecutive failures`); // [!code highlight] | ||
| } | ||
|
|
||
| if (consecutiveFailures >= 3) { // [!code highlight] | ||
| await restartService(serviceUrl); // [!code highlight] | ||
| consecutiveFailures = 0; // [!code highlight] | ||
| } | ||
| } else { | ||
| consecutiveFailures = 0; | ||
| } | ||
|
|
||
| await sleep("30 seconds"); | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| The workflow tracks `consecutiveFailures` across iterations. After three consecutive failures, it escalates to a service restart and resets the counter to give the service a recovery window. After ten consecutive failures, a standard error permanently stops the workflow to prevent infinite restart loops. All of this state survives server restarts because the workflow is durable. | ||
|
|
||
| <Callout type="info"> | ||
| `FatalError` is designed for use inside `"use step"` functions to stop step retries. At the workflow level, a standard `throw new Error(...)` produces the same workflow failure state. | ||
| </Callout> | ||
|
|
||
| ## Graceful Shutdown | ||
|
|
||
| To stop a recurring workflow from the outside, use a [hook](/docs/foundations/hooks). Attach the listener once before the loop — when external code sends data to the hook, the loop breaks on the next iteration. | ||
|
|
||
| ```typescript title="workflows/stoppable-job.ts" lineNumbers | ||
| import { sleep, createHook } from "workflow"; | ||
| declare function runJob(): Promise<void>; // @setup | ||
|
|
||
| export async function stoppableJobWorkflow() { | ||
| "use workflow"; | ||
|
|
||
| const stopHook = createHook<{ reason: string }>({ // [!code highlight] | ||
| token: "stop-recurring-job", // [!code highlight] | ||
| }); // [!code highlight] | ||
|
|
||
| let stopped = false; | ||
| stopHook.then(() => { stopped = true; }); // [!code highlight] | ||
|
|
||
| while (true) { | ||
| if (stopped) { break; } // [!code highlight] | ||
|
|
||
| await runJob(); | ||
| await sleep("1 hour"); | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| To stop the workflow, call `resumeHook()` with the custom token from any external context: | ||
|
|
||
| ```typescript title="app/api/stop-job/route.ts" lineNumbers | ||
| import { resumeHook } from "workflow/api"; | ||
|
|
||
| export async function POST() { | ||
| await resumeHook("stop-recurring-job", { reason: "Manual shutdown" }); | ||
| return Response.json({ stopped: true }); | ||
| } | ||
| ``` | ||
|
|
||
| The custom token `"stop-recurring-job"` lets external code address the hook without needing to know the workflow's run ID. See [Custom Tokens for Deterministic Hooks](/docs/foundations/hooks#custom-tokens-for-deterministic-hooks) for more on this pattern. | ||
|
|
||
| <Callout type="info"> | ||
| If you don't need to pass data when stopping the workflow, you can cancel a run directly using the `Run` object from [`start()`](/docs/api-reference/workflow-api/start): `await run.cancel()`. The hook-based approach above is more flexible since it lets the workflow receive a reason or other data before stopping. | ||
| </Callout> | ||
|
|
||
| ## Related Documentation | ||
|
|
||
| - [`sleep()` API Reference](/docs/api-reference/workflow/sleep) - Full documentation on duration formats and date-based sleep | ||
| - [Common Patterns](/docs/foundations/common-patterns) - Sequential, parallel, timeout, and composition patterns | ||
| - [Hooks & Webhooks](/docs/foundations/hooks) - Pause workflows and resume them with external data | ||
| - [Starting Workflows](/docs/foundations/starting-workflows) - Trigger workflows with `start()` and track execution | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
consecutiveFailures >= 10escape hatch is dead code becauseconsecutiveFailuresresets to 0 at>= 3, so it can never reach 10 — the workflow restarts the service in an infinite loop with no termination.