How the daily run works
See How workflows work for the design side.
The daily run is what happens after a workflow is built: its jobs move through a series of statuses as dependencies are satisfied, work is dispatched to agents, and results come back. Operators watch and act on the run in the Processes view.
Lifecycle
Status families
Every job status falls into one of these families (the statuses reference lists them all):
| Family | Meaning |
|---|---|
| Held | Held; won't run until released. |
| Waiting | Eligible but waiting on something — a dependency, its start time, a resource, or an agent. |
| Running | Actively starting or running. |
| Pending-term | Finished, but completion is still being processed. Transitional, not stuck. |
| Terminal | Done — finished OK, fixed, failed, initialization error, missed start time, cancelled, or skipped. |
| Under review | A failed job an operator has flagged for review. Not terminal: the workflow does not complete while a job is under review. |
Under review holds the workflow open. Flagging a failed job Under Review releases dependents that wait for it to fail (or to finish either way), but the workflow stays in process until you resolve the job — Mark Fixed, Restart, Skip, Cancel, Mark Finished OK or Mark Failed — or Close the workflow.
Two-phase completion
When a job finishes, it first enters a pending-term status (e.g. FINISHED_OK_PENDING_TERM)
while completion is made durable, then settles to its final status. Dependent jobs are not
released until that settles — so a brief pending-term state is normal, not a hang. (Cancellation
and initialization errors release dependents immediately.)
Recovery
The run survives a restart. Every status change is saved to the job's record as it happens, so when the service starts again it reloads the active workflow instances and their jobs from those records and carries on from where they were. In-flight jobs are reconciled rather than lost.
- A job waiting is usually working as designed — identify what it's waiting on (dependency, start time, resource, machine) before treating it as a fault.
- Pending-term is transitional. If a job is stuck there for a long time, that's worth investigating; a momentary pending-term is not.
- On hold is intentional until released.
Related topics