Skip to main content

Use thresholds and resources

Thresholds and resources let jobs coordinate beyond simple "run after" ordering: gate a job on a value, or limit how many jobs hit a shared system at once.

What this solves

Simple "run after" ordering can't express "wait until enough is done" or "don't let too many jobs hit this system at once", so jobs either start too early or overwhelm a shared database or host.

Thresholds: gate a job on a value​

A threshold is a named number. A job can depend on it and wait until it reaches a required value: useful for "wait until N things are done" style gating.

To use a threshold, complete the following steps:

  1. Go to Toolkit, open the Thresholds tab, select New Threshold, and give it a Name and a starting Value. Everything is set in one dialog over the list.
  2. In a job's Dependencies tab, add a threshold dependency and set the Operator and the Value it requires.

Resources: limit concurrency​

A resource is a named pool with a total capacity. A job reserves units while it runs and releases them when it finishes, so you can cap how many jobs run against a shared system (a core database, say) at the same time. The total is shared by every environment: the units held in all of them together never pass it.

To use a resource, complete the following steps:

  1. Go to Toolkit, open the Resources tab, select New Resource, and give it a Name and a Total capacity.
  2. In a job's Dependencies tab, add a resource dependency and set the units it needs in Required — or choose Use all for a job that must have the whole resource to itself.
Good to know
  • On Processes, a job waiting on a resource shows a Wait Resource Dependency status until it can have what it needs; a job waiting on a threshold shows Wait Threshold Dependency. The units are taken only when the job is dispatched, so if another job takes them first it goes back to Wait Resource Dependency — naming the resource it could not get — and tries again.
  • Use all is for the job that must not share: it waits until nothing in any environment is using the resource, then takes all of it. While it waits, the resource is reserved against jobs of the same or lower priority in every environment, so a steady trickle of small jobs, in this environment or another, cannot keep it waiting forever.
  • Thresholds and resources are workspace-scoped: the Toolkit and the job editor's dependency pickers list only the thresholds and resources in workspaces your role can see. Saving a workflow that adds or changes a dependency on one you cannot view is refused; dependencies you leave alone or remove are not checked, so the rest of the workflow stays editable. See Dependencies on a threshold or resource you cannot view.
  • Size a resource's total to the real limit of the system you're protecting.
  • To see who is holding a resource right now, use Current Allocations on its row and pick an environment from the Environment list: you get that environment's units in use against the total, what's available across every environment, and a row per live allocation naming the job and workflow instance holding it. A job waiting while its own environment is idle usually means another environment holds the units.
  • Units held in an environment that is inactive or has been deleted are listed there too, and drain the resource for every environment. Once you know their jobs have stopped, select Release on each row (Resources Edit). Deleting an environment releases its units for you. See Releasing units an environment no longer uses.
  • Deleting a threshold or resource that a job depends on is refused, and the confirmation lists the jobs that depend on it — so you know what to unpick first.

Related topics