Skip to main content

Worker performance options

View Markdown

Each Worker can be configured by providing custom Worker options (WorkerOptions) at instantiation. Options are specific to individual Workers and do not affect other members of your fleet.

Executor slot options​

The maxConcurrentWorkflowTaskExecutionSize and maxConcurrentActivityExecutionSize options define the number of total available Workflow Task and Activity Task slots for a Worker.

caution
  • Worker tuners supersede the existing maxConcurrentXXXTask style Worker options. Using both styles will cause an error at Worker initialization time.

Configuring poller options​

The Temporal SDKs support Poller Autoscaling, which automatically selects an appropriate number of pollers based on need. Using this feature results in more efficient poller usage, better throughput, and schedule-to-start latency improvements.

Poller Autoscaling does not change your Worker count, compute capacity, or task execution limits.

tip

PollerBehaviorAutoscaling is only enabled in Temporal Server v1.28.0 and later.

Poller Autoscaling by default in Temporal Cloud​

Temporal Cloud is enabling Poller Autoscaling by default for all Namespaces in stages. After Poller Autoscaling is enabled for your Namespace, Workers on a supported SDK version use it for every poller type whose poller count and poller behavior options are unset. Poller Autoscaling replaces the SDK's fixed default poller counts for those poller types, with no configuration change required. The change takes effect the next time each Worker starts. Workers that are already running keep their current polling behavior until they restart.

To check whether a Worker uses Poller Autoscaling, select the Worker on the Workers tab in the Temporal UI, or call the DescribeWorker API and check the IsAutoscaling field for each task type..

The following SDK versions use Poller Autoscaling by default when it's enabled for the Namespace:

SDKMinimum version
Go1.47.0
Java1.38.0
Python1.36.0
.NET1.22.0
Ruby1.11.0
Rust0.6.0

The TypeScript SDK release that uses Poller Autoscaling by default is not yet available. To use Poller Autoscaling with the currently released TypeScript SDK, enable it explicitly.

Workers with a specific fixed poller count explicitly configured will not have poller autoscaling automatically enabled. Temporal applies the default separately to Workflow, Activity, and Nexus Task pollers. For example, if you configure a fixed Workflow Task poller count, Workflow Task pollers stay fixed, while Activity and Nexus Task pollers with unset options use Poller Autoscaling.

Enable poller autoscaling explicitly​

You can enable Poller Autoscaling in your Worker by setting the *_task_poller_behavior options to PollerBehaviorAutoscaling. Names vary by SDK; see the SDK examples below. Use this approach when:

  • Poller Autoscaling isn't yet enabled for your Namespace.
  • When your SDK version doesn't use it by default.
  • When you connect to a self-hosted Temporal Service that supports Poller Autoscaling.
  • If you need to configure the Poller Autoscaling settings (min, initial, max pollers). Auto-enrollment uses the default settings.

Poller Autoscaling also provides minimum, maximum, and initial poller counts. In every SDK, the defaults are a minimum of 1, an initial count of 5, and a maximum of 100 pollers for each poller type. Temporal recommends using these default bounds for most workloads rather than manually tuning them.

Retain fixed poller settings​

To keep fixed poller counts after Poller Autoscaling is enabled for your Namespace, explicitly configure a fixed poller count or a fixed poller behavior for each poller type you want to keep fixed, then restart your Workers. The fixed behavior is named PollerBehaviorSimpleMaximum or PollerBehavior.SimpleMaximum in most SDKs, and uses type: 'simple-maximum' in TypeScript. Refer to your SDK's API reference in the SDK examples below for its configuration options.

SDK examples​

Select a concept to highlight the matching poller option:

Go SDK docs

Highlight a concept
w := worker.New(c, "my-task-queue", worker.Options{
WorkflowTaskPollerBehavior: worker.NewPollerBehaviorAutoscaling(worker.PollerBehaviorAutoscalingOptions{}),
ActivityTaskPollerBehavior: worker.NewPollerBehaviorAutoscaling(worker.PollerBehaviorAutoscalingOptions{}),
NexusTaskPollerBehavior: worker.NewPollerBehaviorAutoscaling(worker.PollerBehaviorAutoscalingOptions{}),
})

Cache options (Java SDK)​

A Workflow Cache is created and shared between all Workers on a single host. It's designed to limit the resources used by the cache for each host/process. These options are defined on WorkerFactoryOptions:

  • WorkerFactoryOptions#workflowCacheSize defines the maximum number of cached Workflow Executions. Each cached Workflow contains at least one Workflow thread and its resources (memory, etc.).
  • maxWorkflowThreadCount defines the maximum number of Workflow threads that may exist concurrently at any time.

These cache options limit the resource consumption of the in-memory Workflow cache. Workflow cache options are shared between all Workers because the Workflow cache is tightly integrated with the resource consumption of the entire host. This includes memory and the total thread count, which should be limited per host/JVM.

For Go, use SetStickyWorkflowCacheSize. For Python, use the max_cached_workflows Worker option.

"Large value" drawbacks​

There are drawbacks when you use "large values everywhere." As with any multithreading system, specifying excessively large values without monitoring with the SDK and system metrics leads to constant resource contention/stealing This decreases the total throughput and increases latency jitter of the system.

Invariants (JavaSDK only)​

These properties should always be true for a Worker's configuration.

Perform this sanity check after the adjustments to Worker settings.

  1. workflowCacheSize should be ≤ maxWorkflowThreadCount. Each Workflow has at least one Workflow thread.
  2. maxConcurrentWorkflowTaskExecutionSize should be ≤ maxWorkflowThreadCount. It's recommended that maxWorkflowThreadCount be at least 2x of maxConcurrentWorkflowTaskExecutionSize. This is because having more Worker slots than the Workflow cache size will lead to resource allocation issues between executors and cause unpredictable delays.
  3. maxConcurrentWorkflowTaskPollers should be significantly ≤ maxConcurrentWorkflowTaskExecutionSize. And maxConcurrentActivityTaskPollers should be significantly ≤ maxConcurrentActivityExecutionSize. The number of pollers should always be lower than the number of executors.