Worker Versioning

Worker Versioning is a Temporal feature that allows you to confidently deploy new changes to the Workflows running on your Workers without breaking them. Temporal enables this by helping you manage different builds or versions, formally called Worker Deployment Versions.

Worker Versioning unlocks important benefits for users of blue-green or rainbow deployments.

• Ramping traffic gradually to a new Worker Deployment Version.
• Verifying a new Deployment Version with tests before sending production traffic to it.
• Instant rollback when you detect that a new Deployment Version is broken.

In addition, Worker Versioning introduces Workflow Pinning. For pinned Workflow Types, each execution runs entirely on the Worker Deployment Version where it started. You need not worry about making breaking code changes to running, pinned Workflows.

To use Workflow Pinning, we recommend using rainbow deployments.

tip

Watch this Temporal Replay 2025 talk to learn more about Worker Versioning and see a demo.

note

Worker Versioning is currently available in Public Preview.

Minimum versions:

• Go SDK version v1.35.0
• Python v1.11
• Java v1.29
• Typescript v1.12
• Other SDKs: coming soon!

Self-hosted users:

• Minimum Temporal CLI version v1.4.0
• Minimum Temporal Server version: v1.28.0
• Minimum Temporal UI Version v2.38.0

Getting Started with Worker Versioning

To get started with Worker Versioning, you should understand some concepts around versioning and deployments.

• A Worker Deployment is a deployment or service across multiple versions. In a rainbow deployment, more than two active Deployment Versions can run at once.
• A Worker Deployment Version is a version of a deployment or service. It can have multiple Workers polling on multiple Task Queues, but they all run the same build.
• A Build ID, in combination with a Worker Deployment name, identifies a single Worker Deployment Version.
• When a versioned worker polls on a task queue, that task queue becomes part of that Worker's version. That version's Worker Deployment controls how the task queue matches Workflow Tasks with Workers.
• Using Workflow Pinning, you can declare each Workflow type to have a Versioning Behavior, either Pinned or Auto-Upgrade.
  • A Pinned Workflow is guaranteed to complete on a single Worker Deployment Version.
  • An Auto-Upgrade Workflow will move to the latest Worker Deployment Version automatically whenever you change the current version. Auto-upgrade Workflows are not restricted to a single Deployment Version and need to be kept replay-safe manually, i.e. with patching.
  • Both Pinned and Auto-Upgrade Workflows are guaranteed to start only on the Current or Ramping Version of their Worker Deployment.
  • Workflow Pinning is designed for use with rainbow deployments. See Deployment Systems.
• Each Worker Deployment has a single Current Version which is where workflows are routed to unless they were previously pinned on a different version. Other versions can continue polling to allow pinned Workflows to finish executing, or in case you need to roll back. If no current version is specified, the default is unversioned.
• Each Worker Deployment can have a Ramping Version which is where a configurable percentage of Workflows are routed to unless they were previously pinned on a different version. The ramp percentage can be in the range [0, 100]. Workflows that don't go to the Ramping Version will go to the Current Version. If no Ramping Version is specified, 100% of new Workflows and auto-upgrade Workflows will go to the Current Version.

Setting up your deployment system

If you haven't already, you'll want to pick a container deployment solution for your Workers.

You also need to pick among three common deployment strategies:

• A rolling deployment strategy upgrades Workers in place with little control over how quickly they cut over and only a slow ability to roll Workers back. Rolling deploys have minimal footprint but tend to provide lower availability than the other strategies and are incompatible with Worker Versioning.
• A blue-green deployment strategy maintains two "colors," or Worker Deployment Versions simultaneously and can control how traffic is routed between them. This allows you to maximize your uptime with features like instant rollback and ramping. Worker Versioning enables the routing control that blue-green deployments need.
• A rainbow deployment strategy is like blue-green but with more colors, allowing Workflow Pinning. You can deploy new revisions of your Workflows freely while older versions drain. Using Worker Versioning, Temporal lets you know when all the Workflows of a given version are drained so that you can sunset it.

Configuring a Worker for Versioning

You'll need to add a few additional configuration parameters to your Workers to toggle on Worker Versioning. There are three new parameters, with different names depending on the language:

• UseVersioning: This enables the Versioning functionality for this Worker.
• A Version to identify the revision that this Worker will be allowed to execute. This is a combination of a deployment name and a build ID number.
• (Optional) The Default Versioning Behavior. If unset, you'll be required to specify the behavior on each Workflow. Or you can default to Pinned or Auto-Upgrade.

Follow the example for your SDK below:

Go

buildID:= mustGetEnv("MY_BUILD_ID")
w := worker.New(c, myTaskQueue, worker.Options{
      DeploymentOptions: worker.DeploymentOptions{
          UseVersioning: true,
          Version: worker.WorkerDeploymentVersion{
              DeploymentName: "llm_srv",
              BuildId:        buildID,
          },
          DefaultVersioningBehavior: workflow.VersioningBehaviorAutoUpgrade,
},
})

Java

WorkerOptions options =
    WorkerOptions.newBuilder()
        .setDeploymentOptions(
            WorkerDeploymentOptions.newBuilder()
                .setVersion(new WorkerDeploymentVersion("llm_srv", "1.0"))
                .setUseVersioning(true)
                .setDefaultVersioningBehavior(VersioningBehavior.AUTO_UPGRADE)
                .build())
        .build();

Python

Worker(
    client,
    task_queue="mytaskqueue",
    workflows=workflows,
    activities=activities,
    deployment_config=WorkerDeploymentConfig(
        version=WorkerDeploymentVersion(
            deployment_name="llm_srv",
            build_id=my_env.build_id),
        use_worker_versioning=True,
        default_versioning_behavior=VersioningBehavior.AUTO_UPGRADE
    ),

)

TypeScript

const myWorker = await Worker.create({
workflowsPath: require.resolve('./workflows'),
taskQueue,
workerDeploymentOptions: {
    useWorkerVersioning: true,
    version: {	buildId: '1.0', deploymentName: 'llm_srv'},
    defaultVersioningBehavior: 'AUTO_UPGRADE',
},
connection: nativeConnection,
});

PHP

PHP example coming soon.

.NET

var myWorker = new TemporalWorker(
    Client,
    new TemporalWorkerOptions(taskQueue)
    {DeploymentOptions = new(new("llm_srv", "1.0"), true)
      { DefaultVersioningBehavior = VersioningBehavior.AutoUpgrade },
    }.AddWorkflow<MyWorkflow>());

Ruby

worker = Temporalio::Worker.new(
  client: client,
  task_queue: task_queue,
  workflows: [MyWorkflow],
  deployment_options: Temporalio::Worker::DeploymentOptions.new(
      version: Temporalio::WorkerDeploymentVersion.new(
          deployment_name: 'llm_srv',
          build_id: '1.0'
      ),
      use_worker_versioning: true,
      default_versioning_behavior: Temporalio::VersioningBehavior::AUTO_UPGRADE
  )
)

Which Default Versioning Behavior should you choose?

If you are using blue-green deployments, you should default to Auto-Upgrade and should not use Workflow Pinning.

Otherwise, if your Worker and Workflows are new, we suggest not providing a DefaultVersioningBehavior. In general, each Workflow Type should be annotated as Auto-Upgrade or Pinned. If all of your Workflows will be short-running for the foreseeable future, you can default to Pinned.

Many users who are migrating to Worker Versioning will start by defaulting to Auto-Upgrade until they have had time to annotate their Workflows. This default is the most similar to the legacy behavior. Once each Workflow Type is annotated, you can remove the DefaultVersioningBehavior.

Rolling out changes with the CLI

Next, deploy your Worker with the additional configuration parameters. Before making any Workflow revisions, you can use the temporal CLI to check which of your Worker versions are currently polling:

You can view the Versions that are part of a Deployment with temporal worker deployment describe:

temporal worker deployment describe --name="$MY_DEPLOYMENT"

To activate a Deployment Version, use temporal worker deployment set-current-version, specifying the deployment name and a Build ID:

temporal worker deployment set-current-version \
    --deployment-name "YourDeploymentName" \
    --build-id "YourBuildID"

To ramp a Deployment Version up to some percentage of your overall Worker fleet, use set-ramping version, with the same parameters and a ramping percentage:

temporal worker deployment set-ramping-version \
    --deployment-name "YourDeploymentName" \
    --build-id "YourBuildID" \
    --percentage=5

You can verify that Workflows are cutting over to that version with describe -w YourWorkflowID:

temporal workflow describe -w YourWorkflowID

That returns the new Version that the workflow is running on:

Versioning Info:

  Behavior               AutoUpgrade
  Version                llm_srv.2.0
  OverrideBehavior       Unspecified

Marking a Workflow Type as Pinned

You can mark a Workflow Type as pinned when you register it by adding an additional Pinned parameter. This will cause it to remain on its original deployed version:

Go

// w is the Worker configured as in the previous example
w.RegisterWorkflowWithOptions(HelloWorld, workflow.RegisterOptions{
	// or workflow.VersioningBehaviorAutoUpgrade
    VersioningBehavior: workflow.VersioningBehaviorPinned,
})

Java

@WorkflowInterface
public interface HelloWorld {
  @WorkflowMethod
  String hello();
}

public static class HelloWorldImpl implements HelloWorld {
@Override
@WorkflowVersioningBehavior(VersioningBehavior.PINNED)
public String hello() {
return "Hello, World!";
}
}

Python

@workflow.defn(versioning_behavior=VersioningBehavior.PINNED)
class HelloWorld:
    @workflow.run
    async def run(self):
        return "hello world!"

TypeScript

setWorkflowOptions({ versioningBehavior: 'PINNED' }, helloWorld);
export async function helloWorld(): Promise<string> {
  return 'hello world!';
}

PHP

PHP example coming soon.

.NET

[Workflow(VersioningBehavior = VersioningBehavior.Pinned)]
public class HelloWorld
{
    [WorkflowRun]
    public async Task<string> RunAsync()
    {
        return "hello world!";
    }
}

Ruby

class HelloWorld < Temporalio::Workflow::Definition
  workflow_versioning_behavior Temporalio::VersioningBehavior::PINNED

  def execute
    'hello world!'
  end
end

You can check your set of Deployment Versions with temporal worker deployment describe:

temporal worker deployment describe --name="$MY_DEPLOYMENT"

Moving a pinned Workflow

Sometimes you'll need to manually move a set of pinned workflows off of a version that has a bug to a version with the fix.

If you need to move a pinned Workflow to a new version, use temporal workflow update-options:

temporal workflow update-options \
    --workflow-id="MyWorkflowId" \
    --versioning-override-behavior=pinned \           
    --versioning-override-pinned-version="$MY_DEPLOYMENT_VERSION"

You can move several Workflows at once matching a --query parameter:

temporal workflow update-options \
  --query="TemporalWorkerDeploymentVersion=$MY_BAD_DEPLOYMENT_VERSION" \
  --versioning-override-behavior=pinned \
  --versioning-override-pinned-version="$MY_PATCH_DEPLOYMENT_VERSION"

In this scenario, you may also need to use the other Versioning APIs to patch your Workflow.

Sunsetting an old Deployment Version

A Worker Deployment Version moves through the following states:

1. Inactive: The version exists because a Worker with that version has polled the server. If this version never becomes Active, it will never be Draining or Drained.
2. Active: The version is either Current or Ramping, so it is accepting new Workflows and existing auto-upgrade Workflows.
3. Draining: The version stopped being Current or Ramping, and it has open pinned Workflows running on it. It is possible to be Draining and have no open pinned Workflows for a short time, since the drainage status is updated periodically.
4. Drained: The version was draining and now all the pinned Workflows that were running on it are closed.

You can see these statuses when you describe a Worker Deployment in the WorkerDeploymentVersionStatus of each VersionSummary, or by describing the version directly. When a version is Draining or Drained, that is displayed in a value called DrainageStatus. Periodically, the Temporal Service will refresh this status by counting any open pinned workflows using that version. On each refresh, DrainageInfo.last_checked_time is updated. Eventually, DrainageInfo will report that the version is fully drained. At this point, no Workflows are still running on that version and no more will be automatically routed to it, so you can consider shutting down the running Workers.

You can monitor this by checking WorkerDeploymentInfo.VersionSummaries or with temporal worker deployment describe-version:

temporal worker deployment describe-version \
    --deployment-name "YourDeploymentName" \
    --build-id "YourBuildID"

Worker Deployment Version:
  Version                  llm_srv.1.0
  CreateTime               5 hours ago
  RoutingChangedTime       32 seconds ago
  RampPercentage           0
  DrainageStatus           draining
  DrainageLastChangedTime  31 seconds ago
  DrainageLastCheckedTime  31 seconds ago

Task Queues:
     Name        Type  
  hello-world  activity
  hello-world  workflow

If you have implemented Queries on closed pinned Workflows, you may need to keep some Workers running to handle them.

Adding a pre-deployment test

Before deploying a new Workflow revision, you can test it with synthetic traffic.

To do this, utilize pinning in your tests, following the examples below:

Go

workflowOptions := client.StartWorkflowOptions{
	ID:        "MyWorkflowId",
	TaskQueue: "MyTaskQueue",
	VersioningOverride: client.VersioningOverride{
		Behavior: workflow.VersioningBehaviorPinned,
		PinnedVersion: "$MY_TEST_DEPLOYMENT_VERSION",
	},
}
// c is an initialized Client
we, err := c.ExecuteWorkflow(context.Background(), workflowOptions, HelloWorld, "Hello")

Java

Java example coming soon.

Python

Python example coming soon.

TypeScript

const handle = await client.workflow.start('helloWorld', {
taskQueue: 'MyTaskQueue',
workflowId: 'MyWorkflowId',
versioningOverride: {
      pinnedTo: { buildId: '1.0', deploymentName: 'deploy-name' },
},
});

PHP

PHP example coming soon.

.NET

var workerV1 = new WorkerDeploymentVersion("deploy-name", "1.0");
var handle = await Client.StartWorkflowAsync(
    (HelloWorld wf) => wf.RunAsync(),
      	new(id: "MyWorkflowId", taskQueue: "MyTaskQueue")
      	{
           VersioningOverride = new VersioningOverride.Pinned(workerV1),
        }
);

Ruby

worker_v1 = Temporalio::WorkerDeploymentVersion.new(
  deployment_name: 'deploy-name',
  build_id: '1.0'
)
handle = env.client.start_workflow(
  HelloWorld,
  id: 'MyWorkflowId',
  task_queue: 'MyTaskQueue',
  versioning_override: Temporalio::VersioningOverride.pinned(worker_v1)
)

This covers the complete lifecycle of working with Worker Versioning. We are continuing to improve this feature, and we welcome any feedback or feature requests using the sidebar link!