Skip to main content

Migrate to Cloud

Migrating to Temporal Cloud from a self-hosted Cluster will have different requirements depending on your usage. This guide provides some guidance based on our experience helping customers of all sizes successfully migrate.

What to expect from a migration​

Depending on your Workflows’ requirements, the migration process may be as simple as changing a few parameters, or may require more extensive code changes. There are two aspects to consider when migrating: your Temporal Client connection code and your Workflow Executions. Here’s a high-level overview of what you can expect:

  • Introduce another Temporal Client to your Starter and Worker Processes: Configure and deploy a new Temporal Client so that Temporal Cloud becomes responsible for new Workflow Executions.
  • Migrate Workflow Executions: There are different approaches for new, running, and completed Workflow Executions.
    • New Workflow Executions: When you no longer need to send Signals or Queries to your self-hosted Cluster, you can deprecate your old Client code. Until then, your self-hosted Cluster can receive relevant traffic, while new Workflow Executions are sent to Temporal Cloud.
    • Running Workflow Executions: Short-running Workflows can often be drained and then started again on Temporal Cloud. Long-running Workflows that cannot be drained might require you to implement more code changes to pass the state of the currently running Workflow to Temporal Cloud.
    • Completed Workflow Executions: Completed Workflow Execution History cannot be automatically migrated to Temporal Cloud. Refer to Multi-Cluster Replication for more information.

Updating Client connection code in your Workers​

Whether you’re self-hosting Temporal or using Temporal Cloud, you manage runtime of your code. To migrate your Workflows to Temporal Cloud, you need to change some parameters in the Client connection code, such as updating the namespace and gRPC endpoint.

The changes needed to direct your Workflow to your Temporal Cloud Namespace are only a few lines of code, including:

Migrating your Workflow Executions​

A Temporal Cluster stores the complete Event History for the entire lifecycle of a Workflow Execution. To migrate from a self-hosted Temporal Cluster to Temporal Cloud, take into account the current state, Event History, and any future expectations of your Workflow Executions.

New Workflows are automatically executed on Temporal Cloud. Once you’ve made the code changes in Step 1, and your new code is deployed, new Workflow Executions will be sent to Temporal Cloud. Existing Workflows must receive Signals to migrate and re-execute on Cloud. If you maintain your self-hosted instance, you will still be able to use it to access any execution history from before your migration. You can also export JSON from your previous execution history, that you can then import into your own analytics system.

Running Workflows can either be drained or migrated. If your Workflow can be completed before any compelling event which drives a move to Temporal Cloud, those Workflows can be automatically restarted on Temporal Cloud. If your Workflows need to run continuously, you must migrate Workflows while they are running. To accomplish this migration, cancel your current Workflow and pass the current state to a new Workflow in Temporal Cloud. Refer to this repository for an example of migrating running Workflows in Java.

When performing a live migration, make sure your Worker capacity can support the migration load. Both a Signal and a Query will be executed during the course of the migration. Also, the Query API loads the entire history of Workflows into Workers to compute the result (if they are not already cached). That means that your self-hosted cluster Worker capacity will need to support having those executions in memory to serve those requests. The volume of these requests might be high to execute against all the matches to a ListFilter.

Considerations when resuming Workflows on a new Cluster or Namespace​

  • Skipping Steps: If your Workflow steps cannot guarantee idempotency, determine whether you need to skip those steps when resuming the execution in the target Namespace.
  • Elapsed Time: If your Workflow is “resuming sleep” when in the target Namespace, determine how you will calculate the delta for the sleep invocation in the new execution.
  • Child Relationships: If your Workflow has Child Workflow relationships (other than Detached Parent Close Policy children), determine how you can pass the state of those children into the parent to execute the child in a resumed state.
  • Heartbeat state: If you have long running activities relying on heartbeat state, determine how you can resume these activities in the target Namespace.
  • Child Workflows with the same type as their Parent types are returned in List Filters used to gather relevant executions. Unless these are Detached ParentClosePolicy children, this is not what you want since the Parent/Child relationship will not be carried over to the target Namespace.
  • Long running activities that use heartbeat details will not receive the latest details in the target Namespace.
  • Duration between Awaitables inside a Workflow definition needs to be considered for elapsed time accuracy when resuming in the target Namespace.
  • When Signaling directly from one Workflow to another, make sure to handle NotFound executions in the target Namespace. The Workflows may resume out of order.

Other considerations when migrating​

Multi-Cluster Replication​

Multi-Cluster Replication is an experimental feature which asynchronously replicates Workflow Executions from active Clusters to other passive Clusters for backup and state reconstruction. Migrating Execution History from a self-hosted Temporal Cluster to Temporal Cloud is not currently supported. However, a migration tool based on Multi-Cluster Replication, which will enable this, is currently in development for Temporal Cloud. If you have used this feature locally or you are interested in using it to migrate to Temporal Cloud, create a support ticket or watch this space for more information about public availability.