Skip to main content

How does Temporal handle application data?

This guide provides an overview of data handling using a Data Converter on the Temporal Platform.

A Data Converter is a Temporal SDK component that serializes and encodes data entering and exiting a Temporal Cluster. It is used by the Temporal SDK framework to serialize/deserialize data such as input and output of Activities and Workflows that need to be sent over the wire to the Temporal Cluster.

Data Converter encodes and decodes data

Data Converter encodes and decodes data

The Data Converter encodes data from your application to a Payload before it is sent to the Temporal Cluster in the Client call. When the Temporal Server sends the encoded data back to the Worker, the Data Converter decodes it for processing within your application. This technique ensures that all your sensitive data exists in its original format only on hosts that you control.

The main pieces of data that run through the Data Converter are arguments and return values:

  • The Client:
    • Encodes Workflow, Signal, and Query arguments.
    • Decodes Workflow and Query return values.
  • The Worker:
    • Decodes Workflow, Signal, and Query arguments.
    • Encodes Workflow and Query return values.
    • Decodes and encodes Activity arguments and return values.

Each piece of data (like a single argument or return value) is encoded as a Payload Protobuf message, which consists of binary data and key-value metadata.

For details, see the API references:

What is a Payload?​

A Payload represents binary data such as input and output from Activities and Workflows. Payloads contain metadata that describe the binary data, such as its data type or other arbitrary values for use by custom encoders/converters.

When processed through the SDK, the default Data Converter serializes your data/value to a Payload before sending it to the Temporal Server. The default Data Converter processes supported type values to Payloads, and you can create a custom Payload Converter to convert your custom object types.

You can additionally apply custom codecs (such as for encryption or compression) on your Payloads to wrap them into new encoded Payloads.

What is a default Data Converter?​

Each Temporal SDK includes and uses a default Data Converter. The default Data Converter converts objects to bytes using a series of Payload Converters and supports binary, Protobufs, and JSON formats. It encodes values in the following order:

  • Null
  • Byte array
  • Protobuf JSON
  • JSON

For example:

  • If a value is an instance of a Protobuf message, it is encoded with proto3 JSON.
  • If a value isn't null, binary, or a Protobuf, it is encoded as JSON. If any part of it is not serializable as JSON, an error is thrown.

The default Data Converter serializes objects based on their root type, rather than nested types. The JSON serializers of some SDKs cannot process lists with Protobuf children objects without implementing a custom Data Converter.

What is a custom Data Converter?​

A custom Data Converter extends the default Data Converter with custom logic for Payload conversion or encoding.

You can create a custom Data Converter to alter formats (for example, using MessagePack instead of JSON) or add compression and encryption.

You can customize the default Data Converter behavior in two ways:

  • A Payload Converter serializes data, converting objects to bytes and back. To convert custom objects or data types to Payloads and back, use a custom Payload Converter and set it on a Data Converter.
  • A Payload Codec encodes and decodes Payloads, with bytes-to-bytes conversion. To use custom encryption and/or compression logic, create a custom Payload Codec with your encryption/compression logic in the encode function and your decryption/decompression logic in the decode function.

Custom Data Converters are not applied to all data; for example, Search Attributes are simple values and persisted unencoded so they can be indexed for searching.

A customized Data Converter can have the following three components:

For details on how to implement custom Payload Converters in your SDK, see Custom Payload conversion for the following SDKs:

For details on how to implement custom encryption and compression in your SDK, see Data Encryption.

What is a Payload Converter?​

A Payload Converter serializes data, converting values to bytes and back.

When you request a Workflow Execution through your Client and pass a data input, the input is serialized using a Data Converter that runs it through a set of Payload Converters. When your Workflow Execution starts, this data input is deserialized and passed as input to your Workflow.

For more information, see the API references.

For supported values, see default Data Converter.

Custom payload conversion​

If you use custom objects or types that are not supported by the Payload Converters provided in the SDKs, you can create a custom Payload Converter and configure the Data Converter with it to run the specific conversions.

You can set multiple encoding Payload Converters to run your conversions. When the Data Converter receives a value for conversion, it passes through each Payload Converter in sequence until the converter that handles the data type does the conversion.

For details on how to use the Payload Converter for custom data types, see Custom Payload Conversion in the following langauages:

What is a Failure Converter?​

A Failure Converter converts error objects to proto Failures and back. The default Failure Converter copies error messages and stack traces as plain text.

For details, see the API references.

You can make a custom Failure Converter, but if you use multiple SDKs, you must implement the same logic in each. Creating a custom Failure Converter is not yet supported in Java.

Failure messages and stack traces are not encoded as codec-capable Payloads by default; you must explicitly enable encoding these common attributes on failures. If your errors might contain sensitive information, you can encrypt the message and stack trace by configuring the default Failure Converter to use your encoded attributes, in which case it moves your message and stack_trace fields to a Payload that's run through your codec.

What is a Payload Codec?​

A Payload Codec transforms an array of Payloads (for example, a list of Workflow arguments) into another array of Payloads.

The Payload Codec is an optional step that happens between the wire and the Payload Converter:

User code <--> Payload Converter <--> Payload Codec <--> Wire <--> Temporal Server

When serializing to Payloads, the Payload Converter is applied first to convert your objects to bytes, followed by codecs that convert bytes to bytes. When deserializing from Payloads, codecs are applied first to last to reverse the effect, followed by the Payload Converter.

For details, see the API references.

Use a custom Payload Codec to transform your Payloads; for example, implementing compression and/or encryption on your Workflow Execution data.

Encryption​

Using end-to-end encryption in your custom Data Converter ensures that sensitive application data is secure when handled by the Temporal Server.

Apply your encryption logic in a custom Payload Codec and use it locally to encrypt data. You maintain all the encryption keys, and the Temporal Server sees only encrypted data. Your data exists unencrypted only on the Client and the Worker process that is executing the Workflows and Activities, on hosts that you control.

For details, see Securing your data.

The following samples use encryption (AES GCM with 256-bit key) in a custom Data Converter:

What is remote data encoding?​

Remote data encoding is exposing your Payload Codec via HTTP endpoints to support remote encoding and decoding.

Running your encoding remotely allows you to use it with tctl to encode/decode data for several commands including tctl workflow start and with Temporal Web UI to encode and decode data in your Workflow Execution details view.

To run data encoding/decoding remotely, use a Codec Server. A Codec Server is an HTTP server that is configured to use your custom Payload Codec.

Before you use a remote data encoder to encode/decode your data, ensure that you consider all the security implications of running codecs remotely. For example, codecs that perform encryption may need to be secured to prevent decryption by untrusted callers.

Encoding data on the Web UI and tctl​

You can perform some operations on your Workflow Execution using tctl and the Web UI, such as starting or sending a Signal to an active Workflow Execution using tctl or canceling a Workflow Execution from the Web UI, which might require inputs that contain sensitive data.

To encode this data, specify your Codec Server endpoints with the tctl command and configure your Web UI to use the Codec Server endpoints.

Decoding data on the Web UI and tctl​

If you use custom encoding in your custom Data Converter, Payload data handled by the Temporal Cluster is encoded. Since the Web UI uses the Visibility database to show events and data stored on the Temporal Server, all data in the Workflow Execution History in your Web UI or tctl shows in the encoded format.

To see the original format of data in your Web UI and tctl, create a Codec Server with a remote data encoder and use the Payload Codec to decode your data locally.

Note that a remote data encoder is a separate system with access to your encryption keys and exposes APIs to encode and decode any data with the Payload Codec used. Evaluate and ensure that your remote data encoder endpoints are secured and only authorized users have access to them.

Samples:

What is a Codec Server?​

A Codec Server is an HTTP/HTTPS server that uses a custom Payload Codec to decode your data remotely through endpoints.

A Codec Server follows the Temporal Codec Server Protocol. It implements two endpoints:

  • /encode
  • /decode

Each endpoint receives and responds with a JSON body that has a payloads property with an array of Payloads. The endpoints run the Payloads through a Payload Codec before returning them.

Most SDKs provide example Codec Server implementation samples, listed here:

Usage​

When you apply custom encoding with encryption or compression on your Workflow data, it is stored in the encrypted/compressed format on the Temporal Server. For details on what data is encoded, see Securing your data.

To see decoded data when using the CLI or Web UI to perform some operations on a Workflow Execution, configure the Codec Server endpoint in the Web UI and CLI. When you configure the Codec Server endpoints, the CLI and Web UI send the encoded data to the Codec Server, and display the decoded data received from the Codec Server.

For details on creating your Codec Server, see Codec Server Setup.

After you start your Codec Server, configure your Codec Server endpoints.

Codec Server setup​

Use a Codec Server to decode your encoded payloads and integrate it with the Temporal Web UI and CLI commands when debugging your Workflows.

A Codec Server is an HTTP or HTTPS Server that you create and host. It must be configured to use a Payload Codec with the required decode logic and encryption keys. Temporal Cloud requires an HTTPS Codec Server.

The Codec Server is independent of the Temporal Server and decodes your encrypted payloads through endpoints. When you set the codec endpoint in the Temporal Web UI, the Web UI uses the remote endpoint to send encoded payloads to the Codec Server and receive decoded payloads from the Codec Server. See API contract requirements. Decoded payloads are then displayed in the Workflow Execution Event History on the Web UI.

Note that when you use a Codec Server, the decoded payloads are visible only to you on the Web UI; payloads on the Temporal Server (whether on Temporal Cloud or a self-hosted Temporal Cluster) remain encrypted.

Because you create, operate, and manage access to your Codec Server in your controlled environment, ensure that you consider the following:

  • When you set your codec endpoint with your Web UI, expect your Codec Server to receive a large number of requests per Workflow Execution from the Web UI.
  • Ensure that you secure access your Codec Server. For details, see Authorization.
  • The Temporal Web UI only displays the decoded payloads received from your Codec Server in real-time; it does not store or send the data back to the Temporal Server (whether on Cloud or self-hosted Temporal Cluster).
  • You might have latencies introduced in the Web UI when sending and receiving payloads to the Codec Server.

To create a Codec Server, you need the following components:

  • A Payload Codec with the requisite keys and logic to decode your encrypted payloads. You can use the Payload Codec that you applied with your Data Converter to encode your Payloads and configure it with your Codec Server. However, if you are writing your Codec Server in a different SDK from the one that applies the Data Converter, ensure that your logic and keys are correctly replicated.
  • Key management infrastructure or plan for sharing your encryption keys between the Workers and your Codec Server.
  • CORS configuration on the HTTP/HTTPS endpoints in your Codec Server for sending and receiving requests from the Temporal Web UI.
  • Optional: Secure access through VPN and access control. For details, see Authorization.

For examples on how to create your Codec Server, see following Codec Server implementation samples:

API contract specifications​

When you create your Codec Server to handle requests from the Web UI, the following requirements must be met.

Endpoints​

The Web UI/CLI calls the POST method with the /decode endpoint.

In your Codec Server, create a /decode path and pass the incoming payload to the decode method in your Payload Codec.

You can also add a verification step to check whether the incoming request has the required authorization to access the decode logic in your Payload Codec.

Headers​

Each request from the Web UI to your Codec Server includes the following headers:

  • Content-Type: application/json: Ensure that your Codec Server can accommodate this MIME type.

  • X-Namespace: {namespace}: This is a custom HTTP Header. Ensure that the CORS configuration in your Codec Server includes this header.

  • [Optional] Authorization: <credentials>: Include this in your CORS configuration when enabling authorization with your Codec Server.

For details on setting up authorization, see Authorization.

Request body​

The general specification for the POST request body contains payloads. By default, all field values in your payload are base64 encoded, regardless of whether they are encrypted by your custom codec implementation.

The following example shows a sample POST request body with base64 encoding.

{
  "payloads": [{
    "metadata": {
      "encoding": <base64EncodedEncodingHint>
    },
    "data": <encryptedPayloadData>
  }, ...]
}

CORS​

Enable Cross-Origin Resource Sharing (CORS) requests on your Codec Server to receive HTTP/HTTPS requests from the Temporal Web UI.

At a minimum, enable the following responses from your Codec Server to allow requests coming from the Temporal Web UI:

  • Access-Control-Allow-Origin
  • Access-Control-Allow-Methods
  • Access-Control-Allow-Headers

For example, for Temporal Cloud Web UI hosted at https://cloud.temporal.io, enable the following in your Codec Server:

  • Access-Control-Allow-Origin: https://cloud.temporal.io
  • Access-Control-Allow-Methods: POST, GET, OPTIONS
  • Access-Control-Allow-Headers: X-Namespace, Content-Type

For details on what a sample request/response looks like from the Temporal Web UI, see Sample Request/Response. If setting authorization, include Authorization in your Access-Control-Allow-Headers. For details on setting up authorization, see Authorization.

Authorization​

To enable authorization from the Web UI (for both self-hosted Cluster and Temporal Cloud), your Codec Server must be an HTTPS Server.

Temporal Cloud

The Temporal Cloud UI provides an option to pass access tokens (JWT) to your Codec Server endpoints. Use the access tokens to validate access and then return decoded payloads from the Codec Server.

You can enable this by selecting Pass access token in your Codec Server endpoint interface where you add your endpoint. Enabling this option in the Temporal Cloud UI adds an authorization header to each request sent to the Codec Server endpoint that you set.

In your Codec Server implementation, verify the signature on this access token (in your authorization header) against our JWKS endpoint.

The token provided from Temporal Cloud UI contains the email identifier of the person requesting access to the payloads. Based on the permissions you have provided to the user in your access control systems, set conditions in your Codec Server whether to return decoded payloads or just return the original encoded payloads.

Self-hosted Temporal Cluster

On self-hosted Temporal Clusters, configure authorization in the Web UI configuration in your Temporal Cluster setup.

With this enabled, you can pass access tokens to your Codec Server and validate the requests from the Web UI to the Codec Server endpoints that you set.

Note that with self-hosted Temporal Clusters, you must explicitly configure authorization specifications for the Web UI and CLI.

The following sample provides implementation examples for applying authentication on your Codec Server using the Go SDK.

Sample request/response​

Consider the following sample request/response when creating and hosting a Codec Server with the following specifications:

  • Scheme: https
  • Host: dev.mydomain.com/codec
  • Path: /decode
HTTP/1.1 POST /decode
Host: https://dev.mydomain.com/codec
Content-Type: application/json
X-Namespace: myapp-dev.acctid123
Authorization: Bearer <token>

{"payloads":[{"metadata":{"encoding":"anNvbi9wcm90b2J1Zg==","messageType":"dGVtcG9yYWxfc2hvcC5vcmNoZXN0cmF0aW9ucy52MS5TdGFydFNob3BwaW5nQ2FydFJlcXVlc3Q="},"data":"eyJjYXJ0SWQiOiJleGFtcGxlLWNhcnQiLCJzaG9wcGVySWQiOiJ5b3VyLXNob3BwZXItaWQtZXhhbXBsZSIsImVtYWlsIjoieW91ci1lbWFpbEBkb21haW4uY29tIn0"}]}

200 OK
Content-Type: application/json

{
  "payloads": [{
    "metadata":{
      "encoding": "json/protobuf",
      "messageType": "temporal_shop.orchestrations.v1.StartShoppingCartRequest"
    },
    "data":{
      "cartId":"example-cart",
      "shopperId":"your-shopper-id-example",
      "email":"your-email@domain.com"
    }}]
}

Hosting your Codec Server​

Your Codec Server can be hosted at an organization level or locally.

Organization-level hosting​

Hosting the Codec Server for your organization simplifies both key management used for decryption and versioning the codec itself. Consider the following details for a multi-tenant approach to setting up your Codec Server:

  • Ingress: Your server will require ingress configuration for your users to access the server.
  • Authorization: You must set explicit authorization checks to validate requests to your Codec Server.

Local hosting​

Locally hosting the Codec Server is simpler to get started. However, consider the following before choosing to do so:

  • A single URL configuration is accepted for the Cloud account. This means some agreed-upon policy on the URL must be made for everyone using the Namespaces in this account. For example, if you configure your remote codec endpoint to be http://localhost:8080/codec, every developer must host your Codec Server locally at that port.

    Alternatively, you can use the local hosts file to allow each developer to choose where to host. For example, configure the remote codec endpoint as http://codec.server and allow each developer to control what it maps to locally.

  • Distributing encryption keys that can decrypt the payloads at your organization can be a security risk.

Set your Codec Server endpoints with Web UI and CLI​

After you create your Codec Server and expose the requisite endpoints, set the endpoints in your Web UI and CLI.

Web UI​

On Temporal Cloud and self-hosted Temporal Clusters, you can configure a Codec Server endpoint to be used for a Namespace in the Web UI.

Codec Server endpoint Namespace setting

Codec Server endpoint Namespace setting

To set a Codec Server endpoint on a Namespace, do the following.

  1. In the Web UI, go to Namespaces, select the Namespace where you want to configure the Codec Server endpoint, and click Edit.
  2. In the Codec Server section on the Namespace configuration page, enter your Codec Server endpoint and port number.
  3. Optional: If your Codec Server is configured to authenticate requests from Temporal Web UI, enable Pass access token to send a JWT access token with the HTTPS requests.
  4. Optional: If your Codec Server is configured to verify origins of requests, enable Include cross-origin credentials.

Setting a Codec Server endpoint on a Namespace enables it for all users on the Namespace. On Temporal Cloud, you must have Namespace Admin privileges to add a Codec Server endpoint on the Namespace.

All users on a Namespace have the option to override the Namespace-level setting at the browser level. Overriding the Namespace-level endpoint only affects your browser. This can be useful when developing, testing, or troubleshooting encoding functionality.

Codec Server endpoint browser setting

Codec Server endpoint browser setting

To set a browser override for the Namespace-level endpoint, do the following.

  1. Navigate to Workflows in your Namespace.
  2. In the top-right corner, select Configure Codec Server.
  3. Select whether you want to use the Namespace-level (or Cluster-level for self-hosted Cluster) or the browser-level Codec Endpoint setting as the default for your browser. In Temporal Cloud:
    • Use Namespace-level settings, where available. Otherwise, use my browser setting. Uses the Namespace-level Codec Server endpoint by default. If no endpoint is set on the Namespace, your browser setting is applied.
    • Use my browser setting and ignore Namespace-level setting. Applies your browser-level setting by default, overriding the Namespace-level Codec Server endpoint.
  4. Enter your Codec Server endpoint and port number.
  5. Optional: If your Codec Server is configured to authenticate requests from Temporal Web UI, enable Pass access token to send a JWT access token with the HTTPS requests.
  6. Optional: If your Codec Server is configured to verify origins of requests, enable Include cross-origin credentials.

In self-hosted Temporal Clusters where you set up your UI Server, you can also set the codec endpoint in the UI server configuration file, as shown in the following example.

codec:
    endpoint: {{ default .Env.TEMPORAL_CODEC_ENDPOINT "{namespace}"}}

Start the UI server to use this endpoint on the Web UI for decoding data in Workflow Executions in the specified Namespace.

CLI​

In self-hosted Temporal Clusters, after the Codec Server is started, provide the exposed endpoint to CLI using the --codec_endpoint command option.

For example, if you are running your Codec Server locally and expose port 8888 as your endpoint, run the following command to set the codec endpoint globally.

temporal env set --codec-endpoint "http://localhost:8888"

If your codec endpoint is not set globally, use the --codec-endpoint option with your CLI commands. For example, to see the decoded output of the Workflow Execution "yourWorkflow" in the Namespace "yourNamespace", run the following command.

temporal --codec-endpoint "http://localhost:8888" --namespace "yourNamespace" workflow show --workflow-id "yourWorkflow"  --run-id "<yourRunId>" --output "table"

For details, see the CLI reference.

Currently in Temporal Cloud, you can set the Codec Server endpoint only from the Web UI.