Search Attributes
This page discusses the following:
What is a Search Attribute?
A Search Attribute is an indexed field used in a List Filter to filter a list of Workflow Executions that have the Search Attribute in their metadata.
Each Search Attribute is a key-value pair metadata object included in a Workflow Execution's Visibility information. This information is available in the Visibility store.
Search Attribute values are not encrypted because the Temporal Server must be able to read these values from the Visibility store when retrieving Workflow Execution details.
Temporal provides some default Search Attributes, such as ExecutionStatus, the current state of your Workflow Executions.
You can also create custom Search Attribute keys in your Visibility store and assign values when starting a Workflow Execution or in Workflow code.
When using Continue-As-New or a Temporal Cron Job, Search Attribute keys are carried over to the new Workflow Run by default. Search Attribute values are only available for as long as the Workflow is.
Search Attributes are most effective for search purposes or tasks requiring collection-based result sets. For business logic in which you need to get information about a Workflow Execution, consider one of the following:
- Storing state in a local variable and exposing it with a Query.
- Storing state in an external datastore through Activities and fetching it directly from the store.
If your business logic requires high throughput or low latency, store and fetch the data through Activities. You might experience lag due to time passing between the Workflow's state change and the Activity updating the Visibility store.
Default Search Attributes
A Temporal Service has a set of default Search Attributes already available. Default Search Attributes are set globally in any Namespace. These Search Attributes are created when the initial index is created.
| NAME | TYPE | DEFINITION |
|---|---|---|
| BatcherUser | Keyword | Used by internal batcher Workflow that runs in TemporalBatcher Namespace division to indicate the user who started the batch operation. |
| BinaryChecksums | Keyword List | List of binary Ids of Workers that run the Workflow Execution. Deprecated since server version 1.21 in favor of the BuildIds search attribute. |
| BuildIds | Keyword List | List of Worker Build Ids that have processed the Workflow Execution, formatted as versioned:{BuildId} or unversioned:{BuildId}, or the sentinel unversioned value. Available from server version 1.21. |
| CloseTime | Datetime | The time at which the Workflow Execution completed. |
| ExecutionDuration | Int | The time needed to run the Workflow Execution (in nanoseconds). Available only for closed Workflows. |
| ExecutionStatus | Keyword | The current state of the Workflow Execution. |
| ExecutionTime | Datetime | The time at which the Workflow Execution actually begins running; same as StartTime for most cases but different for Cron Workflows and retried Workflows. |
| HistoryLength | Int | The number of events in the history of Workflow Execution. Available only for closed Workflows. |
| HistorySizeBytes | Long | The size of the Event History. |
| RunId | Keyword | Identifies the current Workflow Execution Run. |
| StartTime | Datetime | The time at which the Workflow Execution started. |
| StateTransitionCount | Int | The number of times that Workflow Execution has persisted its state. Available only for closed Workflows. |
| TaskQueue | Keyword | Task Queue used by Workflow Execution. |
| TemporalChangeVersion | Keyword List | Stores change/version pairs if the GetVersion API is enabled. |
| TemporalScheduledStartTime | Datetime | The time that the Workflow is schedule to start according to the Schedule Spec. Can be manually triggered. Set on Schedules. |
| TemporalScheduledById | Keyword | The Id of the Schedule that started the Workflow. |
| TemporalSchedulePaused | Boolean | Indicates whether the Schedule has been paused. Set on Schedules. |
| WorkflowId | Keyword | Identifies the Workflow Execution. |
| WorkflowType | Keyword | The type of Workflow. |
-
All default Search Attributes are reserved and read-only. You cannot create a custom one with the same name or alter the existing one.
-
Search attributes are not encrypted in the system. Do not use PII as either the search attribute name or the value.
-
ExecutionStatus values correspond to Workflow Execution statuses: Running, Completed, Failed, Canceled, Terminated, ContinuedAsNew, TimedOut.
-
StartTime, CloseTime, and ExecutionTime are stored as dates but are supported by queries that use either EpochTime in nanoseconds or a string in RFC3339Nano format (such as "2006-01-02T15:04:05.999999999Z07:00").
-
ExecutionDuration is stored in nanoseconds but is supported by queries that use integers in nanoseconds, Golang duration format, or "hh:mm:ss" format.
-
CloseTime, HistoryLength, StateTransitionCount, and ExecutionDuration are present only in a closed Workflow Execution.
-
ExecutionTime can differ from StartTime in retry and cron use cases.
You can use the default Search Attributes in a List Filter, such as in the Temporal Web UI or with the temporal workflow list commands, under the following conditions:
- Without advanced Visibility, you can only use the
=operator with a single default Search Attribute in your List Filter. For example:temporal workflow list --query "ExecutionStatus = 'Completed'"ortemporal workflow list --query "WorkflowType = 'YourWorkflow'". - With advanced Visibility, you can combine default Search Attributes in a List Filter to get a list of specific Workflow Executions.
For example:
temporal workflow list --query "WorkflowType = 'main.YourWorkflowDefinition' and ExecutionStatus != 'Running' and (StartTime > '2022-06-07T16:46:34.236-08:00' or CloseTime < '2022-06-08T16:46:34-08:00')"
Custom Search Attributes
You can create custom Search Attributes with unique key names that are relevant to your business needs.
Use custom Search Attributes in a List Filter, such as in the Temporal Web UI or with the temporal workflow list commands, under the following conditions:
- Without advanced Visibility, you cannot use a custom Search Attribute in your List Filter.
- With advanced Visibility, you can create multiple custom Search Attributes and use them in combinations with List Filters to get a list of specific Workflow Executions.
For example:
temporal workflow list --query "WorkflowType = 'main.YourWorkflowDefinition' and YourCustomSA = 'YourCustomSAValue' and (StartTime > '2022-06-07T16:46:34.236-08:00' or CloseTime < '2022-06-08T16:46:34-08:00')"- With Temporal Server v1.19 and earlier, you must integrate Elasticsearch to use custom Search Attributes with List Filters.
- With Temporal Server v1.20 and later, custom Search Attribute capabilities are available on MySQL (v8.0.17 or later), PostgreSQL (v12 and later), and SQLite (v3.31.0 and later), in addition to Elasticsearch.
If you use Elasticsearch as your Visibility store, your custom Search Attributes apply globally and can be used across Namespaces. However, if using any of the supported SQL databases with Temporal Server v1.20 and later, your custom Search Attributes are associated with a specific Namespace and can be used for Workflow Executions in that Namespace.
See custom Search Attributes limits for limits on the number and size of custom Search Attributes you can create.
Supported types
Custom Search Attributes must be one of the following types:
- Bool
- Datetime
- Double
- Int
- Keyword
- KeywordList
- Text
Note:
- Double is backed up by
scaled_floatElasticsearch type with scale factor 10000 (4 decimal digits). - Datetime is backed up by
datetype with milliseconds precision in Elasticsearch 6 anddate_nanostype with nanoseconds precision in Elasticsearch 7. - Int is 64-bit integer (
longElasticsearch type). - Keyword and Text types are concepts taken from Elasticsearch. Each word in a Text is considered a searchable keyword.
For a UUID, that can be problematic because Elasticsearch indexes each portion of the UUID separately.
To have the whole string considered as a searchable keyword, use the Keyword type.
For example, if the key
ProductIdhas the value of2dd29ab7-2dd8-4668-83e0-89cae261cfb1:- As a Keyword it would be matched only by
ProductId = "2dd29ab7-2dd8-4668-83e0-89cae261cfb1. - As a Text it would be matched by
ProductId = 2dd8, which could cause unwanted matches.
- As a Keyword it would be matched only by
- With Temporal Server v1.19 and earlier, the Keyword type can store a list of values.
- With Temporal Server v1.20 and later, the Keyword type supports only a single value. To store a list of values, use KeywordList.
- The Text type cannot be used in the "Order By" clause.
Custom Search Attributes limits
The following table lists the maximum number of custom Search Attributes you can create per Namespace by supported Visibility database.
| Search Attribute type | MySQL (v8.0.17 and later) | PostgreSQL (v12 and later) | SQLite (v3.31.0 and later) | Temporal Cloud |
|---|---|---|---|---|
| Bool | 3 | 3 | 3 | 20 |
| Datetime | 3 | 3 | 3 | 20 |
| Double | 3 | 3 | 3 | 20 |
| Int | 3 | 3 | 3 | 20 |
| Keyword | 10 | 10 | 10 | 40 |
| KeywordList | 3 | 3 | 3 | 5 |
| Text | 3 | 3 | 3 | 5 |
Temporal does not impose a limit on the number of custom Search Attributes you can create with Elasticsearch. However, Elasticsearch sets a default mapping limit that may apply. Custom Search Attributes are an advanced Visibility feature and are not supported on Cassandra.
Size limits for a custom Search Attribute:
- The default single Search Attribute value size limit is 2 KB.
- The maximum total Search Attribute size is 40 KB.
- The maximum total characters per Search Attribute value is 255.
For Temporal Cloud specific configurations, see the Defaults, limits, and configurable settings -Temporal Cloud guide.
Usage
Search Attributes available in your Visibility store can be used with Workflow Executions for the Temporal Service. To actually have results from the use of a List Filter, Search Attributes must be added to a Workflow Execution as metadata.
- To create custom Search Attributes in your Visibility store, see Create custom Search Attributes.
- To remove a custom Search Attribute from the Visbility store, see Remove custom Search Attributes. Removing custom Search Attributes is not supported on Temporal Cloud.
- To rename a custom Search Attribute on Temporal Cloud, see
tcld namespace search-attributes rename.
With Workflows you can do the following:
- Set the value of Search Attributes in your Workflow
- Update the value set for a Search Attribute from within the Workflow code
- Remove the value set for a Search Attribute from within the Workflow code
- How to manage Search Attributes using the Go SDK
- How to manage Search Attributes using the Java SDK
- How to manage Search Attributes using the PHP SDK
- How to manage Search Attributes using the Python SDK
- How to manage Search Attributes using the TypeScript SDK
- How to manage Search Attributes using the .NET SDK
- To get a list of Search Attributes using the Temporal CLI, issue
temporal operator search-attribute list. See Search Attributes.
After you add and set your Search Attributes, use your default or custom Search Attributes in a List Filter.