Many use-cases may have hundreds, thousands, or tens of thousands of Workflows. This means, finding the one you want can be a non-trivial exercise. To address this, Temporal Server has built-in functionality that enables you to search for Workflows via the CLI(tctl), Web UI, and SDK APIs.
To make this search functionality even better, the Temporal Server supports an integration with Elasticsearch. By integrating with Elasticsearch, custom key-value search attributes can be added to Workflow metadata which can be referenced in SQL-like search queries.
Without Elasticsearch, search functionality is limited to listing Workflows by Id and type (open/closed), and search queries are run directly against the Temporal Server database. Elasticsearch takes on the search query request load, relieving potential performance issues that can occur. This is why we highly recommend running Temporal with Elasticsearch for any use-case that spawns more than just a few Workflows. Follow the Elasticsearch setup instructions to do that. The rest of this page assumes your instance of Temporal is integrated with Elasticsearch.
Search attributes are indexed and queryable pieces of Workflow metadata. They are represented as key-value pairs. There are many attributes provided by default, but you can also add your own. Keys must be registered within the dynamic config file so that the Temporal Server knows the attribute key name and value type. If there are multiple instances of the Server running across multiple hosts, then keys must be registered across all of the dynamic config files. To add new search attributes, you can use the CLI or edit the dynamic config file manually.
Here are the attributes that are provided by default:
There are some special considerations for the Default attributes:
- ExecutionStatus, CloseTime, NamespaceId, ExecutionTime, HistoryLength, RunId, StartTime, WorkflowId, WorkflowType are reserved by Temporal and are read-only.
- ExecutionStatus is a mapping of an integer to a state:
- StartTime, CloseTime and ExecutionTime are stored as integers, but are supported by queries using both EpochTime in nanoseconds and a string in RFC3339 format (ex. "2006-01-02T15:04:05+07:00").
- CloseTime, HistoryLength are only present in a closed Workflow.
- ExecutionTime is meant for queries against Workflows that will run in the future (retry/cron use-cases).
There is no hard limit on the number of attributes you can add. However we recommend enforcing the following limits:
- Number of keys: 100 per Workflow
- Size of value: 2kb per value
- Total size of key and values: 40kb per Workflow
Search attributes must be one of the following types:
Note that keyword and string types are concepts taken from Elasticsearch. Each word in a string is considered a searchable keyword. For a UUID, that can be problematic as Elasticsearch will index each portion of the UUID separately. To have the whole string considered as a searchable keyword, use the keyword type.
For example, if the key
SomeUniqueId has the value of "2dd29ab7-2dd8-4668-83e0-89cae261cfb1"
- As a keyword it would only be matched by
- As a string it would be matched by
SomeUniqueId= "2dd8", which could cause unwanted matches.
Also note that the string type cannot be used in the "Order By" clause.
There are some pre-allowlisted search attributes that are handy for testing, that have their types indicated in their name:
In the Temporal Web UI, use the "Basic/Advanced" button to switch to "Advanced" mode and type the query into the search box.
The following criteria applies to search queries:
- These operators are supported in search queries:
- AND, OR, ()
- =, !=, >, >=, <, <=
- BETWEEN ... AND
- ORDER BY
- You will only be able to search for Workflows from one Namespace at a time.
- The default pagesize is 1000, and cannot be larger than 10,000.
- The range on a Temporal timestamp search query (StartTime, CloseTime, ExecutionTime) cannot be larger than 9223372036854775807 (maxInt64 - 1001)
- Search queries using a time range will have a 1ms resolution.
- Search query column names are case sensitive.
ListWorkflowsearch query may take longer when retrieving a large number of Workflows (10M+).
- To retrieve a large number of Workflows without ordering, use the
- To efficiently count the number of Workflows, use the
- To list only open Workflows, add
CloseTime = missingto the end of the query.
- If you use the retry/cron feature to search for Workflows that will start execution within a certain time range, you can add predicates on ExecutionTime.
ExecutionTime > 2019-01-01T10:00:00-07:00. Note that, if a predicate on
ExecutionTimeis included, only the cron/retry Workflow will be returned.
- Increase the Docker memory to anything higher than 6GB (Navigate to Docker -> Preferences -> Advanced -> Memory).
- Use the Quick start guide to clone the temporalio/docker-compose repo.
- Start the Temporal Server using the
docker-compose-es.ymlfile, which contains Cassandra, Kafka, Zookeeper, and Elasticsearch.
From the Docker output log, make sure Elasticsearch and Temporal started correctly.
Allow list search attributes.
Note that starting a Workflow with search attributes, but without Elasticsearch, will succeed as normal. However the Workflow will not be searchable by those attributes.