Much of the behavior of a Temporal Cluster is configured using the
development.yaml file and may contain the following top-level sections:
Changing any properties in the
development.yaml file requires a process restart for changes to take effect.
Configuration parsing code is available here.
global section contains process-wide configuration. See below for a minimal configuration (optional parameters are commented out.)
membership section controls the following membership layer parameters.
The amount of time the service will attempt to join the gossip layer before failing.
Default is 10s.
Used by gossip protocol to communicate with other hosts in the same Cluster for membership info.
Use IP address that is reachable by other hosts in the same Cluster.
If there is only one host in the Cluster, you can use 127.0.0.1.
net.ParseIP for supported syntax, only IPv4 is supported.
Configures the Cluster's metric subsystem. Specific provides are configured using provider names as the keys.
The prefix to be applied to all outgoing metrics.
The set of key-value pairs to be reported as part of every metric.
A map from tag name string to tag values string list.
This is useful to exclude some tags that might have unbounded cardinality.
The value string list can be used to whitelist values of that excluded tag to continue to be included.
For example, if you want to exclude
task_queue because it has unbounded cardinality, but you still want to see a whitelisted value for
statsd sections supports the following settings:
hostPort: The host:port of the statsd server.
prefix: Specific prefix in reporting to
flushInterval: Maximum interval for sending packets. (Default 300ms).
flushBytes: Specifies the maximum UDP packet size you wish to send. (Default 1432 bytes).
prometheus sections supports the following settings:
framework: The framework to use, currently supports
tally, default is
tally. We plan to switch default to
opentelemetryonce its API become stable.
listenAddress: Address for Prometheus to scrape metrics from. The Temporal Server uses the Prometheus client API, and the
listenAddressconfiguration is used to listen for metrics.
handlerPath: Metrics handler path for scraper; default is
m3 sections supports the following settings:
hostPort: The host:port of the M3 server.
service: The service tag to that this client emits.
queue: M3 reporter queue size, default is 4k.
packetSize: M3 reporter max packet size, default is 32k.
port: If specified, this will initialize pprof upon process start on the listed port.
tls section controls the SSL/TLS settings for network communication and contains two subsections,
internode section governs internal service communication among roles where the
frontend governs SDK client communication to the Frontend Service role.
Each of these subsections contain a
server section and a
server contains the following parameters:
certFile: The path to the file containing the PEM-encoded public key of the certificate to use.
keyFile: The path to the file containing the PEM-encoded private key of the certificate to use.
requireClientAuth: boolean - Requires clients to authenticate with a certificate when connecting, otherwise known as mutual TLS.
clientCaFiles: A list of paths to files containing the PEM-encoded public key of the Certificate Authorities you wish to trust for client authentication. This value is ignored if
requireClientAuthis not enabled.
See the server samples repo for sample TLS configurations.
Below is an example enabling Server TLS (https) between SDKs and the Frontend APIs:
client section generally needs to be provided to specify an expected DNS SubjectName contained in the presented server certificate via the
serverName field; this is needed as Temporal uses IP to IP communication.
You can avoid specifying this if your server certificates contain the appropriate IP Subject Alternative Names.
rootCaFiles field needs to be provided when the client's host does not trust the Root CA used by the server.
The example below extends the above example to manually specify the Root CA used by the Frontend Services:
Below is an additional example of a fully secured cluster using mutual TLS for both frontend and internode communication with manually specified CAs:
Note: In the case that client authentication is enabled, the
internode.server certificate is used as the client certificate among services. This adds the following requirements:
internode.servercertificate must be specified on all roles, even for a frontend-only configuration.
Internode server certificates must be minted with either no Extended Key Usages or both ServerAuth and ClientAuth EKUs.
If your Certificate Authorities are untrusted, such as in the previous example, the internode server Ca will need to be specified in the following places:
persistence section holds configuration for the data store / persistence layer.
Below is an example minimal specification for a password-secured Cluster using Cassandra.
The following top level configuration items are required:
Required - The number of history shards to create when initializing the Cluster.
Warning: This value is immutable and will be ignored after the first run. Please ensure you set this value appropriately high enough to scale with the worst case peak load for this Cluster.
Required - The name of the data store definition that should be used by the Temporal server.
Required - the name of the data store definition that should be used by the Temporal visibility server.
Required - contains named data store definitions to be referenced.
Each definition is defined with a heading declaring a name (ie:
visibility: above), which contains a data store definition.
Data store definitions must be either
cassandra data store definition can contain the following values:
hosts: Required - "," separated Cassandra endpoints, e.g. "192.168.1.2,192.168.1.3,192.168.1.4".
port: Default: 9042 - Cassandra port used for connection by
user: Cassandra username used for authentication by
password: Cassandra password used for authentication by
keyspace: Required - the Cassandra keyspace.
datacenter: The data center filter arg for Cassandra.
maxConns: The max number of connections to this data store for a single TLS configuration.
tls: See TLS below.
sql data store definition can contain the following values:
user: Username used for authentication.
password: Password used for authentication.
pluginName: Required - SQL database type.
- Valid values:
- Valid values:
databaseName- required - the name of SQL database to connect to.
connectAddr- required - the remote address of the database, e.g. "192.168.1.2".
connectProtocol- required - the protocol that goes with the
- Valid values:
- Valid values:
connectAttributes- a map of key-value attributes to be sent as part of connect
maxConns- the max number of connections to this data store.
maxIdleConns- the max number of idle connections to this data store
maxConnLifetime- is the maximum time a connection can be alive.
tls- See below.
mtls sections can contain the following values:
serverName- name of the server hosting the data store.
certFile- path to the cert file.
keyFile- path to the key file.
caFile- path to the ca file.
enableHostVerification- boolean -
trueto verify the hostname and server cert (like a wildcard for Cassandra cluster). This option is basically the inverse of
InSecureSkipVerifyin http://golang.org/pkg/crypto/tls/ for more info.
keyFile are optional depending on server config, but both fields must be omitted to avoid using a client certificate.
log section is optional and contains the following possible values:
stdout- boolean -
trueif the output needs to go to standard out.
level- sets the logging level.
- Valid values - debug, info, warn, error or fatal, default to info.
outputFile- path to output log file.
What is Multi-Cluster Replication? Multi-Cluster Replication is a feature which asynchronously replicates Workflow Executions from active Clusters to other passive Clusters, for backup and state reconstruction.
clusterMetadata contains the local cluster information. The information is used in Multi-Cluster Replication
What is Multi-Cluster Replication?
Multi-Cluster Replication is a feature which asynchronously replicates Workflow Executions from active Clusters to other passive Clusters, for backup and state reconstruction.
currentClusterName- required - the name of the current cluster. Warning: This value is immutable and will be ignored after the first run.
replicationConsumer- determines which method to use to consume replication tasks. The type may be either
failoverVersionIncrement- the increment of each cluster version when failover happens.
masterClusterName- the master cluster name, only the master cluster can register/update namespace. All clusters can do namespace failover.
clusterInformation- contains the local cluster name to
ClusterInformationdefinition. The local cluster name should be consistent with
ClusterInformationsections consist of:
enabled- boolean - whether a remote cluster is enabled for replication.
rpcAddress- indicate the remote service address (host:port). Host can be DNS name. Use
dns:///prefix to enable round-robin between IP address for DNS name.
services section contains configuration keyed by service role type.
There are four supported service roles:
Below is a minimal example of a
frontend service definition under
There are two sections defined under each service heading:
rpc contains settings related to the way a service interacts with other services. The following values are supported:
grpcPortis the port on which gRPC will listen.
membershipPort: Port used to communicate with other hosts in the same Cluster for membership info. Each service should use different port. If there are multiple Temporal Clusters in your environment (Kubernetes for example), and they have network access to each other, each cCluster should use different membershipPort.
bindOnLocalHost: Determines whether uses
127.0.0.1as the listener address.
bindOnIP: Used to bind service on specific IP, or
net.ParseIPfor supported syntax, only IPv4 is supported, mutually exclusive with
Note: Port values are currently expected to be consistent among role types across all hosts.
publicClient a required section describing the configuration needed to for worker to connect to Temporal server for background server maintenance.
hostPortIPv4 host port or DNS name to reach Temporal frontend, reference
dns:/// prefix to enable round-robin between IP address for DNS name.