The Temporal microservice orchestrator is a key component of the Temporal platform that provides coordination and management of microservices as they interact with one another to complete complex workflows. The orchestrator provides features such as task management, error handling, retries, and compensation to ensure that workflows are executed accurately and consistently, even in the face of failures or unexpected events.
Using Temporal in application development can help simplify the development of complex, time-sensitive workflows and improve the reliability and scalability of the applications. By using the Temporal orchestrator, developers can focus on the implementation of their business logic, and leave the coordination and management of the underlying microservices to the platform.
Illustration
Microservices architecture, for example, is used in company XYZ. The microservices approach to software development helps their teams deploy faster, but it comes with some issues, one of which is data consistency. How can data changes in microservice A be propagated to microservices B and C? send it through an event?
Yes, that works, but what if B updates itself and C had hiccups and just could not make the update?
Then that means we need to have a mechanism that allows us to handle such failures, make retries, and what else? How many situations like the one described above require us to write failure and retry logic?
Hence, the use of Temporal as a microservice orchestrator helps us solve the issues stated above.
Prerequisites
The prerequisite for this tutorial is having Go and Docker installed and configured.
Verify that you've installed Go by opening a command prompt and typing the following command
go version
It should return something similar to this
go version go1.19.3 darwin/amd64
Also, verify docker is installed with
docker --version
#Docker version 20.10.22, build 3a2c30b
Choice
Temporal can be used inside and outside your project directory, as long as your application can access it. Your individual needs and the project's infrastructure will determine your choice.
It may be simpler to maintain your project's dependencies and setup if you decide to execute Temporal inside of your project directory because everything is contained there. However, running Temporal outside of your project directory might improve the degree of concern separation and make it simpler to manage numerous projects that share a single Temporal instance.
However, in this guide, we will run the temporal server inside our project directory.
Hello-workflow
We will be using the simple hello-workflow to orchestrate tasks within a distributed system. The starter and workers are the two main components that enable the distributed system to run, and in this guide, we will be implementing the starter and worker.
Here's what our project directory will look like.
├── dynamicconfig
│ ├── development-cass.yaml
│ └── development-sql.yaml
| └── docker.yaml
|
├── src
│ ├── helloworkflow
| | └── workflow.go
│ ├── starter
| | └── main.go
| └── worker
| └── main.go
├── .env
├── docker-compose.yml
You create a new Go project, say tutorial-guide and cd into it, then open the directory on your text editor. You create dynamicconfig folder in the root directory of your project and add the 3 yaml files. These files are needed to have the temporal-server up.
# development-cass.yaml
system.forceSearchAttributesCacheRefreshOnRead:
- value: true # Dev setup only. Please don't turn this on in production.
constraints: {}
# development-sql.yaml
limit.maxIDLength:
- value: 255
constraints: {}
system.forceSearchAttributesCacheRefreshOnRead:
- value: true # Dev setup only. Please don't turn this on in production.
constraints: {}
# docker.yaml
apiVersion: v1
clusters:
- cluster:
certificate-authority: path-to-your-ca.rt #e.g/System/Volumes/Data/Users/<name>/.minikube/ca.crt
server: https://192.168.99.100:8443
name: minikube
contexts:
- context:
cluster: minikube
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate: path-to-proxy-client-ca.key #e.g/System/Volumes/Data/Users/<name>/.minikube/proxy-client-ca.key
client-key: path-to-proxy-client-ca.key #e.g/System/Volumes/Data/Users/<name>/.minikube/proxy-client-ca.key
Use docker.yaml file to override the default dynamic config value (they are specified when creating the service config). More information about the docker.yaml file and how to use it [https://github.com/temporalio/docker-compose/tree/main/dynamicconfig]
# docker-compose.yaml
version: '3.5'
services:
elasticsearch:
container_name: temporal-elasticsearch
environment:
- cluster.routing.allocation.disk.threshold_enabled=true
- cluster.routing.allocation.disk.watermark.low=512mb
- cluster.routing.allocation.disk.watermark.high=256mb
- cluster.routing.allocation.disk.watermark.flood_stage=128mb
- discovery.type=single-node
- ES_JAVA_OPTS=-Xms256m -Xmx256m
- xpack.security.enabled=false
image: elasticsearch:${ELASTICSEARCH_VERSION}
networks:
- temporal-network
expose:
- 9200
volumes:
- /var/lib/elasticsearch/data
postgresql:
container_name: temporal-postgresql
environment:
POSTGRES_PASSWORD: temporal
POSTGRES_USER: temporal
image: postgres:${POSTGRESQL_VERSION}
networks:
- temporal-network
expose:
- 5432
volumes:
- /var/lib/postgresql/data
temporal:
container_name: temporal
depends_on:
- postgresql
- elasticsearch
environment:
- DB=postgresql
- DB_PORT=5432
- POSTGRES_USER=temporal
- POSTGRES_PWD=temporal
- POSTGRES_SEEDS=postgresql
- DYNAMIC_CONFIG_FILE_PATH=config/dynamicconfig/development-sql.yaml
- ENABLE_ES=true
- ES_SEEDS=elasticsearch
- ES_VERSION=v7
image: temporalio/auto-setup:${TEMPORAL_VERSION}
networks:
- temporal-network
ports:
- 7233:7233
labels:
kompose.volume.type: configMap
volumes:
- ./dynamicconfig:/etc/temporal/config/dynamicconfig
temporal-admin-tools:
container_name: temporal-admin-tools
depends_on:
- temporal
environment:
- TEMPORAL_CLI_ADDRESS=temporal:7233
image: temporalio/admin-tools:${TEMPORAL_VERSION}
networks:
- temporal-network
stdin_open: true
tty: true
temporal-ui:
container_name: temporal-ui
depends_on:
- temporal
environment:
- TEMPORAL_ADDRESS=temporal:7233
- TEMPORAL_CORS_ORIGINS=http://localhost:3000
image: temporalio/ui:${TEMPORAL_UI_VERSION}
networks:
- temporal-network
ports:
- 8080:8080
networks:
temporal-network:
driver: bridge
name: temporal-network
Add the environment variables needed for the docker-compose.yaml file.
# .env
COMPOSE_PROJECT_NAME=temporal
CASSANDRA_VERSION=3.11.9
ELASTICSEARCH_VERSION=7.16.2
MYSQL_VERSION=8
POSTGRESQL_VERSION=13
TEMPORAL_VERSION=1.19.1
TEMPORAL_UI_VERSION=2.9.1
The source code for the workflow that the Temporal Server runs is located in the workflow.go file. It is in charge of generating activities, monitoring their completion, and controlling the workflow. It specifies every action the workflow must do and is started by an event, such as a message from a queue or the addition of a new data item to the system.
// workflow.go
package helloworkflow
import (
"context"
"time"
"go.temporal.io/sdk/workflow"
)
func Workflow(ctx workflow.Context, name string) (string, error) {
ao := workflow.ActivityOptions{
ScheduleToStartTimeout: time.Minute,
StartToCloseTimeout: time.Minute,
}
ctx = workflow.WithActivityOptions(ctx, ao)
logger := workflow.GetLogger(ctx)
var result string
err := workflow.ExecuteActivity(ctx, Activity, name).Get(ctx, &result)
if err != nil {
logger.Error("Activity failed", "Error", err)
}
return result, nil
}
func Activity(ctx context.Context, name string) (string, error) {
return "Hello " + name, nil
}
The starter is responsible for coordinating the workflow executions within the system. It is responsible for scheduling the tasks to be executed and keeping the task statuses up to date. The starter also acts as a gateway for users to interact with the system via its API.
Now, we add these to our main.go file under the starter directory.
// starter/main.go
package main
import (
"context"
"log"
"github.com/theifedayo/hello-workflow/src/helloworkflow"
"go.temporal.io/sdk/client"
)
func main() {
c, err := client.NewClient(client.Options{
})
if err != nil {
log.Fatalln("Unable to make client", err)
}
defer c.Close()
workflowOptions := client.StartWorkflowOptions{
ID: "hello_world_workflowID",
TaskQueue: "hello-world",
}
we, err := c.ExecuteWorkflow(context.Background(), workflowOptions, helloworkflow.Workflow, "ifedayo")
if err != nil {
log.Fatalln("Unable to execute workflow", err)
}
var result string
// store the result of the run
err = we.Get(context.Background(), &result)
if err != nil {
log.Fatalln("Unable to get workflow result", err)
}
log.Println("workflow result:", result)
}
Workers are responsible for executing the tasks. These workers are deployed across different nodes in the system, and their job is to receive the tasks from the starter and execute them. The workers are also responsible for sending the results back to the starter. The Workers can be scaled up or down depending on the workload.
// worker/main.go
package main
import (
"log"
"github.com/theifedayo/hello-workflow/src/helloworkflow"
"go.temporal.io/sdk/client"
"go.temporal.io/sdk/worker"
)
func main() {
c, err := client.NewClient(client.Options{})
if err != nil {
log.Fatalln("Unable to make client", err)
}
defer c.Close()
w := worker.New(c, "hello-world", worker.Options{})
w.RegisterWorkflow(helloworkflow.Workflow)
w.RegisterActivity(helloworkflow.Activity)
err = w.Run(worker.InterruptCh())
if err != nil {
log.Fatalln("Unable to start workflow", err)
}
}
Starting Everything Up
To start our workflow, we need our temporal server up. Go to your terminal and cd to your project directory and run
docker-compose up
This will start up all the services in our docker-compose.yaml.
Next up, we start our worker.
In general, you should start the workers before the starter, as the starter typically depends on the workers being available to perform their tasks. This means that the workers should be started and initialized before the starter begins to coordinate their activities.
go run src/worker/main.go
2023/02/16 14:03:42 INFO No logger configured for temporal client. Created default one.
2023/02/16 14:03:43 INFO Started Worker Namespace default TaskQueue hello-world WorkerID 4465@Ifedayos-MacBook-Pro.local@
And finally, we start the workflow
go run src/starter/main.go
2023/02/16 14:05:52 INFO No logger configured for temporal client. Created default one.
2023/02/16 14:05:52 workflow result: Hello ifedayo
Viewing Workflows
Navigating to the browser on localhost:8080, we have a UI that gives more information about the workflow
Temporal also provides a CLI tool for interacting with the Temporal server, tctl, which also performs various administrative tasks, such as starting and stopping workflows, querying workflow information, and managing workflow executions.
tctl workflow list
WORKFLOW TYPE | WORKFLOW ID | RUN ID | TASK QUEUE | START TIME | EXECUTION TIME | END TIME
Workflow | hello_world_workflowID | 0454098f-cdd9-4f64-af8b-ab77a6f86c35 | hello-world | 13:05:52 | 13:05:52 | 13:05:52
Conclusion
In this writing, we've learned how what temporal is, why use it, an illustration of using it in a company, choices of setting up a temporal server, how to start the server inside your go project directory, running workflow and worker, and finally viewing your workflows in a UI or CLI.
Cheers 🥂! You're one step closer to building more reliable applications with Temporal.
Top comments (0)