Chapter 4: Visibility and Search #

In Chapter 3: Task Queues and Matching, we saw how workflows get their work done through workers. But think about this: what if you have thousands of workflows running at the same time? How do you find the one that belongs to customer Alice? Or how do you find all workflows that are stuck on a payment problem?

You don’t want to open every workflow and read its history. That would be like reading every book in a library to find one recipe. Instead, you go to the library catalogue. You search for “baking” and the catalogue points you to the exact shelf.

In Temporal, that catalogue is called Visibility. Workflows can be found using a searchable index and a custom query language, just like searching a library database.

A Concrete Use Case: Find Alice’s Running Orders #

Imagine you run an online shop with a workflow for every order. You want to answer a simple question:

Show me every OrderWorkflow that is currently running for customer Alice.

You could start the workflow with an extra piece of metadata called a search attribute, then later query for it.

Start an order workflow and attach the owner name:

temporal workflow start \
  --task-queue orders \
  --workflow-id order-123 \
  --search-attribute "OrderOwner=Alice"

What happens? The CLI sends a start request to Temporal’s Public API. The workflow starts normally, but it also gets an indexed attribute: OrderOwner is set to "Alice".

Later, list all running workflows for Alice:

temporal workflow list \
  --query "ExecutionStatus = 'Running' AND OrderOwner = 'Alice'"

What happens? Temporal searches its index and returns a list of workflow IDs. It doesn’t scan every workflow from scratch. It uses an index, like a library catalogue.

Key Concepts #

Let’s break down the main pieces in a beginner-friendly way.

Custom Search Attributes #

A custom search attribute is a named, typed field attached to a workflow execution. It’s like adding a sticky note to a book before you put it on the shelf.

Temporal gives you some built-in attributes, such as:

• ExecutionStatus (for example, Running, Completed, Failed)
• WorkflowType
• StartTime

You can also define your own, like OrderOwner, OrderAmount, or CustomerTier. Each custom attribute must be registered with a type: Text, Keyword, Int, Double, Bool, Datetime, and so on.

For example, you might define:

Once these exist, you can attach them to workflows and search for them.

Visibility #

Visibility is Temporal’s ability to answer the question “What workflows are running?” It’s the part of the server that keeps an index of workflow execution data so you can search it.

There are two kinds:

• Standard Visibility: Uses Temporal’s own database. It supports a basic set of queries.
• Advanced Visibility: Uses a dedicated search backend like Elasticsearch. This enables the full query language and is what most examples in this chapter assume.

The Query Language #

Temporal has a SQL-like query language. It looks like the WHERE clause from SQL. Some examples:

ExecutionStatus = 'Running'

OrderOwner = 'Alice' AND OrderAmount > 100

WorkflowType = 'OrderWorkflow' OR WorkflowType = 'ReturnWorkflow'

You can use AND, OR, =, !=, <, >, BETWEEN, IN, and more.

The Search Backend and Query Parsers #

Behind the scenes, Temporal needs to understand your query and then talk to the search backend.

Two important libraries appear in the server’s go.mod:

require (
    github.com/olivere/elastic/v7 v7.0.32
    github.com/temporalio/sqlparser v0.1.0
)

• temporalio/sqlparser is used to parse SQL-like queries like the one above.
• olivere/elastic is a Go client for Elasticsearch. Temporal uses it to send queries to the search backend.

The server translates your SQL-like query into an Elasticsearch query automatically. You don’t need to know Elasticsearch’s query language yourself.

Using Visibility to Solve the Use Case #

Let’s see how to use search in practice, both from the CLI and from the Go SDK.

From the CLI #

You already saw the CLI command:

temporal workflow list \
  --query "ExecutionStatus = 'Running' AND OrderOwner = 'Alice'"

Example output (simplified):

  WorkflowId  Type            StartTime
  order-123   OrderWorkflow   2026-04-10 14:22:01
  order-456   OrderWorkflow   2026-04-10 15:04:11

Only running workflows with OrderOwner = 'Alice' are shown. Notice that you didn’t look at the workflow histories. You used the index.

From the Go SDK #

In Go, you can list workflows with the ListWorkflow method:

res, _ := c.ListWorkflow(context.Background(), client.ListWorkflowOptions{
    Query: "ExecutionStatus = 'Running' AND OrderOwner = 'Alice'",
})

for res.HasNext() {
    e, _ := res.Next()
    fmt.Println(e.Execution.WorkflowId)
}

What happens? The SDK sends a ListWorkflowExecutions request with your query to Temporal. The server searches the visibility index, finds matching workflows, and returns them one by one. The loop prints each workflow ID.

What Happens Under the Hood? #

Now let’s peek inside Temporal’s server to understand how a query is processed.

Step-by-Step Walkthrough #

Let’s walk through it in plain words:

1. Your client sends a query like ExecutionStatus = 'Running' AND OrderOwner = 'Alice' to the frontend service.
2. The frontend calls the Visibility Store, which is the component responsible for searching workflows.
3. The Visibility Store uses temporalio/sqlparser to parse the query into a structure it can understand.
4. That structure is translated into a search request for Elasticsearch using olivere/elastic.
5. Elasticsearch searches its index and returns matching workflow IDs.
6. The results are formatted and sent back to your client.

All of this happens behind what feels like a simple command.

Diving Into the Code #

In the Temporal server repository, the visibility code lives under common/persistence/visibility. The code is complex, but the core idea can be shown in a simplified form.

First, the query is parsed:

stmt, err := sqlparser.Parse("ExecutionStatus = 'Running' AND OrderOwner = 'Alice'")
// stmt is now a parsed AST (Abstract Syntax Tree)

Then the AST is converted to an Elasticsearch query:

esQuery := convertToElastic(stmt)
hits, err := elasticClient.Search().
    Index("temporal-visibility").
    Query(esQuery).
    Do(ctx)

In the real code, convertToElastic is a larger function, but this is the essence: parse -> convert -> search.

Here is another way to picture it:

// common/persistence/visibility/store.go (simplified)
func (s *VisibilityStore) List(query string) ([]WorkflowRow, error) {
    stmt, err := sqlparser.Parse(query)          // 1. Parse query
    dsl := convertToElastic(stmt)                // 2. Translate to Elasticsearch DSL
    hits, err := s.es.Search().Query(dsl).Do(ctx) // 3. Search index
    return mapHitsToRows(hits), nil              // 4. Format results
}

This file is just an example, but it shows how the libraries work together.

• sqlparser is like the librarian who reads your search request.
• convertToElastic writes that request in the “language” the catalogue understands.
• elasticClient actually walks to the shelves and finds the cards.

Why This Feels Like “Magic” #

When you type a Temporal query, you don’t think about SQL parsing, ASTs, or Elasticsearch DSL. You just ask a question and get an answer. That’s the magic of an abstraction.

The same idea works for SDKs. You can query for workflows by custom attributes, by time ranges, by workflow type, or by status. You can even paginate through thousands of results.

For example, to find all workflows with an order amount above $500:

temporal workflow list \
  --query "OrderAmount > 500"

To find workflows that failed yesterday:

temporal workflow list \
  --query "ExecutionStatus = 'Failed' AND StartTime BETWEEN '2026-04-01' AND '2026-04-02'"

Each of these goes through the same pipeline: parse, convert, search, return.

Conclusion #

Visibility and Search make Temporal useful at scale. Workflows can be found by attributes, not by manually opening each one. It’s the library catalogue for your workflow executions.

You learned:

• Custom search attributes let you attach metadata to workflows.
• Temporal’s query language is SQL-like and beginner-friendly.
• The server translates queries using temporalio/sqlparser and searches Elasticsearch with olivere/elastic.
• You can search from the CLI or from SDKs with a simple query string.

Now that you know how to find workflows, what happens to workflows that are old and closed? Does the system keep them forever? That’s where archival and object storage come in.

Continue to Chapter 5: Archival and Object Storage.

Generated by AI Codebase Knowledge Builder