Ashok Nagaraj

Posted on May 8, 2023

otel-cli - push otel traces with ease

#opentelemetry #bash

The problem

otel-cli is a command-line interface tool that simplifies OpenTelemetry instrumentation and enables developers to quickly get started with distributed tracing. It supports a wide range of programming languages and frameworks, and provides a variety of exporters, including Jaeger, Zipkin, and Prometheus, among others.

One of the main advantages of using otel-cli is its simplicity. It provides a simple and intuitive interface that makes it easy for developers to get started with tracing, and provides real-time feedback on the traces generated by your application, making it easy to identify and diagnose issues in real-time.

Another advantage of otel-cli is its flexibility. It supports multiple exporters, giving developers the flexibility to send traces to a wide range of monitoring and observability tools. Additionally, it is an open-source tool, which means that it is available for free and can be used and modified by anyone.

Installation

On mac

brew tap equinix-labs/otel-cli
brew install otel-cli

On other systems
Get it from the releases link

Configuration

❯ docker run --name=otel-col -p 4318:4318 -p 4317:4317  -d otel/opentelemetry-collector
638e85e867aa4b69b356b5053991ce097d946a0aa9ecfa7568c867113209a307

otel-demo/python-http on  main [!⇡] via 🐍 v3.9.7 on ☁️
❯ docker ps
CONTAINER ID   IMAGE                          COMMAND                  CREATED         STATUS        PORTS                                                                 NAMES
638e85e867aa   otel/opentelemetry-collector   "/otelcol --config /…"   2 seconds ago   Up 1 second   0.0.0.0:4317-4318->4317-4318/tcp, 55678-55679/tcp                     otel-col

Workflow

Simple operation

# pushing in grpc mode
❯ OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 otel-cli exec --name=test-001 "sleep 1" --verbose --protocol grpc
❯ docker logs otel-col 2>&1 | grep -C5 test-001
InstrumentationScope otel-cli/exec
Span #0
    Trace ID       : 874ff9770b8f13f3276e1837b377e861
    Parent ID      :
    ID             : 36e4f9b9a33b4278
    Name           : test-001
    Kind           : Client
    Start time     : 2023-05-08 13:28:39.799345 +0000 UTC
    End time       : 2023-05-08 13:28:40.816506833 +0000 UTC
    Status code    : Ok
    Status message :

# pushing in http mode (default)
❯ OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 otel-cli exec --name=test-002 "sleep 1" --verbose
❯ docker logs otel-col 2>&1 | grep -C5 test-002
InstrumentationScope otel-cli/exec
Span #0
    Trace ID       : 5eff81fbe83ab6bd2027d00a289093f8
    Parent ID      :
    ID             : 85ae91f4d46f73c3
    Name           : test-002
    Kind           : Client
    Start time     : 2023-05-08 13:30:00.372481 +0000 UTC
    End time       : 2023-05-08 13:30:01.382757833 +0000 UTC
    Status code    : Ok
    Status message :

Note the below failure modes

# pushing to grpc endpoint in http mode
❯ OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 otel-cli exec --name=test-001 "sleep 1" --verbose
2023/05/08 18:58:01 OpenTelemetry error: traces export: Post "http://localhost:4317/v1/traces": net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x06\x04\x00\x00\x00\x00\x00\x00\x05\x00\x00@\x00"

# pushing to http endpoint in grpc mode
❯ OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 otel-cli exec --name=test-001 "sleep 1" --verbose --protocol grpc
2023/05/08 18:59:46 OpenTelemetry error: traces export: context deadline exceeded: rpc error: code = Unavailable desc = connection error: desc = "error reading server preface: http2: frame too large"

Child spans

❯ OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 otel-cli exec --name=test-demo-parent --kind producer "OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 otel-cli exec --name=test-demo-child --kind consumer sleep 1" --verbose
❯ docker logs otel-col 2>&1 | egrep -C5 "test-demo"
InstrumentationScope otel-cli/exec
Span #0
    Trace ID       : e383c3e41d049f3b11a85dad316643b7
    Parent ID      : c359f233b3ad20ee
    ID             : 9a70f7873e23b236
    Name           : test-demo-child
    Kind           : Consumer
    Start time     : 2023-05-08 13:40:42.674921 +0000 UTC
    End time       : 2023-05-08 13:40:43.684543334 +0000 UTC
    Status code    : Ok
    Status message :
--
InstrumentationScope otel-cli/exec
Span #0
    Trace ID       : e383c3e41d049f3b11a85dad316643b7
    Parent ID      :
    ID             : c359f233b3ad20ee
    Name           : test-demo-parent
    Kind           : Producer
    Start time     : 2023-05-08 13:40:42.659909 +0000 UTC
    End time       : 2023-05-08 13:40:43.721519167 +0000 UTC
    Status code    : Ok
    Status message :
Attributes:
     -> command: Str(OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 otel-cli exec --name=test-demo-child --kind consumer sleep 1)

With attributes

❯ OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 otel-cli exec --name=test-attr -a foo=bar -a pi=3.14 "sleep 1" --verbose
❯ docker logs otel-col 2>&1 | egrep -C10 "test-attr"
Resource SchemaURL:
Resource attributes:
     -> service.name: Str(otel-cli)
ScopeSpans #0
ScopeSpans SchemaURL:
InstrumentationScope otel-cli/exec
Span #0
    Trace ID       : dca4cf94916d31c29c160a23a8418024
    Parent ID      :
    ID             : 5a4fe44b46241c90
    Name           : test-attr
    Kind           : Client
    Start time     : 2023-05-08 13:43:08.812387 +0000 UTC
    End time       : 2023-05-08 13:43:09.82861175 +0000 UTC
    Status code    : Ok
    Status message :
Attributes:
     -> foo: Str(bar)
     -> pi: Double(3.14)
     -> command: Str(sleep 1)
    {"kind": "exporter", "data_type": "traces", "name": "logging"}

With status update (exit status is considered)

❯ OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 otel-cli exec --name=test-status  --fail "ls /nothing" --verbose
ls: /nothing: No such file or directory
❯ docker logs otel-col 2>&1 | egrep -C12 "test-status"
2023-05-08T13:44:49.513Z    info    TracesExporter  {"kind": "exporter", "data_type": "traces", "name": "logging", "resource spans": 1, "spans": 1}
2023-05-08T13:44:49.513Z    info    ResourceSpans #0
Resource SchemaURL:
Resource attributes:
     -> service.name: Str(otel-cli)
ScopeSpans #0
ScopeSpans SchemaURL:
InstrumentationScope otel-cli/exec
Span #0
    Trace ID       : 183374319aa00c0672b0b9921a1cf862
    Parent ID      :
    ID             : 6e47444744c67a35
    Name           : test-status
    Kind           : Client
    Start time     : 2023-05-08 13:44:49.346037 +0000 UTC
    End time       : 2023-05-08 13:44:49.3641295 +0000 UTC
    Status code    : Error
    Status message : command failed: exit status 1
Attributes:
     -> command: Str(ls /nothing)
Events:
SpanEvent #0
     -> Name: command failed
     -> Timestamp: 2023-05-08 13:44:49.364126 +0000 UTC
     -> DroppedAttributesCount: 0

Trace propagation with a file based trace-parent carrier

❯ OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
❯ carrier=$(mktemp)
❯ otel-cli span -s test-span -k server -n "traceparent-demo" --tp-print --tp-carrier $carrier
# trace id: f0563604541c82962e9b2434f4522e86
#  span id: 17bfae9cd27127dd
TRACEPARENT=00-f0563604541c82962e9b2434f4522e86-17bfae9cd27127dd-01
❯ otel-cli exec --service test-span -n "traceparent-demo-client" --kind client --tp-carrier $carrier "sleep 1"

❯ docker logs otel-col 2>&1 | egrep -C5 "traceparent-demo"
InstrumentationScope otel-cli/span
Span #0
    Trace ID       : f0563604541c82962e9b2434f4522e86
    Parent ID      :
    ID             : 17bfae9cd27127dd
    Name           : traceparent-demo
    Kind           : Server
    Start time     : 2023-05-08 13:48:31.241478 +0000 UTC
    End time       : 2023-05-08 13:48:31.241782 +0000 UTC
    Status code    : Unset
    Status message :
--
InstrumentationScope otel-cli/exec
Span #0
    Trace ID       : f0563604541c82962e9b2434f4522e86
    Parent ID      : 17bfae9cd27127dd
    ID             : 47674b2d5cc516ea
    Name           : traceparent-demo-client
    Kind           : Client
    Start time     : 2023-05-08 13:50:28.751103 +0000 UTC
    End time       : 2023-05-08 13:50:29.776050709 +0000 UTC
    Status code    : Ok
    Status message :

Spans in the background

❯ sockdir=$(mktemp -d)
❯ otel-cli span background -s "span-example" -n test-bg-span --sockdir $sockdir --timeout 60s &
[1] 49185
✦ ❯ otel-cli span event --name "cool thing" --attrs "foo=bar" --sockdir $sockdir
✦ ❯ otel-cli span event --name "another cool thing" --attrs "foo=bar,pi=3.14" --sockdir $sockdir
✦ ❯ otel-cli span event --name "final cool thing" --attrs "foo=bar,pi=3.14" --sockdir $sockdir
✦ ❯ otel-cli span end --sockdir $sockdir
[1]  + 49185 done       otel-cli span background -s "span-example" -n test-bg-span --sockdir $sockdir

❯ docker logs otel-col 2>&1 | egrep -C5 "span-example|cool thing"
Resource SchemaURL:
Resource attributes:
     -> service.name: Str(span-example)
ScopeSpans #0
ScopeSpans SchemaURL:
InstrumentationScope otel-cli/span
Span #0
    Trace ID       : 944487fedd6ea482908fbb26f83f0329
--
    End time       : 2023-05-08 14:18:31.290052 +0000 UTC
    Status code    : Unset
    Status message :
Events:
SpanEvent #0
     -> Name: cool thing
     -> Timestamp: 2023-05-08 14:17:53.720251 +0000 UTC
     -> DroppedAttributesCount: 0
     -> Attributes::
          -> foo: Str(bar)
SpanEvent #1
     -> Name: another cool thing
     -> Timestamp: 2023-05-08 14:18:06.132737 +0000 UTC
     -> DroppedAttributesCount: 0
     -> Attributes::
          -> foo: Str(bar)
          -> pi: Double(3.14)
SpanEvent #2
     -> Name: final cool thing
     -> Timestamp: 2023-05-08 14:18:15.368533 +0000 UTC
     -> DroppedAttributesCount: 0
     -> Attributes::
          -> foo: Str(bar)
          -> pi: Double(3.14)

Official git repo

DEV Community

otel-cli - push otel traces with ease

The problem

Installation

Configuration

Workflow

Top comments (0)

Read next

Solving Maze in the Rat Problem:

YouTube Karaoke - SvelteKit app powered by AssemblyAI

What’s New in WordPress 6.7: Key Features and Updates for Developers

How to Run Databases on Kubernetes: An 8-Step Guide