Distributed Tracing
How to solve technical problems
When solving problems, it is important to know how clients and eva communicate with each other and how eva’s multiple modules talk to each other during the platform operation. As eva uses ISTIO as service mesh, it is possible to view dialogs systemic errors using an ISTIO package tool called Jaeger
Like Dapper and OpenZipkit, it is an open code distributed tracing system used to monitor and solve problems in microservices-based system such as:
    Distributed context propagation
    Distributed transactions monitoring
    Root cause analysis
    Service dependence analysis
    Latency / performance optimization
For further information, visit

Accessing Jaeger

Important:
    Before accessing Jaeger, keep in mind that it deals with sensible information.
By default, Jaeger is not accessible through the internet. To access it, forward the port using the commands below:
1
$ gcloud container clusters get-credentials <cluster_name> --zone <zone>
2
--project <Project ID>
Copied!
1
$ kubectl port-forward --namespace istio-system $(kubectl get pod --namespace
2
istio-system --selector="app=jaeger" --output
3
jsonpath='{.items[0].metadata.name}') 16686:16686
Copied!
If there are no errors, the following message appears:
1
$ Forwarding from 127.0.0.1:16686 -> 16686
Copied!

Looking for traces of a specific session ID

With access to Jaeger, open your browser with the address: http://localhost:16686
Could not load image
In the upper right corner, you can search for traces related to a specific “sessioncode”, fill the TAG field with a session ID such as:
1
sessionCode=0c2d189f-ebfc-4253-9152-77e29de946ad
Copied!
If you need to search for traces of a failed requisition, use the command:
1
error=true sessionCode=0c2d189f-ebfc-4253-9152-77e29de946ad
Copied!

Understanding the Trace

Before understanding na eva log in Jaeger, it is important to understand two concepts: span and trace.
A span is a logical work unit in Jaeger that has an operation name, start time and duration. Spans can be aligned and ordered to model causal relationships.
The Trace is the whole execution path though all modules that the requisition went though and can be viewed as an acyclical ordered spans chart.
With the search done and after clicking the selected trace:
It is possible to view the spans that happened during the call.
The red icon (!) helps to find the span containing the detailed error, for example:

Requisition headers, opentracing standard

ISTIO uses Envoy to integrate tracing by offering options to install several tracking back-ends and proxy configurations to send tracking spans automatically to them.
In eva, the default tool is Jaeger.
To make a correlation between the many requisitions that deals with a dialog, eva was changed to forward the headers below in all internal calls between modules. Doing so, it is possible to correlate independently from the instances in execution.
The headers defined in the Opentracing specifications are:
    x-request-id
    x-b3-traceid
    x-b3-spanid
    x-b3-parentspanid
    x-b3-sampled
    x-b3-flags
    x-ot-span-context

Adding a custom ID to the trace

Sometimes it is necessary to add a custom ID to the requisition to the broker so there is an identifier that can be used to search traces.
For further information, check:
Last modified 5mo ago