Observability for Ollama with Dynatrace

OpenLLMetry lets you trace any model running locally or remotely on Ollama. With 5 minutes of work you can get complete view of your system directly into Dynatrace. See how below.

Step 1

Install the Traceloop SDK and initialize it. It will automatically log all calls to Ollama, whether you're running a model locally or remotely, with prompts and completions as separate spans.

import ollama
from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import workflow

Traceloop.init(app_name="joke_generation_service")

@workflow(name="joke_creation")
def create_joke():
		response = ollama.chat(
        model="llama3",
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke about OpenTelemetry",
            },
        ],
    )

    return response["message"]["content"]

Step 2

Go to your Dynatrace environment and create a new access token under Manage Access Tokens. The access token needs the following permission scopes that allow the ingest of OpenTelemetry spans, metrics and logs (openTelemetryTrace.ingest, metrics.ingest, logs.ingest). Set the following environment variables.

TRACELOOP_BASE_URL=https://<YOUR_ENV>.live.dynatrace.com\api\v2\otlp
TRACELOOP_HEADERS=Authorization=Api-Token%20<YOUR_ACCESS_TOKEN>

Docs

Observability for Ollama with Dynatrace

Step 1

Step 2

Discover use cases

More things you can connect to {platform}

Anthropic

Watsonx

LlamaIndex

Bedrock

Gemini

Mistral AI

OpenAI

together.ai

Cohere

LangChain

Chroma