Observability for Ollama with Instana

OpenLLMetry lets you trace any model running locally or remotely on Ollama. With 5 minutes of work you can get complete view of your system directly into Instana. See how below.

Step 1

Install the Traceloop SDK and initialize it. It will automatically log all calls to Ollama, whether you're running a model locally or remotely, with prompts and completions as separate spans.

import ollama
from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import workflow

Traceloop.init(app_name="joke_generation_service")

@workflow(name="joke_creation")
def create_joke():
		response = ollama.chat(
        model="llama3",
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke about OpenTelemetry",
            },
        ],
    )

    return response["message"]["content"]

Step 2

Configure your Instana agent to receive traces and metrics from OpenLLMetry

# In your Instana OS agent configuration file
com.instana.plugin.opentelemetry:
  enabled: true
  grpc:
    enabled: true
        
# In the environment of your application
TRACELOOP_BASE_URL=instana-agent-hostname:4317

Docs

Observability for Ollama with Instana

Step 1

Step 2

Discover use cases

More things you can connect to {platform}

OpenAI

Gemini

Mistral AI

Bedrock

LlamaIndex

Anthropic

together.ai

Cohere

LangChain

Watsonx

Chroma