Observability for Ollama with Splunk

Step 1

Install the Traceloop SDK and initialize it. It will automatically log all calls to Ollama, whether you're running a model locally or remotely, with prompts and completions as separate spans.

import ollama
from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import workflow

Traceloop.init(app_name="joke_generation_service")

@workflow(name="joke_creation")
def create_joke():
		response = ollama.chat(
        model="llama3",
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke about OpenTelemetry",
            },
        ],
    )

    return response["message"]["content"]

Step 2

Configure your collector to collect traces and send them to Splunk Observability Cloud; point OpenLLMetry to send traces to your Splunk collector

# In your Splunk collector configuration
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: "0.0.0.0:4317"
      http:
        endpoint: "0.0.0.0:4318"
exporters:
  # Traces
  sapm:
    access_token: "${SPLUNK_ACCESS_TOKEN}"
    endpoint: "https://ingest.${SPLUNK_REALM}.signalfx.com/v2/trace"
    sending_queue:
      num_consumers: 32
 pipelines:
  traces:
    receivers: [jaeger, otlp, sapm, zipkin]
    processors:
    - memory_limiter
    - batch
    #- resource/add_environment
    exporters: [sapm]
        
# In the environment of your application
TRACELOOP_BASE_URL=http://splunk-otel-collector:4318

Docs

Observability for Ollama with Splunk

Step 1

Step 2

Discover use cases

More things you can connect to {platform}

Bedrock

LlamaIndex

OpenAI

Watsonx

together.ai

Anthropic

Chroma

Mistral AI

Cohere

Gemini

LangChain