Observability for Gemini with Splunk

Step 1

Install the Traceloop SDK and initialize it. It will automatically log all calls to Gemini or any other VertexAI models, with prompts and completions as separate spans.

import google.generativeai as genai
from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import workflow

Traceloop.init(app_name="joke_generation_service")

genai.configure(api_key=os.environ.get("GENAI_API_KEY"))

@workflow(name="joke_creation")
def create_joke():
    model = genai.GenerativeModel("gemini-1.0-pro-latest")
    response = model.generate_content(
        "Give me ten interview questions for the role of program manager.",
    )

    return response.text

Step 2

Configure your collector to collect traces and send them to Splunk Observability Cloud; point OpenLLMetry to send traces to your Splunk collector

# In your Splunk collector configuration
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: "0.0.0.0:4317"
      http:
        endpoint: "0.0.0.0:4318"
exporters:
  # Traces
  sapm:
    access_token: "${SPLUNK_ACCESS_TOKEN}"
    endpoint: "https://ingest.${SPLUNK_REALM}.signalfx.com/v2/trace"
    sending_queue:
      num_consumers: 32
 pipelines:
  traces:
    receivers: [jaeger, otlp, sapm, zipkin]
    processors:
    - memory_limiter
    - batch
    #- resource/add_environment
    exporters: [sapm]
        
# In the environment of your application
TRACELOOP_BASE_URL=http://splunk-otel-collector:4318

Docs

Observability for Gemini with Splunk

Step 1

Step 2

Discover use cases

More things you can connect to {platform}

Bedrock

LlamaIndex

OpenAI

Watsonx

together.ai

Anthropic

Chroma

Mistral AI

Cohere

LangChain

Ollama