Observability for Gemini with Highlight

OpenLLMetry lets you trace models running on VertexAI. With 5 minutes of work you can get complete view of your system directly into Honeycomb. See how below.

Step 1

Install the Traceloop SDK and initialize it. It will automatically log all calls to Gemini or any other VertexAI models, with prompts and completions as separate spans.

import google.generativeai as genai
from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import workflow

Traceloop.init(app_name="joke_generation_service")

genai.configure(api_key=os.environ.get("GENAI_API_KEY"))

@workflow(name="joke_creation")
def create_joke():
    model = genai.GenerativeModel("gemini-1.0-pro-latest")
    response = model.generate_content(
        "Give me ten interview questions for the role of program manager.",
    )

    return response.text

Step 2

Route traces to Highlights’s OTLP endpoint and set the Highlight project ID in the headers.

TRACELOOP_BASE_URL=https://otel.highlight.io:4318
TRACELOOP_HEADERS="x-highlight-project=<YOUR_HIGHLIGHT_PROJECT_ID>"

Docs

Observability for Gemini with Highlight

Step 1

Step 2

Discover use cases

More things you can connect to {platform}

Chroma

OpenAI

Cohere

Ollama

Watsonx

Bedrock

Anthropic

LangChain

LlamaIndex

Mistral AI

together.ai