Observability for OpenAI with Instana

Step 1

Install the Traceloop SDK and initialize it. It will automatically log all calls to OpenAI, with prompts and completions as separate spans.

import openai
from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import workflow

Traceloop.init(app_name="joke_generation_service")

@workflow(name="joke_creation")
def create_joke():
    completion = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": "Tell me a joke about opentelemetry"}],
    )

    return completion.choices[0].message.content

import * as traceloop from "@traceloop/node-server-sdk";
import OpenAI from "openai";

Traceloop.init({ app_name="joke_generation_service" })
const openai = new OpenAI();

class MyLLM {
  @traceloop.workflow("joke_creation")
  async create_joke():
      completion = await openai.chat.completions.create({
          model: "gpt-3.5-turbo",
          messages: [{"role": "user", "content": "Tell me a joke about opentelemetry"}],
      })

      return completion.choices[0].message.content
}

Step 2

Configure your Instana agent to receive traces and metrics from OpenLLMetry

# In your Instana OS agent configuration file
com.instana.plugin.opentelemetry:
  enabled: true
  grpc:
    enabled: true
        
# In the environment of your application
TRACELOOP_BASE_URL=instana-agent-hostname:4317

Docs