Observability for Anthropic with New Relic

OpenLLMetry lets you trace prompts and embedding calls of Anthropic. With 5 minutes of work you can get complete view of your system directly into New Relic. See how below.

Step 1

Install the Traceloop SDK and initialize it. It will automatically log all calls to Anthropic, with prompts and completions as separate spans.

import boto3
import json

from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import task, workflow

Traceloop.init(app_name="joke_generation_service")
client = Anthropic()

import * as traceloop from "@traceloop/node-server-sdk";
import Anthropic from "@anthropic-ai/sdk";

traceloop.initialize({
  appName: "sample_anthropic",
  apiKey: process.env.TRACELOOP_API_KEY,
  disableBatch: true,
});
const anthropic = new Anthropic({});

async function main() {
  const completion = await anthropic.messages.create({
    max_tokens: 1024,
    model: "claude-3-opus-20240229",
    messages: [
      {
        role: "user",
        content: "How does a court case get to the Supreme Court?",
      },
    ],
  });
  console.log(completion.content);
}

main();

Step 2

Get an access key from New Relic dashboard and set the following environment variables.

TRACELOOP_BASE_URL=https://otlp.nr-data.net:443
TRACELOOP_HEADERS="api-key=<YOUR_NEWRELIC_LICENSE_KEY>"

Docs

Discover use cases

Trace prompts and completions in Anthropic

Call any Claude model by Anthropic and see prompts, completions, and token usage for your call. Both Claude Instant and Claude 2 are supported.

See Code Example

Observability for Anthropic with New Relic

Step 1

Step 2

Discover use cases

Trace prompts and completions in Anthropic

More things you can connect to {platform}

LlamaIndex

LangChain

Watsonx

together.ai

Ollama

Bedrock

OpenAI

Mistral AI

Cohere

Gemini

Chroma