Observability for Cohere with Grafana Tempo

Step 1

Install the Traceloop SDK and initialize it. It will automatically log all calls to Cohere, with prompts and completions as separate spans.

import boto3
import json

from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import task, workflow

Traceloop.init(app_name="joke_generation_service")
co = cohere.Client(os.environ.get("COHERE_API_KEY"))

import * as traceloop from "@traceloop/node-server-sdk";
import { CohereClient } from "cohere-ai";

traceloop.initialize({
  appName: "sample_cohere",
  apiKey: process.env.TRACELOOP_API_KEY,
  disableBatch: true,
});

const cohere = new CohereClient({
  token: process.env.COHERE_API_KEY ?? "",
});

const sampleGenerate = async () => {
  return await traceloop.withWorkflow({ name: "sample_generate" }, async () => {
    const response = await cohere.generate({
      model: "command-light",
      prompt: "What happened to Pluto?",
      k: 1,
      temperature: 2,
    });

    return response.generations?.[0].text;
  });
};

traceloop.withAssociationProperties({}, async () => {
  console.log(await sampleGenerate());
});

Step 2

Make sure you have a Grafana Agent installed and running in your system. Go to the Grafana Cloud account page under https://grafana.com/orgs/<your org name>, and click on Send Traces under Tempo. In Grafana Data Source settings, note the URL value. Click Generate now to generate an API key and copy it. Note also the Stack ID value (you can find it in the URL https://grafana.com/orgs/<Your Org Name>/stacks/<Stack ID>). Add this to the configuration of your Grafana Agent.

TRACELOOP_BASE_URL=https://api.honeycomb.io
TRACELOOP_HEADERS="x-honeycomb-team=<YOUR_API_KEY>"

Docs

Observability for Cohere with Grafana Tempo

Step 1

Step 2

Discover use cases

Trace prompts and completions in Cohere

More things you can connect to {platform}

Chroma

LlamaIndex

Gemini

together.ai

LangChain

Ollama

Bedrock

Watsonx

Anthropic

Mistral AI

OpenAI