Observability for LangChain with Honeycomb

Step 1

Install the Traceloop SDK and initialize it. It will automatically create traces from all the chains you've created through LangChain, with prompts and completions as separate spans.

from langchain.schema import SystemMessage, HumanMessage
from langchain.prompts import ChatPromptTemplate, HumanMessagePromptTemplate
from langchain_community.chat_models import ChatOpenAI
from langchain.chains import LLMChain, SequentialChain, TransformChain

from traceloop.sdk import Traceloop

Traceloop.init(app_name="langchain_example")

chat = ChatOpenAI(temperature=0)

transform = TransformChain(
    input_variables=["subject"],
    output_variables=["prompt"],
    transform=lambda subject: {"prompt": f"Tell me a joke about {subject}."},
)

first_prompt_messages = [
    SystemMessage(content="You are a funny sarcastic nerd."),
    HumanMessage(content="{prompt}"),
]
first_prompt_template = ChatPromptTemplate.from_messages(first_prompt_messages)
first_chain = LLMChain(llm=chat, prompt=first_prompt_template, output_key="joke")

second_prompt_messages = [
    SystemMessage(content="You are an Elf."),
    HumanMessagePromptTemplate.from_template(
        "Translate the joke below into Sindarin language:\n {joke}"
    ),
]
second_prompt_template = ChatPromptTemplate.from_messages(second_prompt_messages)
second_chain = LLMChain(llm=chat, prompt=second_prompt_template)

workflow = SequentialChain(
    chains=[transform, first_chain, second_chain], input_variables=["subject"]
)
print(workflow({"subject": "OpenTelemetry"}))


import * as traceloop from "@traceloop/node-server-sdk";

import { createOpenAIToolsAgent, AgentExecutor } from "langchain/agents";
import { SerpAPI } from "@langchain/community/tools/serpapi";
import { Calculator } from "langchain/tools/calculator";
import { ChatOpenAI } from "@langchain/openai";
import { pull } from "langchain/hub";

traceloop.initialize({
  appName: "sample_langchain",
  apiKey: process.env.TRACELOOP_API_KEY,
  disableBatch: true,
});

const llm = new ChatOpenAI({});
const tools = [new Calculator(), new SerpAPI()];
const prompt = await pull(
  "hwchase17/openai-tools-agent",
);
const agent = await createOpenAIToolsAgent({ llm, tools, prompt });
const agentExecutor = new AgentExecutor({
  agent,
  tools,
});
const result = await agentExecutor.invoke({
  input:
    "By searching the Internet, find how many albums " + 
    "has Boldy James dropped since 2010 and how many " + 
    "albums has Nas dropped since 2010? Find who dropped " + 
    "more albums and show the difference in percent.",
});
return result;

Step 2

Route the traces to Honeycomb’s endpoint and set the API key

TRACELOOP_BASE_URL=https://api.honeycomb.io
TRACELOOP_HEADERS="x-honeycomb-team=<YOUR_API_KEY>"

Docs

Observability for LangChain with Honeycomb

Step 1

Step 2

Discover use cases

Trace chains built with LangChain

More things you can connect to {platform}

Anthropic

LlamaIndex

Gemini

Ollama

OpenAI

Bedrock

together.ai

Cohere

Watsonx

Mistral AI

Chroma