Observability for LangChain with Highlight

Step 1

Install the Traceloop SDK and initialize it. It will automatically create traces from all the chains you've created through LangChain, with prompts and completions as separate spans.

from langchain.schema import SystemMessage, HumanMessage
from langchain.prompts import ChatPromptTemplate, HumanMessagePromptTemplate
from langchain_community.chat_models import ChatOpenAI
from langchain.chains import LLMChain, SequentialChain, TransformChain

from traceloop.sdk import Traceloop

Traceloop.init(app_name="langchain_example")

chat = ChatOpenAI(temperature=0)

transform = TransformChain(
    input_variables=["subject"],
    output_variables=["prompt"],
    transform=lambda subject: {"prompt": f"Tell me a joke about {subject}."},
)

first_prompt_messages = [
    SystemMessage(content="You are a funny sarcastic nerd."),
    HumanMessage(content="{prompt}"),
]
first_prompt_template = ChatPromptTemplate.from_messages(first_prompt_messages)
first_chain = LLMChain(llm=chat, prompt=first_prompt_template, output_key="joke")

second_prompt_messages = [
    SystemMessage(content="You are an Elf."),
    HumanMessagePromptTemplate.from_template(
        "Translate the joke below into Sindarin language:\n {joke}"
    ),
]
second_prompt_template = ChatPromptTemplate.from_messages(second_prompt_messages)
second_chain = LLMChain(llm=chat, prompt=second_prompt_template)

workflow = SequentialChain(
    chains=[transform, first_chain, second_chain], input_variables=["subject"]
)
print(workflow({"subject": "OpenTelemetry"}))


import * as traceloop from "@traceloop/node-server-sdk";

import { createOpenAIToolsAgent, AgentExecutor } from "langchain/agents";
import { SerpAPI } from "@langchain/community/tools/serpapi";
import { Calculator } from "langchain/tools/calculator";
import { ChatOpenAI } from "@langchain/openai";
import { pull } from "langchain/hub";

traceloop.initialize({
  appName: "sample_langchain",
  apiKey: process.env.TRACELOOP_API_KEY,
  disableBatch: true,
});

const llm = new ChatOpenAI({});
const tools = [new Calculator(), new SerpAPI()];
const prompt = await pull(
  "hwchase17/openai-tools-agent",
);
const agent = await createOpenAIToolsAgent({ llm, tools, prompt });
const agentExecutor = new AgentExecutor({
  agent,
  tools,
});
const result = await agentExecutor.invoke({
  input:
    "By searching the Internet, find how many albums " + 
    "has Boldy James dropped since 2010 and how many " + 
    "albums has Nas dropped since 2010? Find who dropped " + 
    "more albums and show the difference in percent.",
});
return result;

Step 2

Route traces to Highlights’s OTLP endpoint and set the Highlight project ID in the headers.

TRACELOOP_BASE_URL=https://otel.highlight.io:4318
TRACELOOP_HEADERS="x-highlight-project=<YOUR_HIGHLIGHT_PROJECT_ID>"

Docs

Observability for LangChain with Highlight

Step 1

Step 2

Discover use cases

Trace chains built with LangChain

More things you can connect to {platform}

Chroma

OpenAI

Cohere

Ollama

Gemini

Watsonx

Bedrock

Anthropic

LlamaIndex

Mistral AI

together.ai