Observability for LangChain with New Relic

Step 1

Install the Traceloop SDK and initialize it. It will automatically create traces from all the chains you've created through LangChain, with prompts and completions as separate spans.

from langchain.schema import SystemMessage, HumanMessage
from langchain.prompts import ChatPromptTemplate, HumanMessagePromptTemplate
from langchain_community.chat_models import ChatOpenAI
from langchain.chains import LLMChain, SequentialChain, TransformChain

from traceloop.sdk import Traceloop

Traceloop.init(app_name="langchain_example")

chat = ChatOpenAI(temperature=0)

transform = TransformChain(
    input_variables=["subject"],
    output_variables=["prompt"],
    transform=lambda subject: {"prompt": f"Tell me a joke about {subject}."},
)

first_prompt_messages = [
    SystemMessage(content="You are a funny sarcastic nerd."),
    HumanMessage(content="{prompt}"),
]
first_prompt_template = ChatPromptTemplate.from_messages(first_prompt_messages)
first_chain = LLMChain(llm=chat, prompt=first_prompt_template, output_key="joke")

second_prompt_messages = [
    SystemMessage(content="You are an Elf."),
    HumanMessagePromptTemplate.from_template(
        "Translate the joke below into Sindarin language:\n {joke}"
    ),
]
second_prompt_template = ChatPromptTemplate.from_messages(second_prompt_messages)
second_chain = LLMChain(llm=chat, prompt=second_prompt_template)

workflow = SequentialChain(
    chains=[transform, first_chain, second_chain], input_variables=["subject"]
)
print(workflow({"subject": "OpenTelemetry"}))


import * as traceloop from "@traceloop/node-server-sdk";

import { createOpenAIToolsAgent, AgentExecutor } from "langchain/agents";
import { SerpAPI } from "@langchain/community/tools/serpapi";
import { Calculator } from "langchain/tools/calculator";
import { ChatOpenAI } from "@langchain/openai";
import { pull } from "langchain/hub";

traceloop.initialize({
  appName: "sample_langchain",
  apiKey: process.env.TRACELOOP_API_KEY,
  disableBatch: true,
});

const llm = new ChatOpenAI({});
const tools = [new Calculator(), new SerpAPI()];
const prompt = await pull(
  "hwchase17/openai-tools-agent",
);
const agent = await createOpenAIToolsAgent({ llm, tools, prompt });
const agentExecutor = new AgentExecutor({
  agent,
  tools,
});
const result = await agentExecutor.invoke({
  input:
    "By searching the Internet, find how many albums " + 
    "has Boldy James dropped since 2010 and how many " + 
    "albums has Nas dropped since 2010? Find who dropped " + 
    "more albums and show the difference in percent.",
});
return result;

Step 2

Get an access key from New Relic dashboard and set the following environment variables.

TRACELOOP_BASE_URL=https://otlp.nr-data.net:443
TRACELOOP_HEADERS="api-key=<YOUR_NEWRELIC_LICENSE_KEY>"

Docs

Observability for LangChain with New Relic

Step 1

Step 2

Discover use cases

Trace chains built with LangChain

More things you can connect to {platform}

LlamaIndex

Watsonx

together.ai

Ollama

Bedrock

Anthropic

OpenAI

Mistral AI

Cohere

Gemini

Chroma