Observability for LangChain with Splunk

Step 1

Install the Traceloop SDK and initialize it. It will automatically create traces from all the chains you've created through LangChain, with prompts and completions as separate spans.

from langchain.schema import SystemMessage, HumanMessage
from langchain.prompts import ChatPromptTemplate, HumanMessagePromptTemplate
from langchain_community.chat_models import ChatOpenAI
from langchain.chains import LLMChain, SequentialChain, TransformChain

from traceloop.sdk import Traceloop

Traceloop.init(app_name="langchain_example")

chat = ChatOpenAI(temperature=0)

transform = TransformChain(
    input_variables=["subject"],
    output_variables=["prompt"],
    transform=lambda subject: {"prompt": f"Tell me a joke about {subject}."},
)

first_prompt_messages = [
    SystemMessage(content="You are a funny sarcastic nerd."),
    HumanMessage(content="{prompt}"),
]
first_prompt_template = ChatPromptTemplate.from_messages(first_prompt_messages)
first_chain = LLMChain(llm=chat, prompt=first_prompt_template, output_key="joke")

second_prompt_messages = [
    SystemMessage(content="You are an Elf."),
    HumanMessagePromptTemplate.from_template(
        "Translate the joke below into Sindarin language:\n {joke}"
    ),
]
second_prompt_template = ChatPromptTemplate.from_messages(second_prompt_messages)
second_chain = LLMChain(llm=chat, prompt=second_prompt_template)

workflow = SequentialChain(
    chains=[transform, first_chain, second_chain], input_variables=["subject"]
)
print(workflow({"subject": "OpenTelemetry"}))


import * as traceloop from "@traceloop/node-server-sdk";

import { createOpenAIToolsAgent, AgentExecutor } from "langchain/agents";
import { SerpAPI } from "@langchain/community/tools/serpapi";
import { Calculator } from "langchain/tools/calculator";
import { ChatOpenAI } from "@langchain/openai";
import { pull } from "langchain/hub";

traceloop.initialize({
  appName: "sample_langchain",
  apiKey: process.env.TRACELOOP_API_KEY,
  disableBatch: true,
});

const llm = new ChatOpenAI({});
const tools = [new Calculator(), new SerpAPI()];
const prompt = await pull(
  "hwchase17/openai-tools-agent",
);
const agent = await createOpenAIToolsAgent({ llm, tools, prompt });
const agentExecutor = new AgentExecutor({
  agent,
  tools,
});
const result = await agentExecutor.invoke({
  input:
    "By searching the Internet, find how many albums " + 
    "has Boldy James dropped since 2010 and how many " + 
    "albums has Nas dropped since 2010? Find who dropped " + 
    "more albums and show the difference in percent.",
});
return result;

Step 2

Configure your collector to collect traces and send them to Splunk Observability Cloud; point OpenLLMetry to send traces to your Splunk collector

# In your Splunk collector configuration
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: "0.0.0.0:4317"
      http:
        endpoint: "0.0.0.0:4318"
exporters:
  # Traces
  sapm:
    access_token: "${SPLUNK_ACCESS_TOKEN}"
    endpoint: "https://ingest.${SPLUNK_REALM}.signalfx.com/v2/trace"
    sending_queue:
      num_consumers: 32
 pipelines:
  traces:
    receivers: [jaeger, otlp, sapm, zipkin]
    processors:
    - memory_limiter
    - batch
    #- resource/add_environment
    exporters: [sapm]
        
# In the environment of your application
TRACELOOP_BASE_URL=http://splunk-otel-collector:4318

Docs

Observability for LangChain with Splunk

Step 1

Step 2

Discover use cases

Trace chains built with LangChain

More things you can connect to {platform}

Bedrock

LlamaIndex

OpenAI

Watsonx

together.ai

Anthropic

Chroma

Mistral AI

Cohere

Gemini

Ollama