Observability for LlamaIndex with Grafana Tempo

Step 1

Install and initialize the SDK in your code. It will automatically detect the workflow structure in build traces.

import asyncio

import chromadb
import os

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, StorageContext
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from traceloop.sdk import Traceloop

os.environ["TOKENIZERS_PARALLELISM"] = "false"

Traceloop.init(app_name="llama_index_example")

chroma_client = chromadb.EphemeralClient()
chroma_collection = chroma_client.create_collection("quickstart")

# define embedding function
embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-base-en-v1.5")

# load documents
documents = SimpleDirectoryReader("./data/paul_graham/").load_data()

# set up ChromaVectorStore and load in data
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context, embed_model=embed_model
)


async def main():
    # Query Data
    query_engine = index.as_query_engine()
    response = await query_engine.aquery("What did the author do growing up?")

    print(response)


if __name__ == "__main__":
    asyncio.run(main())

import * as traceloop from "@traceloop/node-server-sdk";
import {
  OpenAIEmbedding,
  SimpleDirectoryReader,
  SimpleVectorStore,
  VectorStoreIndex,
  serviceContextFromDefaults,
  storageContextFromDefaults,
} from "llamaindex";

traceloop.initialize({
  appName: "sample_llamaindex",
  apiKey: process.env.TRACELOOP_API_KEY,
  disableBatch: true,
});


const documents = await new SimpleDirectoryReader().loadData({
  // Download from:
  // https://github.com/traceloop/openllmetry-js/tree/main/packages/sample-app/data/paul_graham
  directoryPath: "data/paul_graham", 
});

const serviceContext = serviceContextFromDefaults({ embedModel });
const storageContext = await storageContextFromDefaults({ vectorStore });

const index = await VectorStoreIndex.fromDocuments(documents, {
  storageContext,
  serviceContext,
});

const queryEngine = index.asQueryEngine();

const res = await queryEngine.query("What did the author do growing up?");
console.log(res)

Step 2

Make sure you have a Grafana Agent installed and running in your system. Go to the Grafana Cloud account page under https://grafana.com/orgs/<your org name>, and click on Send Traces under Tempo. In Grafana Data Source settings, note the URL value. Click Generate now to generate an API key and copy it. Note also the Stack ID value (you can find it in the URL https://grafana.com/orgs/<Your Org Name>/stacks/<Stack ID>). Add this to the configuration of your Grafana Agent.

TRACELOOP_BASE_URL=https://api.honeycomb.io
TRACELOOP_HEADERS="x-honeycomb-team=<YOUR_API_KEY>"

Docs

Observability for LlamaIndex with Grafana Tempo

Step 1

Step 2

Discover use cases

Trace a LlamaIndex-based RAG pipeline

Trace a LlamaIndex-based Agent

More things you can connect to {platform}

Chroma

Gemini

together.ai

LangChain

Ollama

Bedrock

Watsonx

Cohere

Anthropic

Mistral AI

OpenAI