Observability for Watsonx with Dynatrace

Step 1

Install the Traceloop SDK and initialize it. It will automatically log all calls to Watsonx, with prompts and completions as separate spans.

from ibm_watson_machine_learning.foundation_models import Model

from traceloop.sdk import Traceloop

Traceloop.init(app_name="watsonx_example")


def get_credentials(api_key):
    return {
        "url": "https://us-south.ml.cloud.ibm.com",
        "apikey": api_key,
    }


iam_api_key = os.environ["IAM_API_KEY"]
project_id = os.environ["PROJECT_ID"]

prompt_input = """Calculate result

Input:
what is the capital of China.

Output:
"""

model_id = "meta-llama/llama-2-70b-chat"
parameters = {
    "decoding_method": "sample",
    "max_new_tokens": 60,
    "min_new_tokens": 10,
    "random_seed": 111,
    "temperature": 0.9,
    "top_k": 50,
    "top_p": 1,
    "repetition_penalty": 2
}

model = Model(
    model_id=model_id,
    params=parameters,
    credentials=get_credentials(iam_api_key),
    project_id=project_id
    )

prompt_input = "What is the the captical of China"
generated_response = model.generate(prompt=prompt_input)

Step 2

Go to your Dynatrace environment and create a new access token under Manage Access Tokens. The access token needs the following permission scopes that allow the ingest of OpenTelemetry spans, metrics and logs (openTelemetryTrace.ingest, metrics.ingest, logs.ingest). Set the following environment variables.

TRACELOOP_BASE_URL=https://<YOUR_ENV>.live.dynatrace.com\api\v2\otlp
TRACELOOP_HEADERS=Authorization=Api-Token%20<YOUR_ACCESS_TOKEN>

Docs

Discover use cases

Trace prompts and completions in Watsonx

Call Watsonx and see prompts, completions, and token usage for your call.

See Code Example

Trace your RAG retrieval pipeline

Build a RAG pipeline with Chroma and OpenAI. See vectors returned from Chroma, full prompt in OpenAI and responses

See Code Example

Observability for Watsonx with Dynatrace

Step 1

Step 2

Discover use cases

Trace prompts and completions in Watsonx

Trace your RAG retrieval pipeline

More things you can connect to {platform}

Anthropic

LlamaIndex

Ollama

Bedrock

Gemini

Mistral AI

OpenAI

together.ai

Cohere

LangChain

Chroma