LLM Observability with Google Cloud Trace and OpenLLMetry

Google Cloud, also known as Google Cloud Platform (GCP), is a cloud provider including over 150+ products and services. Among these products and services is Cloud Trace, which is a part of the Google Cloud Observability suite alongside Cloud Monitoring and Cloud Logging.

Traceloop’s OpenLLMetry library enables instrumenting LLM frameworks in an OTel-aligned manner and supports writing that instrumentation data as distributed traces to Cloud Trace. Traceloop’s OpenLLMetry library can be additionally configured to also write to Cloud Monitoring and to Cloud Logging.

​

Minimal Integration: Cloud Trace only

​

Dependencies

• opentelemetry-exporter-gcp-trace
• traceloop-sdk

​

OpenLLMetry Configuration

# ...
from opentelemetry.exporter.cloud_trace import CloudTraceSpanExporter
from traceloop.sdk import Traceloop

# ...
trace_exporter = CloudTraceSpanExporter()
Traceloop.init(
    app_name='your-app-name',
    exporter=trace_exporter)

​

Integration with Cloud Trace and Cloud Monitoring

​

Dependencies

• opentelemetry-exporter-gcp-trace
• opentelemetry-exporter-gcp-monitoring
• traceloop-sdk

​

OpenLLMetry Configuration

# ...
from opentelemetry.exporter.cloud_trace import CloudTraceSpanExporter
from opentelemetry.exporter.cloud_monitoring import CloudMonitoringMetricsExporter
from traceloop.sdk import Traceloop

# ...
trace_exporter = CloudTraceSpanExporter()
metrics_exporter = CloudMonitoringMetricsExporter()
Traceloop.init(
    app_name='your-app-name',
    exporter=trace_exporter,
    metrics_exporter=metrics_exporter)

​

Integration with Cloud Trace, Cloud Monitoring, and Cloud Logging

As of 2024-12-05, there is no direct Python OTel exporter for Cloud Logging. However, it is possible to work around this by routing to the OTel Collector and using the googlecloudexporter in the OTel Collector to route logs to Google Cloud Logging. Alternatively, you can use the Python ConsoleLogExporter to log to a file in conjunction with automatic file-based log routing such as via the Ops Agent, direct use of FluentBit, or other routing solutions.

​

Dependencies

​

OTel Dependencies

This is assuming that you write to Cloud Logging via the OTel Collector:

• opentelemetry-collector-contrib
• googlecloudexporter

​

Python Dependencies

• opentelemetry-exporter-gcp-trace
• opentelemetry-exporter-gcp-monitoring
• opentelemetry-exporter-otlp-proto-grpc
• traceloop-sdk

​

OpenLLMetry Configuration

# ...
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
from opentelemetry.exporter.cloud_trace import CloudTraceSpanExporter
from opentelemetry.exporter.cloud_monitoring import CloudMonitoringMetricsExporter
from traceloop.sdk import Traceloop

# ...
trace_exporter = CloudTraceSpanExporter()
metrics_exporter = CloudMonitoringMetricsExporter()

# Will use the 'OTEL_EXPORTER_OTLP_LOGS_*' environment
# variables (such as 'OTEL_EXPORTER_OTLP_LOGS_ENDPOINT')
# to select how to connect to the OTel Collector.   
logs_exporter = OTLPLogExporter()

Traceloop.init(
    app_name='your-app-name',
    exporter=trace_exporter,
    metrics_exporter=metrics_exporter,
    logging_exporter=logs_exporter)

​

Advanced Topics

​

Large Span Attributes

Although OTel Semantic Conventions for GenAI has been moving from prompt/response logging in span attributes to recording prompts/responses in events (logs) — see related pull request —, some versions of OpenLLMetry and related instrumentation packages follow older experimental conventions of recording prompts/responses in span attributes which do not fit within Cloud Trace limits.

The e2e-gen-ai-app-starter-pack code sample provided by Google illustrates a potential workaround; that app, calls Traceloop.init with a custom CloudTraceLoggingSpanExporter which inherits from the official CloudTraceSpanExporter and adds additional capabilities for handling large span attributes.