Observability for together.ai with Instana

OpenLLMetry lets you trace any model running on together.ai platform. With 5 minutes of work you can get complete view of your system directly into Instana. See how below.

Step 1

Install the Traceloop SDK and initialize it. It will automatically log all calls to models served from together.ai, with prompts and completions as separate spans.

from together import Together
from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import workflow

Traceloop.init(app_name="joke_generation_service")

client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))

@workflow(name="joke_creation")
def create_joke():
    response = client.chat.completions.create(
        model="mistralai/Mixtral-8x7B-Instruct-v0.1",
        messages=[{"role": "user", "content": "Tell me a joke about OpenTelemetry."}],
    )

    return response.choices[0].message.content

Step 2

Configure your Instana agent to receive traces and metrics from OpenLLMetry

# In your Instana OS agent configuration file
com.instana.plugin.opentelemetry:
  enabled: true
  grpc:
    enabled: true
        
# In the environment of your application
TRACELOOP_BASE_URL=instana-agent-hostname:4317

Docs

Observability for together.ai with Instana

Step 1

Step 2

Discover use cases

More things you can connect to {platform}

OpenAI

Gemini

Mistral AI

Bedrock

Ollama

LlamaIndex

Anthropic

Cohere

LangChain

Watsonx

Chroma