From vibes to visibility: Why we built Traceloop
May 2025
Author Page
Nir Gazit
CEO @ traceloop. Ex-Google, formerly chief architect at Fiverr. Over 15 years of experience building software. M.Sc in CS from the Hebrew University in Jerusalem.
@
nir_ga
A Comprehensive Guide to Tokenizing Text for LLMs
March 2024
Tokenization, the process of breaking text into smaller units called tokens, forms the foundation of how Large Language Models like BERT and GPT understand and generate human language. This article explores different tokenization algorithms, their applications in Natural Language Processing tasks, common challenges, and how they compare to other text evaluation metrics, highlighting tokenization's crucial role in bridging human and machine understanding.
Read more →
Practical Steps for a Smooth LLM Application Rollout
April 2024
Deploying Large Language Models (LLMs) requires careful planning across security, infrastructure, and monitoring to ensure successful production implementation. This article provides a practical checklist covering essential steps from defining objectives to gathering user feedback, helping teams navigate the complexities of LLM deployment while maintaining optimal performance.
Read more →
Lessons Learned from Using LLMs to Evaluate LLMs
February 2024
While Large Language Models (LLMs) are commonly used to evaluate other LLMs' performance, new research shows they produce highly inconsistent scores when rating the same text multiple times, making them unreliable as evaluation tools. The article suggests using established, deterministic metrics like BLEU and ROUGE instead, or implementing statistical methods that account for this scoring variability, to more accurately compare different LLM models and prompts.
Read more →
Introducing OpenLLMetry — Extending OpenTelemetry to LLMs
October 2023
OpenLLMetry has been released as an open-source extension to OpenTelemetry, providing observability tools for LLM and AI applications without vendor lock-in, similar to how OpenTelemetry revolutionized cloud observability. The tool offers instrumentations for various LLM platforms (like OpenAI and Anthropic), vector databases, and frameworks, allowing developers to track LLM responses, prompts, and performance metrics while maintaining the flexibility to use any observability platform they choose.
Read more →
