Guardrails

​

What Are Guardrails?

• Prevent harmful outputs - Block inappropriate, biased, or unsafe content before it reaches users
• Enforce business policies - Ensure responses comply with company guidelines and regulatory requirements
• Validate quality - Check for hallucinations, factual accuracy, and relevance in real-time
• Control behavior - Enforce tone, style, and format requirements consistently
• Protect sensitive data - Detect and prevent leakage of PII, credentials, or confidential information

​

How Guardrails Differ from Other Evaluators

​

Use Cases

​

Safety and Content Filtering

• Detect hate speech, profanity, or offensive language
• Block outputs containing violent or explicit content
• Filter responses that could cause psychological harm

​

Regulatory Compliance

• HIPAA compliance for medical information
• GDPR compliance for personal data handling
• Financial services regulations (e.g., avoiding financial advice)
• Industry-specific content guidelines

​

Data Protection

• Detect PII (personally identifiable information)
• Block API keys, passwords, or credentials in responses
• Prevent disclosure of proprietary business information
• Ensure customer data confidentiality

​

Quality Assurance

• Detect hallucinations and factual errors
• Verify response relevance to user queries
• Enforce minimum quality thresholds
• Validate structured output formats

​

Brand and Tone Control

• Enforce communication style guidelines
• Maintain appropriate tone for audience
• Prevent off-brand language or messaging
• Control formality levels

​

Implementation

​

Basic Setup

from traceloop.sdk import Traceloop

Traceloop.init(app_name="your-app-name")

​

Using the @guardrail Decorator

from traceloop.sdk.decorators import guardrail
from openai import AsyncOpenAI

client = AsyncOpenAI()

@guardrail(slug="content_safety_check")
async def get_ai_response(user_message: str) -> str:
    response = await client.chat.completions.create(
        model="gpt-4",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": user_message}
        ],
        temperature=0.7
    )
    return response.choices[0].message.content

​

Medical Chat Example

import asyncio
import os
from openai import AsyncOpenAI
from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import guardrail

Traceloop.init(app_name="medical-chat-example")

client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))

@guardrail(slug="valid_medical_chat")
async def get_doctor_response(conversation_history: list) -> str:
    response = await client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": """You are a medical information assistant.
                You can provide general health information but you are NOT
                a replacement for professional medical advice.
                Always recommend consulting with qualified healthcare providers
                for specific medical concerns."""
            },
            *conversation_history
        ],
        temperature=0,
        max_tokens=500
    )
    return response.choices[0].message.content

async def medical_chat_session():
    conversation_history = []

    print("Medical Chat Assistant (type 'quit' to exit)")
    print("-" * 50)

    while True:
        user_input = input("\nYou: ").strip()

        if user_input.lower() in ['quit', 'exit', 'q']:
            print("Thank you for using Medical Chat Assistant. Stay healthy!")
            break

        conversation_history.append({"role": "user", "content": user_input})

        try:
            response = await get_doctor_response(conversation_history)
            print(f"\nAssistant: {response}")
            conversation_history.append({"role": "assistant", "content": response})
        except Exception as e:
            print(f"Error: {e}")
            conversation_history.pop()

if __name__ == "__main__":
    asyncio.run(medical_chat_session())

​

Multiple Guardrails

@guardrail(slug="content_safety")
@guardrail(slug="pii_detection")
@guardrail(slug="factual_accuracy")
async def generate_response(prompt: str) -> str:
    # Your LLM call here
    pass

​

Creating Guardrail Evaluators

1. Navigate to the Evaluator Library in your Traceloop dashboard
2. Click New Evaluator or select a pre-built evaluator
3. Define your evaluation criteria:
   • For safety checks: Specify content categories to detect and block
   • For compliance: Define regulatory requirements and policies
   • For quality: Set thresholds for relevance, accuracy, or completeness
4. Test the evaluator in a playground to validate behavior
5. Note the evaluator’s slug for use in your code
6. Apply the evaluator using @guardrail(slug="your-evaluator-slug")

​

Best Practices

​

Performance Considerations

• Use selectively - Apply guardrails only where needed, not to every function
• Choose efficient evaluators - Simpler checks run faster than complex LLM-based evaluations
• Consider async execution - Use async/await patterns to maximize throughput
• Monitor latency - Track guardrail execution times and optimize slow evaluators
• Cache when possible - Cache evaluation results for identical inputs

​

Error Handling

from traceloop.sdk.decorators import guardrail

@guardrail(slug="safety_check")
async def get_response(prompt: str) -> str:
    try:
        # Your LLM call
        response = await generate_llm_response(prompt)
        return response
    except Exception as e:
        # Log the error
        logger.error(f"Guardrail or LLM error: {e}")
        # Return safe fallback
        return "I apologize, but I cannot process this request at the moment."

​

Layered Protection

1. Input validation - Check user inputs before processing
2. Output validation - Verify LLM responses before returning
3. Context validation - Ensure proper use of retrieved information
4. Post-processing - Final safety check on formatted outputs

​

Testing Guardrails

• Test in playgrounds - Validate evaluator behavior with sample inputs
• Run experiments - Test guardrails against diverse datasets
• Monitor false positives - Track blocked outputs that should have been allowed
• Monitor false negatives - Watch for policy violations that weren’t caught
• A/B test - Compare user experience with and without specific guardrails

​

Compliance and Auditing

• Log all evaluations - Traceloop automatically tracks all guardrail executions
• Document policies - Maintain clear documentation of what each guardrail checks
• Version control - Track changes to guardrail configurations over time
• Regular audits - Review guardrail effectiveness and update as needed
• Incident response - Have procedures for when guardrails detect violations

​

Configuration Options

@guardrail(
    slug="safety_check",
    # Additional configuration options
    blocking=True,        # Whether to block on evaluation failure
    timeout_ms=5000,      # Maximum evaluation time
    fallback="safe"       # Behavior on timeout or error
)
async def get_response(prompt: str) -> str:
    # Your implementation
    pass

​

Monitoring Guardrail Performance

• Execution frequency - How often each guardrail runs
• Block rate - Percentage of requests blocked by guardrails
• Latency impact - Time added by guardrail evaluation
• Error rate - Guardrail failures or timeouts
• Policy violations - Trends in detected issues over time

​

Integration with Experiments and Monitors

• Experiments - Test guardrail effectiveness on historical data before deployment
• Monitors - Continuously track guardrail performance in production
• Playgrounds - Develop and refine guardrail evaluators interactively

​

Next Steps

• Create custom evaluators for your specific guardrail needs
• Explore pre-built evaluators for common safety and quality checks
• Set up experiments to test guardrails before production
• Configure monitors to track guardrail performance over time