Introduction

Carrot is an LLM platform that helps you go from off-the-shelf models to custom models optimized for your specific use case.

The workflow

1. Trace — Install the Python SDK or add a header to your existing calls. Carrot automatically captures every LLM interaction — inputs, outputs, token usage, and latency.
2. Build — Once enough data is collected, Carrot Labs builds a custom model optimized for your workload.
3. Deploy — Your custom model is served through the same OpenAI-compatible API. Switch to it by changing the model name — no code changes needed.

What you get

• Observability — Full visibility into every LLM call across your application. Browse traces, filter by model or status, and debug issues in the dashboard.
• Custom models — Purpose-built models trained on your real production data, delivering better quality at lower cost.
• OpenAI-compatible API — Use the standard OpenAI SDK. Change base_url and model, and everything works.
• Zero-overhead tracing — The SDK captures traces in the background with no impact on your application's latency.
• Streaming support — Full streaming support for both tracing and inference.

Next steps