What is Respan?

Respan is an LLM engineering platform that unifies observability, evaluations, prompt optimization, and a unified LLM gateway in one place. It allows teams to route LLM traffic through a single endpoint to access 1000+ models, with automatic fallbacks, caching, and spend limits. The platform also provides tracing, monitoring, and evaluation tools to help developers debug, optimize, and ship AI agents reliably.

How to use Respan?

To get started, create an account on Respan and generate an API key. Then, install the Respan SDK (Python or TypeScript) and initialize it to auto-instrument supported LLM providers. For the gateway, point your existing LLM SDK to the Respan endpoint (https://api.respan.ai/api/) and use your API key. You can also use the CLI to set up the gateway automatically. For evaluations, create prompts, datasets, and evaluators, then run experiments to compare versions and iterate.

Core features of Respan

Unified LLM gateway with 1000+ models, automatic fallbacks, caching, and spend limits
Tracing for every LLM call, tool run, retrieval, and agent turn
Monitoring dashboards for requests, errors, cost, latency, and tokens with alerts
Evaluations with LLM judges, code checks, and human review
Prompt management with versioning, deployment, and testing
Integrations with popular frameworks (LangChain, LlamaIndex, Vercel AI SDK, etc.)