Opik
Verifiedby Comet ML
Traces LLM and agent runs, scores them with LLM-as-a-judge and heuristic metrics, and monitors production quality via Python and TypeScript SDKs and 60+ integrations.
- Access
- APIMCPCLI
- Auth
- API Key
- Skills
- 5
Skills
Agent Trace Logging
Captures trace trees of LLM calls, tool executions, and retrieval steps via a track decorator or framework integrations.
LLM As Judge Metrics
Scores outputs for hallucination, moderation, answer relevance, and context precision with built-in or custom judge metrics.
Datasets And Experiments
Evaluates applications against versioned datasets as experiments and compares the results in the Opik dashboard.
Production Monitoring Rules
Applies online evaluation rules to live traces and charts feedback scores, trace counts, and token usage over time.
PyTest CI Evaluation
Tests LLM pipelines on every commit through the PyTest integration so evaluation results are recorded per CI run.
Related Agents
Braintrust
by Braintrust Data
Evaluates and observes LLM applications end to end: capture traces, score them with evals, iterate in a prompt…
Arize Phoenix
by Arize AI
Traces, evaluates and debugs LLM and agent applications from an open-source observability platform built on…
Langfuse
by Langfuse
Open-source LLM observability and analytics. Trace, evaluate, and debug LLM apps with prompt versioning, score…
OpenLLMetry
by Traceloop
Instruments LLM and agent applications with OpenTelemetry-native tracing, exporting spans for prompts, tool calls, and…