switchboard
D

DeepEval

by Confident AI

Unit-tests LLM and agent outputs in a pytest-style workflow, scoring answer relevancy, faithfulness, hallucination and tool correctness in CI.

3
Skills
None
Auth
No
Streaming
No
Push

Skills

Evaluation Metric Suite

Scores outputs on faithfulness, relevancy, hallucination and bias using research-backed metric implementations.

Pytest-Style Assertions

Wraps evaluations as familiar unit tests so quality checks run in existing CI pipelines and fail loudly.

Agent Trace Evaluation

Evaluates multi-step agent traces including tool selection and task completion, not just the final answer.

Code & DevToolsData & Analyticsllm-evaluationpytest-integrationhallucination-metricsrag-metricsregression-testingtool-correctness
Visit Agent

Related Agents

A

Armature

Product analytics and evaluation for agent sessions running against MCP servers, tracking tool calls, failures, and per…

Data & AnalyticsCode & DevTools
APIMCP
0 skillsAPI Key
C

Crawl4AI

Crawls and scrapes web pages into clean Markdown for RAG pipelines, with CSS, XPath, and LLM-driven structured extracti…

Browser & Computer UseResearch & Knowledge
APICLIMCP
5 skillsNone
d

dbt MCP

Exposes the dbt Semantic Layer, Discovery API, and dbt CLI over MCP so agents query governed metrics and inspect model…

Data & AnalyticsCode & DevTools
MCPCLI
3 skillsBearer Token
O

OpenSandbox

Runs AI-agent workloads in isolated Docker or Kubernetes sandboxes, exposing sandbox lifecycle, command, filesystem, an…

Infrastructure & OpsCode & DevTools
APIMCPCLI
5 skillsAPI Key
deepeval
Unit-tests LLM and agent outputs in a pytest-style workflow, scoring answer relevancy, faithfulness, hallucination and tool correctness in CI.
fields
nameDeepEval
providerConfident AI
urlhttps://github.com/confident-ai/deepeval
categoriescode-devtools · data-analytics
accesscli · api
authnone
streamingfalse
pushfalse
verifiedtrue
tagsllm-evaluation, pytest-integration, hallucination-metrics, rag-metrics, regression-testing, tool-correctness
skills
metric-suiteEvaluation Metric SuiteScores outputs on faithfulness, relevancy, hallucinatio…
pytest-testingPytest-Style AssertionsWraps evaluations as familiar unit tests so quality che…
agent-tracingAgent Trace EvaluationEvaluates multi-step agent traces including tool select…
→ https://github.com/confident-ai/deepeval