Llm Eval Projects

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

The LLM Evaluation Framework

AI Observability & Evaluation

A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

🐢 Open-Source Evaluation & Testing library for LLM Agents

the LLM vulnerability scanner

Fast, flexible LLM inference

The platform for LLM evaluations and AI agent testing

Evaluation and Tracking for LLM Experiments and AI Agents

A framework for few-shot evaluation of language models.

Inspect: A framework for large language model evaluations