Promptfoo
- Publisher: Promptfoo, Inc.
- Status:
active - Version:
0.118.2 - Release Date:
2025-09-05 - Date Added:
2025-09-08 - Source URL: https://github.com/promptfoo/promptfoo
Summary
An open-source CLI and library for testing LLM applications. Promptfoo enables automated red teaming, security scans, and prompt evaluations, all configurable via YAML and integrable into CI/CD pipelines. It runs fully locally for maximum data privacy.
Key Takeaways
- Automates red teaming and vulnerability testing for LLMs.
- Detects prompt injections, jailbreaks, leaks, bias, and toxicity.
- Provides 30+ prebuilt attack types plus extensible plugins.
- Runs 100% locally with no external dependencies.
- Supports multiple providers (OpenAI, Anthropic, Hugging Face, local models).
- Configurable test scenarios via YAML or API.
- Web UI for easy setup
- Generates detailed evaluation reports and metrics, dashboard.
- Integrates easily into CI/CD and DevSecOps pipelines.
- Tests align with major threat models and security frameworks.
Related Code
- TBD
Additional Sources
Tags
red-teaming, llm, agentic-ai, devsecops, evaluation, ci/cd, automation, jailbreak, prompt-injection, bias, data-leakage
License
MIT