harvey-labs
An open-source benchmark that measures how well AI agents handle realistic legal work. It provides tasks and evaluation for improving legal agents.
Pick a small issue you can reproduce, then fix it with your AI coding agent.
Share on XLicense: MIT
Overview
Harvey LAB, short for Legal Agent Benchmark, is an open-source project from Harvey AI for measuring how well LLM agents handle realistic legal work. It has two parts: a dataset of tasks with instructions, documents and rubrics, and an execution harness that runs agents on those tasks and scores them. A tutorial walks through one M&A data-room assignment from setup to comparison dashboards.
Key features
- Task dataset with instructions, documents and rubrics
- Execution harness for running and evaluating agents
- All-pass rubric scoring with an LLM judge
- Reports and comparison dashboards
- Adapters for adding models and tasks
Best for
Researchers and teams building or comparing AI agents for legal work who need a shared test set. The README says the task set and harness are still being extended.
- Upstream
- harveyai/harvey-labs
- Fork on GitHub
- Guo-astro/harvey-labs
- Upstream stars
- 1.4k
- Category
- AI agents and LLM tools
- Language
- Python
- License
- MIT
- Forked
- 2026-09-28
- Sync status
- In syncLast synced 2026-10-19
More in AI agents and LLM tools
A collection of AI research skills and guides that helps researchers with idea evaluation, literature research, paper writing, and pre-submission review.
Forked 2026-10-18Last synced 2026-10-19Custom license (see repository)AI agents and LLM toolsGitHub
RLVE supports language model reinforcement learning with adaptive, verifiable environments for generating problems and evaluating model outputs.
Forked 2026-10-17Last synced 2026-10-19License: MITAI agents and LLM toolsGitHub
Provides modular Android development instructions for AI agents, helping them follow specialized workflows. Install individual skills or the full set with Android CLI.
Forked 2026-10-16Last synced 2026-10-19License: Apache-2.0AI agents and LLM toolsGitHub
Pi Agent is a local desktop app for using the Pi AI coding agent on macOS, Windows, and Linux. It helps you manage coding sessions, work with project files, and run saved prompts without setting up a terminal environment.
Forked 2026-10-15Last synced 2026-10-19License: MITAI agents and LLM toolsGitHub