harvey-labs
An open-source benchmark that measures how well AI agents handle realistic legal work. It provides tasks and evaluation for improving legal agents.
Pick a small issue you can reproduce, then fix it with your AI coding agent.
Share on XLicense: MIT
Overview
Harvey LAB, short for Legal Agent Benchmark, is an open-source project from Harvey AI for measuring how well LLM agents handle realistic legal work. It has two parts: a dataset of tasks with instructions, documents and rubrics, and an execution harness that runs agents on those tasks and scores them. A tutorial walks through one M&A data-room assignment from setup to comparison dashboards.
Key features
- Task dataset with instructions, documents and rubrics
- Execution harness for running and evaluating agents
- All-pass rubric scoring with an LLM judge
- Reports and comparison dashboards
- Adapters for adding models and tasks
Best for
Researchers and teams building or comparing AI agents for legal work who need a shared test set. The README says the task set and harness are still being extended.
- Upstream
- harveyai/harvey-labs
- Fork on GitHub
- Guo-astro/harvey-labs
- Upstream stars
- 1.6k
- Category
- AI agents and LLM tools
- Language
- Python
- License
- MIT
- Forked
- 2026-09-28
- Sync status
- In syncLast synced 2026-11-09
More in AI agents and LLM tools
/last30days is an AI-agent skill for researching recent discussions across social platforms, video, news, markets, and the web. It brings scattered signals together in a synthesized brief for trend research and preparation.
Forked 2026-10-22Last synced 2026-11-09License: MITAI agents and LLM toolsGitHub
A portable C99 inference engine for running Kimi K3 on a CPU without a GPU or machine-learning framework. It streams the large checkpoint from disk to support inference with limited RAM, for batch text generation or text chat.
Forked 2026-10-21Last synced 2026-11-09License: Apache-2.0AI agents and LLM toolsGitHub
LayerFS gives AI agents writable workspaces over shared history, so they can work in parallel without copying the base. Use it to checkpoint, branch, rewind, and merge agent changes.
Forked 2026-10-21Last synced 2026-11-09License: MITAI agents and LLM toolsGitHub
Magnitude runs open models locally and tunes kernels for your hardware. Use it to connect local models to supported agents or an OpenAI-compatible client.
Forked 2026-10-19Last synced 2026-11-09License: Apache-2.0AI agents and LLM toolsGitHub