Inspect AI

Inspect 是英国 AI 安全研究院开源的评测框架，专门用来跑语言模型的系统性测试：一组 dataset、一个 solver、一个 scorer 就能定义一项评测，多轮任务、工具调用、多模型对比都在同一套接口里。自带一批现成的安全与能力评测，也能接第三方基准；跑完出可复现的日志，样本级的输入输出都留着，写报告时不用重跑。文档里有从零写第一项评测的教程和常见坑。 ⭐ GitHub: UKGovernmentBEIS/inspect_ai — 3.1k stars 🔗 文档: https://inspect.aisi.org.uk/

Inspect is the open-source evaluation framework from the UK AI Safety Institute, built for systematic testing of language models: a dataset, a solver and a scorer define one evaluation, and multi-turn tasks, tool use and cross-model comparison all sit behind the same interface. It ships a set of safety and capability evals and takes third-party benchmarks; runs produce reproducible logs with sample-level inputs and outputs, so reports do not need re-running. The docs walk through writing a first eval from scratch and list the usual traps. ⭐ GitHub: UKGovernmentBEIS/inspect_ai — 3.1k stars 🔗 Docs: https://inspect.aisi.org.uk/