a meaningful benchmark for ab-lab
I’ve added a new benchmark to ab-lab that reflects my personal experience: regular tasks, even with frontier models, often need an external reviewer who doesn’t have the author’s context.
I’ve added a new benchmark to ab-lab that reflects my personal experience: regular tasks, even with frontier models, often need an external reviewer who doesn’t have the author’s context.
I just released ab-lab. It is a Python tool that lets you run benches with a specific model/harness/setting configuration.