Caustic
Physics problems that frontier AI models fail. Every problem ships with a verified answer and diagnosed failure modes.
Evaluation and RL dataFrontier Physics
In optics, a caustic is the curve where light folds and concentrates, and where the simple ray approximation stops working.
Reasoning fails the same way. A model reaches for a standard technique that works almost everywhere, and on a narrow set of problems that technique returns an answer which is fluent, well-formed, and wrong.
Caustic is that set, collected. Each problem is drawn from frontier physics research, fully self-contained, and resolves to a single answer that can be checked exactly.
Our dataset and approach are reviewed by physicists from top research institutions. The people building Caustic have contributed to benchmarks adopted by leading AI labs, and that experience shapes how each problem is designed, specified, and verified.
Built to push the boundary
Question
Self-contained. Every definition, branch, convention, normalization, and limiting procedure is stated.
Golden answer
A complete derivation, worked step by step, ending in a single verifiable mathematical object.
Failed derivation
The full attempt from the model that got it wrong, recorded verbatim, with its errors left intact.
Failure mode diagnosis
The exact step where the reasoning left the correct path, and what the divergence cost.
Request access
Licensed by agreement, non-exclusive by default with exclusive terms available. Write to us for a sample record.