Research for dependable enterprise AI

AI assistants rarely arrive at a satisfactory solution in one prompt; does this get better after multiple attempts? Sometimes yes, but sometimes it actually gets worse. An AI bug fixer can even mistakenly try to fix an already perfect solution, and in the process, start adding bugs. We find that LLMs do successfully fix bugs, but also frequently damage correct code by hallucinating pseudo-bugs, so do more harm than good overall.

By reframing redaction as a reversible state, rather than a destructive action, we are able to use language models to reason about sensitive data without compromising privacy.

AI benchmarks were meant to measure progress — but are they actually stalling it? We break down how chasing scores can derail real advances in intelligence, and what the ARC Prize gets right.