Braintrust CEO on Where Engineering Actually Matters in AI

About this video

Martin Casado speaks with Ankur Goyal, founder and CEO of Braintrust, about where engineering actually matters in AI and where it doesn't. They cover the open source vs closed source model cycle, why Chinese models are gaining ground faster than spending suggests, whether AI demand will eventually saturate, and the Bash vs SQL benchmark that challenges the "just give it a computer" approach to agents. 00:00 – Introduction 03:10 — What 'Evals' Actually Mean (Hypotheses and the Scientific Method) 07:34 — AI Is Continuous, Systems Are Discrete 08:29 — The Bitter Lesson: Universal Function Approximators vs. Engineering 11:58 — Engineering the Harness, Not the Model 17:47 — Chinese Models: High Token Usage, Low Dollar Share 20:37 — Why Open Source Models Aren't Dominating Yet 27:08 — Frontier Labs, Infinite Capital, and Scaling Limits 30:38 — Demand-Side Saturation: Enterprises Can't Absorb Better Models 38:47 — Bash vs. SQL for Agents: "The Results Are Comical" Follow Martin Casado on X: https://twitter.com/martin_casado Follow Ankur Goyal on X: https://twitter.com/ankrgyl Stay Updated: If you enjoyed this episode, be sure to like, subscribe, and share with your friends! Find a16z on X: https://twitter.com/a16z Find a16z on LinkedIn: https://www.linkedin.com/company/a16z Listen to the a16z Show on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX Listen to the a16z Show on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711 Follow our host: https://x.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see http://a16z.com/disclosures.