Agentic 101 - Washington, DC (State of the Stack) · · Washington, DC ·
Your Agent Works. Is It Suitable for Your Setting?
Abstract
A major roadblock to enterprise adoption of AI is knowing whether a given system will actually be suitable for a specific deployment. The setting in which AI systems are deployed plays a bigger role than the model you choose. Yet nearly every metric and approach that enterprises rely on—such as evaluations, benchmarks, red teaming, and engagement—is designed around the technology, not the setting. I'll describe this disconnect, why it isn't new to AI, and what it would take for enterprises and the startups selling to them to close that distance.
Speaker
Reva Schwartz
Reva Schwartz is a research scientist and linguist, and co-founder of Civitaas Insights, which builds tooling to help organizations determine whether their AI deployments meet the demands of their setting. She also directs FRAME (Forum for Real-World AI Measurement and Evaluation) at Virginia State University's Center for Responsible AI. Reva spent 20+ years in U.S. federal service. She most recently led national AI evaluation efforts at NIST, where she founded the ARIA program, led the institute's work on harmful bias in AI, and served as lead architect of the AI Risk Management Framework Playbook.