Skip to content
← Back to workCase study 01 / 03 · Protected

AI Agent Training: when should the agent act on its own?

Year
2025
Company
Meta
Role
Lead product designer
Key details
Human oversight, AI systems, Agent training, Craft, Model behavior
Snapshot

I set out to simplify the interface. The interface wasn’t the problem.

4×
Completion of started corrections
6 weeks
To rethink and rebuild the experience
3
Spun out projects

Businesses could teach their AI agent when it got something wrong.

The problem was that most never made it to the end. They’d start a correction and drop off before saving, so the changes they wanted to make never reached their agent.

I started by redesigning how businesses corrected their agent. I built a working prototype connected to a live LLM and tested it with real businesses. It showed that a better interface wasn’t enough.

So I went beneath the UI and rethought the agent’s behavior: what it could handle on its own, and when it needed a person involved.

Businesses finally finished what they started. They could correct their agent and clearly see what it had learned, with completion rising 4×.

The work also uncovered three new opportunities that moved onto the roadmap.

Joined live customer calls to understand where businesses were getting stuck
Redesigned the correction experience around what I learned
Built and tested a working prototype connected to a live LLM
Dug beneath the UI to understand what was making the experience so complex
Worked with research science, PM, and engineering to rethink the agent’s behavior
- Designed the shipped experience and how the agent communicates what it learned
Protected case study

Enter a passcode to keep reading.