Verification as the Core Agent Skill

What verification means

(07:19) Um, and I think yeah, like that's definitely fair to question. Um, but uh, yeah, I think I think if you set up your agents well, you can definitely get to a very similar level. Um, and so I'm going to talk about how we do that. Uh so for me I think I'm curious like I guess call in your experience as well but uh for me I think the most important skill that you should have in your toolbox when you work with agents is verification.

(07:55) Uh and by verification I mean the ability for an agent to actually run the code uh or take CPU traces or heap snapshots or uh you know open an iOS simulator whatever you know however your application is exposed to your users it can do the same thing and uh run it for real and actually test and verify it don't work because that's the thing that really closes the loop.

(08:27) Uh it doesn't guarantee your agent writes good code uh but it allows them to at least write correct code uh which is a big a really big step forward for being able to trust your agents. Um I will I can share one example that we have uh within cursor. Uh oops where let me open this. Let's make this uh make me full screen.

There you go.

Run the application
               │
               ▼
Observe real behavior
               │
               ▼
Collect traces and evidence
               │
               ▼
Evaluate, fix, and verify again

The first week at Cursor

(09:05) Uh so for the for cursor's agent window uh so this is actually an interesting story but uh when I joined cursor 5 months ago uh they're actually uh well I was supposed to join a different team I was supposed to join like the cloud agents team uh but then since I have a lot of experience working on react and agents window is a react application uh I was uh I was asked to basically help out with uh the agent window work.

(09:41) Uh but um there wasn't really a lot of like skills to help me. So I just found myself like okay uh agent window is going to launch in like a week, right? we have a really tight deadline and um there was uh you know I was just sitting there like okay I'm going to open up the performant the the Chrome dev tools and just like take a trace look at it myself and try to make sense of this flame graph and keep in mind I was just like in my first week so I had no idea what I was looking at no idea what you know I mean I had some (10:10) idea but you know the codebase was completely fresh to me uh and I realized like my agent had no idea either you know like I would take a screenshot of the tree.

I would download trades. I would send it to it and it be like, "Yeah, it kind of looks like this, you know, uh, and it would like confidently state like it's this thing.

When the human is the verifier

(10:27) " And then I'd try to fix that. And turns out that's not the actual thing. So, this was very very slow process. And if you've ever done any like performance work yourself or, you know, just even development with an agent where you don't have a verification skill, you are the verifier, right?

You you're the bottleneck. (10:47) you you you tell your agent to do something and then it goes off and write some code. Then you open up your you know local dev build and then you start to say oh you know doesn't work. Then you got to copy paste screen uh you know screenshots or console errors or whatever.

Uh and then your agent like slowly kind of like uh you know works with that and then tries to understand it and um fix the thing but then you're constantly just in the loop and and being a bottleneck. (11:14) So there's really no way to parallelize. So the control glass skills are like one of the first skills I built uh for cursor.

Uh and glass by the way is the code name for agents window that we use internally but it's just cursor I guess. Um and so this skill uh is I guess the the the code itself is not super interesting. Your agent can very easily make one for you.